Back to Home
AWS ML Blog··Official Labs

Deploy frontier AI models on AWS Trainium with SGLang-Neuron

中文摘要

SGLang-Neuron现已在AWS Trainium上架,支持使用PyTorch原生接口高效部署前沿AI模型并进行强化学习训练。

English Summary

SGLang-Neuron is now available on AWS Trainium, allowing users to deploy frontier AI models and run reinforcement learning with native PyTorch support using the familiar SGLang interface.

Original Excerpt

SGLang-Neuron is now publicly available through an AWS Deep Learning Container. With SGLang-Neuron, you can serve models and generate reinforcement learning rollouts with SGLang on AWS Trainium using the same SGLang interface you already know, all running on native PyTorch.