Back to Home
SGLang Releases··Papers & Tech

v0.5.23

中文摘要

v0.5.23 更新包含 1007 个 PR,新增多种 LLM、VLM 及 Diffusion 模型,并提升了 DeepSeek-V4.1 性能。

English Summary

v0.5.23 includes 1,007 PRs, introduces new LLM, VLM, and Diffusion models, and improves DeepSeek-V4.1 performance.

Original Excerpt

Highlights 1007 PRs from 257 contributors. New models Model Type Cookbook Muse Spark Open LLM / VLM link Beam-1.0 LLM / VLM link MiniMax-M3.1-Flash LLM / VLM link MiniCPM-V 4.7 LLM / VLM link Kev LLM / VLM link Cosmos3-Nano-Sim (Bimanual, Transfer, Auto) Diffusion link FastH3 Omni Diffusion link FLUX.3 Image Klein Diffusion link HY-Nomas Diffusion link MOSS-Image Diffusion link What's new DeepSeek-V4.1 on GB300 gets up to 1.5x more prefill throughput under load and about 28% lower first-token latency on 32K to 64K prompts (#44403, #45059, #45107). SGLang low-latency mode: the opt-in NVIDIA megakernel and AMD monokernel plugins run DeepSeek-V4.1-Flash decode as fused persistent kernels, for up to 44% faster single-user decode on GB300 and 75% on MI350X (#44945, #45450, #44353, #44621). MoE models such as DeepSeek-V4.1-Flash, GLM-5.3-Flash and gpt-oss reach the first token 30% to 58% sooner on 8K prompts on H200 (#43363). Paged Experts (opt-in) runs MoE models larger than GPU memory on a single consumer GPU, up to 30x faster than CPU offload (#42557, #45787). SGLang now runs on Google TPU7x through SGLang-TorchTPU, a PyTorch-native plugin for TPU. You can now serve LoRA adapters on q…