Back to Home
vLLM Releases··Papers & Tech

v0.32.0

中文摘要

支持结构化决策和通用解析器,v0.32.0 引入新 API 处理多种问题类型。

English Summary

v0.32.0 release introduces a new /v1/systemone endpoint for structured decisions and generic parsing, answering choice, noul, and score questions.

Original Excerpt

v0.32.0 Highlights This release features 814 commits from 363 contributors (121 new)! Structured decisions and generic parsers: the new POST /v1/systemone endpoint answers choice, noul and score questions over a structured state (#59299), is served by default (#60677, #60696), and also runs on Cloudflare/clef-omni (#61064), LiquidAI/d1-3B (#60676) and DiffusionGemma (#60466). The new hf tool and reasoning parser (--tool-call-parser hf, --reasoning-parser hf) reads the checkpoint's response_template, so models that ship it need no model-specific parser (#58604, #58602), in both the Python and Rust frontends (#59005). Low-precision KV cache everywhere: NVFP4 KV cache now runs on SM8x and SM12x (#46963), SM90 (#60620) and with XQA decode on SM12x (#60452), and on MiniMax-M3 sparse attention on SM100 (#59300); GLM-5.x defaults to an FP8 KV cache on SM100, halving KV memory per rank with 2.3-5.5% more throughput at a 7-11% TTFT cost in the PR's benchmark (#60140, #60281); fp8_ds_mla on SM90 gives GLM-5.3-Flash up to 31% more output throughput (#59246); ROCm gains the 4-bit ultraquant_4bit KV cache with about 2x the density of FP8 (#57057). Model Runner V2: a GPU n-gram speculator with v…