Back to Home
RadarAI··Papers & Tech

DeepSeek 将内部推理框架 DeepInfer 全面开源

中文摘要

DeepSeek开源高性能推理框架DeepInfer,支持分布式推理、多硬件平台及结构化输出,助力实现高吞吐、低延迟的大模型推理服务。

English Summary

DeepSeek open-sourced its high-performance inference framework, DeepInfer, supporting distributed computing, multiple hardware platforms, and advanced features for high-throughput, low-latency large model inference.

Original Excerpt

DeepSeek 宣布将内部推理框架 DeepInfer 全面开源,此前已开源大模型、TileLang 编译工具及推理算子库。DeepInfer 集成高性能算子与深度优化的推理引擎,支持分布式推理、推测解码、SWA Bounded Replay、结构化输出等能力,兼容英伟达与华为昇腾平台,已支撑 DeepSeek V4.1 Flash 高吞吐、低延迟的大规模推理服务。