Together AI Releases Together Compiler: Agents Make Inference Up to 2.6x Faster
中文摘要
Together AI发布Together Compiler,利用智能体优化推理,使速度提升高达2.6倍。
English Summary
Together AI released Together Compiler, using agent-driven optimization to boost inference speeds by up to 2.6x over InferenceX configurations.
Original Excerpt
Together AI released Together Compiler, an agent-driven loop that keeps re-optimizing the inference forward pass as traffic shifts. It beat the best InferenceX configurations by up to 2.6x TPS in under 24 hours per model.