返回首页
Towards Data Science··行业媒体

Disaggregation Is a Thousand-GPU Problem

中文摘要

分离预填充与解码仅在大规模GPU集群下才具优势,规模较小时应首选分块预填充。

English Summary

Splitting prefill from decode only benefits thousand-GPU scale clusters; otherwise, chunked prefill is the recommended default.

原文节选

Three conditions that must hold before splitting prefill from decode pays off, and why chunked prefill is the right default below that threshold.