Disaggregation Is a Thousand-GPU Problem
中文摘要
分离预填充与解码仅在大规模GPU集群下才具优势，规模较小时应首选分块预填充。
English Summary
Splitting prefill from decode only benefits thousand-GPU scale clusters; otherwise, chunked prefill is the recommended default.
Original Excerpt
Three conditions that must hold before splitting prefill from decode pays off, and why chunked prefill is the right default below that threshold. The post Disaggregation Is a Thousand-GPU Problem appeared first on Towards Data Science.