Why ChatGPT Speeds Up Mid-Sentence (and Slows Down Later)
中文摘要
本文解析 ChatGPT 生成速度变化的原理，说明了首词生成慢于后续词的原因。
English Summary
This article explores why ChatGPT's generation speed varies, explaining why the first token takes longer than subsequent ones.
Original Excerpt
I used to assume LLM speed was just “the servers are busy today.” Then I actually dug into why the first word takes a moment and the rest… Continue reading on Medium »