Why ChatGPT Speeds Up Mid-Sentence (and Slows Down Later)
中文摘要
本文解析 ChatGPT 生成速度变化的原理，说明了首词生成慢于后续词的原因。
English Summary
This article explores why ChatGPT's generation speed varies, explaining why the first token takes longer than subsequent ones.
原文节选
I used to assume LLM speed was just “the servers are busy today.” Then I actually dug into why the first word takes a moment and the rest… Continue reading on Medium »