Why Temperature 0 Isn't Deterministic
中文摘要
本文解释了为何温度0不代表确定性,分析了Token翻转现象及生成内容在约百个Token后分歧的原因。
English Summary
This article explains why temperature 0 isn't deterministic in LLMs, analyzing token flipping and why completions diverge after approximately one hundred tokens.
原文节选
A one-line formula for how often an LLM's top token flips, and why identical completions fall apart after about a hundred tokens