Back to Home
Towards Data Science··Industry Media

Why Temperature 0 Isn't Deterministic

中文摘要

本文解释了为何温度0不代表确定性,分析了Token翻转现象及生成内容在约百个Token后分歧的原因。

English Summary

This article explains why temperature 0 isn't deterministic in LLMs, analyzing token flipping and why completions diverge after approximately one hundred tokens.

Original Excerpt

A one-line formula for how often an LLM's top token flips, and why identical completions fall apart after about a hundred tokens