Claude Shannon Measured the English Language by Playing Hangman
中文摘要
克劳德·香农通过玩猜词游戏衡量英语熵，揭示了现代语言模型（如GPT）预测文本的理论起源。
English Summary
Claude Shannon used Hangman to measure English entropy, forming the theoretical basis for how modern language models like GPT predict text.
Original Excerpt
Entropy, cross-entropy and why GPT is still playing a guessing game invented in 1948 - the origin story of language models, explained Continue reading on Medium »