The Next AI Lab Might Look More Like a Physics Lab
中文摘要
AI研究可能更趋向物理学，因为GPT-2、LLaMA和Mistral等模型在每个参数中均精确存储了两个比特的信息。
English Summary
AI research may shift toward physics, as models like GPT-2 and LLaMA store exactly two bits per parameter.
Original Excerpt
A model can store exactly two bits per parameter. That number holds across GPT-2, LLaMA, and Mistral, and finding it required… Continue reading on Generative AI »