How a 1.6 Trillion Parameter Model Fits on a Laptop, With C Code You Can Run
中文摘要
一个1.6万亿参数的MoE模型可通过C代码在笔记本电脑上运行，因为其大部分参数在运行时处于休眠状态。
English Summary
A 1.6 trillion parameter MoE model can run on a laptop via C code, as most of its parameters remain inactive during execution.
Original Excerpt
A Mixture-of-Experts model keeps roughly 97 percent of itself asleep at any given moment. That one fact means models far bigger than your… Continue reading on Towards AI »