How Much Does It Actually Cost to Run a Local LLM? (Euros per Million Tokens, Measured)
中文摘要
该研究测量了在RTX 3090上运行本地大模型的电费，发现运行成本与模型大小并不完全成正比。
English Summary
A study measured GPU electricity costs for local LLMs on an RTX 3090, revealing that costs don't strictly correlate with model size.
原文节选
I measured the actual GPU electricity for eight local models on one RTX 3090 — and the cheapest wasn't the smallest, nor the priciest the biggest. The post How Much Does It Actually Cost to Run a Local LLM? (Euros per Million Tokens, Measured) appeared first on Towards Data Science.