MoE Isn’t Free: The Hidden Cost of Moving Tokens Across GPUs
中文摘要
MoE模型并非免费，跨GPU传输Token会带来隐藏的通信成本。
English Summary
MoE models face hidden costs due to the overhead of moving tokens across different GPUs.
Original Excerpt
Your router can choose the perfect expert. But what happens when that expert lives on another machine? Continue reading on Medium »