Back to Home
MarkTechPost··Papers & Tech

NVIDIA Shows 3 Ways to Speed Up MoE Inference on DGX Spark, Lifting Ling-3.0-flash Throughput 42.5%

中文摘要

3种优化使DGX Spark上MoE推理提速42.5%,Ling-3.0-flash性能显著提升。

English Summary

3 optimizations boost MoE inference on DGX Spark by 42.5%, significantly improving Ling-3.0-flash performance.

Original Excerpt

NVIDIA shows how 3 stacked optimizations speed up MoE inference on a single DGX Spark. On Ant Group inclusionAI's 124B Ling-3.0-flash, GSM8K decode throughput rose 42.5% with no detectable accuracy change.