Trading Zeros for Geometry: How Reshaping Transformer Weights to 2:4 Structured Sparsity Halves…
中文摘要
研究通过将 Transformer 权重重构为 2:4 结构化稀疏，利用几何结构优化零值以减半计算量。
English Summary
Researchers reshape Transformer weights into 2:4 structured sparsity, using geometric patterns to replace zeros and halve computational costs.
Original Excerpt
We spent the last five years chasing ghosts inside our weight matrices. And for a long time, we were convinced we were winning. Continue reading on Medium »