Implementing the Reformer Transformer from Scratch: LSH Attention, Reversible Layers, and What the…
中文摘要
本文详解如何从零实现Reformer Transformer，通过LSH注意力机制和可逆层解决标准Transformer的内存瓶颈。
English Summary
This guide explains implementing the Reformer Transformer from scratch, using LSH Attention and Reversible Layers to overcome standard Transformer memory bottlenecks.
Original Excerpt
Standard transformers have a well-known memory problem. The attention mechanism computes relationships between every pair of tokens, which… Continue reading on Medium »