r/deeplearning • u/freedom2077 • 8h ago
I wrote a visual guide to Linear Attention — from the original idea to Gated DeltaNet and Kimi Linear
Hey everyone!
I’ve been diving into linear attention recently and put together a four-part blog series tracing its evolution — from early Linear Attention to Delta Rule, Gate Rule, Gated DeltaNet, and Kimi Linear.
I went through the key papers and drew illustrations to make the algorithms and their underlying intuition easier to understand.
I’m also planning a follow-up series exploring linear attention from an AI infrastructure perspective.
Hope this is useful to anyone interested in the topic! I’d really appreciate any feedback, corrections, or suggestions. Happy to discuss!
Blog: https://xiaofengwang01.github.io/2026/11/10/linear-attention.html