Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth

Devs

Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth | Read Paper on Bytez