SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training

Devs

SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training | Read Paper on Bytez