DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion

Devs

DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion | Read Paper on Bytez