DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion | Bytez

b

DiscoverModelsSearch

DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion

7 months ago

·

arXiv