Shortcut-connected Expert Parallelism for Accelerating Mixture of Experts

Devs

Shortcut-connected Expert Parallelism for Accelerating Mixture of Experts | Read Paper on Bytez