Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts

Devs

Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts | Read Paper on Bytez