Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference | Read Paper on Bytez