HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading | Read Paper on Bytez

Devs

HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading

3 months ago

·

arXiv