InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management

Devs

InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management | Read Paper on Bytez