Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models

Devs

Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models | Read Paper on Bytez