KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization

Devs

KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization | Read Paper on Bytez