QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache

Devs

QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache | Read Paper on Bytez