PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling

Devs

PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling | Read Paper on Bytez