SentenceKV: Efficient LLM Inference via Sentence-Level Semantic KV Caching | Read Paper on Bytez

Devs

SentenceKV: Efficient LLM Inference via Sentence-Level Semantic KV Caching

2 months ago

·

arXiv