Topic: "cache-eviction"
DRSY/EasyKV
Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)
Language: Python - Size: 966 KB - Last synced at: 5 months ago - Pushed at: about 1 year ago - Stars: 58 - Forks: 4
opencoff/go-sieve
SIEVE cache - simpler than LRU
Language: Go - Size: 15.6 KB - Last synced at: 26 days ago - Pushed at: over 1 year ago - Stars: 30 - Forks: 2
Related Topics