Topic: "cache-eviction"
DRSY/EasyKV
Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)
Language: Python - Size: 966 KB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 63 - Forks: 4
opencoff/go-sieve
SIEVE cache - simpler than LRU
Language: Go - Size: 15.6 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 36 - Forks: 3
Related Topics