GitHub / FMInference / H2O

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

JSON API: http://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FMInference%2FH2O
PURL: pkg:github/FMInference/H2O

Stars: 462
Forks: 60
Open issues: 36

License: None
Language: Python
Size: 39.1 MB
Dependencies parsed at: Pending

Created at: about 2 years ago
Updated at: 14 days ago
Pushed at: about 1 year ago
Last synced at: about 5 hours ago

Topics: gpt-3, heavy-hitters, high-throughput, kv-cache, large-language-models, sparsity

Loading...