GitHub / inferflow / inferflow
Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
JSON API: http://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inferflow%2Finferflow
Stars: 242
Forks: 25
Open issues: 8
License: mit
Language: C++
Size: 1.89 MB
Dependencies parsed at: Pending
Created at: over 1 year ago
Updated at: 6 days ago
Pushed at: about 1 year ago
Last synced at: 1 day ago
Commit Stats
Commits: 76
Authors: 9
Mean commits per author: 8.44
Development Distribution Score: 0.329
More commit stats: https://commits.ecosyste.ms/hosts/GitHub/repositories/inferflow/inferflow
Topics: baichuan2, bloom, deepseek, falcon, gemma, internlm, llama2, llamacpp, llm-inference, m2m100, minicpm, mistral, mixtral, mixture-of-experts, model-quantization, moe, multi-gpu-inference, phi-2, qwen