GitHub / mddunlap924 / LLM-Inference-Serving

This repository demonstrates LLM execution on CPUs using packages like llamafile, emphasizing low-latency, high-throughput, and cost-effective benefits for inference and serving.

JSON API: http://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mddunlap924%2FLLM-Inference-Serving
PURL: pkg:github/mddunlap924/LLM-Inference-Serving

Topics: deepspeed, large-language-models, llamacpp, llamafile, llm-inference, llm-serving, llms, vllm

• Files
• Readme
• Dependencies