Topic: "llm-gateway"
BerriAI/litellm
Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
Language: Python - Size: 428 MB - Last synced at: 2 days ago - Pushed at: 2 days ago - Stars: 24,191 - Forks: 3,220
katanemo/archgw
The AI-native proxy server for agents. Arch handles the pesky low-level work in building agentic apps like calling specific tools, routing prompts to the right agents, clarifying vague inputs, unifying access and observability to any LLM, etc.
Language: Rust - Size: 19.2 MB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 2,716 - Forks: 152
APIParkLab/APIPark
🦄云原生、超高性能 AI&API网关，LLM API 管理、分发系统、开放平台，支持所有AI API，不限于OpenAI、Azure、Anthropic Claude、Google Gemini、DeepSeek、字节豆包、ChatGLM、文心一言、讯飞星火、通义千问、360 智脑、腾讯混元等主流模型，统一 API 请求和返回，API申请与审批，调用统计、负载均衡、多模型灾备。一键部署，开箱即用。Cloud native, ultra-high performance AI&API gateway, LLM API management, distribution system, open platform, supporting all AI APIs.
Language: TypeScript - Size: 59.5 MB - Last synced at: about 16 hours ago - Pushed at: about 17 hours ago - Stars: 1,208 - Forks: 165
langdb/ai-gateway
Govern, Secure, and Optimize your AI Traffic. AI Gateway provides unified interface to all LLMs using OpenAI API format with a focus on performance and reliability. Built in Rust.
Language: Rust - Size: 71.9 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 593 - Forks: 24
masaic-ai-platform/open-responses
Open AI platform in your local. Supports Ollama, OpenRouter, Gemini, OpenAI, Deepseek etc.
Language: Kotlin - Size: 21.3 MB - Last synced at: 16 days ago - Pushed at: 16 days ago - Stars: 84 - Forks: 6
LianjiaTech/bella-openapi
Bella OpenAPI是一个提供了丰富的AI调用能力的API网关，可类比openrouter，与之不同的是除了提供聊天补全(chat-completion)能力外，还提供了文本向量化(text-embedding)、语音识别(ASR)、语音合成(TTS)、文生图、图生图等多种AI能力，同时集成了计费、限流和资源管理功能。且集成的所有能力都经过了大规模生产环境的验证。
Language: Java - Size: 3.35 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 41 - Forks: 13
bluewave-labs/LangRoute
This is a robust and configurable LLM proxy server built with Node.js, Express, and PostgreSQL. It acts as an intermediary between your applications and various Large Language Model (LLM) providers
Language: TypeScript - Size: 17.8 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 18 - Forks: 4
teilomillet/hapax
The reliability layer between your code and LLM providers.
Language: Go - Size: 11 MB - Last synced at: 11 days ago - Pushed at: 5 months ago - Stars: 18 - Forks: 2
cloud-apim/otoroshi-llm-extension
Connect, setup, secure and seamlessly manage LLM models using an Universal/OpenAI compatible API
Language: Scala - Size: 9.65 MB - Last synced at: 17 days ago - Pushed at: 17 days ago - Stars: 9 - Forks: 1
Tanantor/langgate
Lightweight AI inference gateway - local model registry & parameter transformer (Python SDK) - with optional Envoy proxy processor and FastAPI registry server deployment options.
Language: Python - Size: 424 KB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 5 - Forks: 0
shengyanli1982/llmproxy
🧭🧭 An intelligent load balancer with smart scheduling that unifies diverse LLMs.
Language: Rust - Size: 1.42 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 5 - Forks: 0
langfuse/oss-llmops-stack
Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-availability, while Langfuse focuses on detailed observability, prompt versioning, and performance evaluations.
Size: 316 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 5 - Forks: 0
knnlabs/Conduit
A unified API gateway for multiple LLM providers with OpenAI-compatible endpoints
Language: C# - Size: 7.57 MB - Last synced at: about 23 hours ago - Pushed at: about 23 hours ago - Stars: 2 - Forks: 0
charmitro/ai-proxy
A lightweight proxy for LLM API calls with guardrails, metrics, and monitoring. A vibe coding experiment.
Language: Go - Size: 51.8 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 2 - Forks: 0
burgonet-eu/gateway
Burgonet Gateway is an enterprise LLM gateway that provides secure access and compliance controls for AI systems
Language: Rust - Size: 8.22 MB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 2 - Forks: 1
frontllm/frontllm-sdk
FrontLLM is your public gateway to LLMs. Request LLM directly from your front-end code. No backend needed.
Language: TypeScript - Size: 102 KB - Last synced at: 20 days ago - Pushed at: 20 days ago - Stars: 1 - Forks: 1
claymore666/MCPConnectionHub
MCP Connection Hub - A unified Model Context Protocol Gateway
Language: Python - Size: 37.1 KB - Last synced at: 3 days ago - Pushed at: about 2 months ago - Stars: 1 - Forks: 1
viniViado/LLMSecOps
LLMSecOps focuses on integrating security practices within the lifecycle of machine learning models. It ensures that models are robust against threats while maintaining compliance and performance standards.
Size: 461 KB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 0 - Forks: 0
2jang/ollama-chutesai-proxy
Ollama-compatible API proxy server with Chutes.ai's LLM services.
Language: Python - Size: 21.5 KB - Last synced at: 25 days ago - Pushed at: 25 days ago - Stars: 0 - Forks: 0
scampion/burgonet-gateway Fork of burgonet-eu/gateway
Burgonet Gateway is an enterprise LLM gateway that provides secure access and compliance controls for AI systems
Language: Rust - Size: 8.22 MB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 0 - Forks: 0
