GitHub topics: ai-gateway
BerriAI/litellm
Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
Language: Python - Size: 403 MB - Last synced at: 13 minutes ago - Pushed at: about 2 hours ago - Stars: 22,413 - Forks: 2,869
cloud-apim/otoroshi-llm-extension
Connect, setup, secure and seamlessly manage LLM models using an Universal/OpenAI compatible API
Language: Scala - Size: 9.19 MB - Last synced at: about 4 hours ago - Pushed at: about 4 hours ago - Stars: 9 - Forks: 1
langdb/ai-gateway
Govern, Secure, and Optimize your AI Traffic. AI Gateway provides unified interface to all LLMs using OpenAI API format with a focus on performance and reliability. Built in Rust.
Language: Rust - Size: 71.8 MB - Last synced at: about 9 hours ago - Pushed at: about 9 hours ago - Stars: 571 - Forks: 18
QuantumNous/new-api
AI模型接口管理与分发系统，支持将多种大模型转为统一格式调用，支持OpenAI、Claude等格式，可供个人或者企业内部管理与分发渠道使用，本项目基于One API二次开发。🍥 The next-generation LLM gateway and AI asset management system supports multiple languages.
Language: Go - Size: 5.29 MB - Last synced at: 1 day ago - Pushed at: 1 day ago - Stars: 7,312 - Forks: 1,426
alibaba/higress
🤖 AI Gateway | AI Native API Gateway
Language: Go - Size: 35.4 MB - Last synced at: 1 day ago - Pushed at: 3 days ago - Stars: 5,280 - Forks: 663
banku50/inference-gateway
An open-source, high-performance gateway unifying multiple LLM providers, from local solutions like Ollama to major cloud providers such as OpenAI, Groq, Cohere, Anthropic, Cloudflare and DeepSeek.
Language: Go - Size: 145 KB - Last synced at: 1 day ago - Pushed at: 2 days ago - Stars: 0 - Forks: 0
manohar9694/monkdb-mcp
An MCP server 🚀 for MonkDB, licensed under Apache 2.0 📜
Language: Python - Size: 89.8 KB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 0 - Forks: 0
Tanantor/langgate
Lightweight AI inference gateway - local model registry & parameter transformer (Python SDK) - with optional Envoy proxy processor and FastAPI registry server deployment options.
Language: Python - Size: 386 KB - Last synced at: 2 days ago - Pushed at: about 1 month ago - Stars: 4 - Forks: 0
kgateway-dev/kgateway.dev
kgateway.dev site and docs
Language: HTML - Size: 35.5 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 2 - Forks: 12
monkdbofficial/monkdb-mcp
An MCP server 🚀 for MonkDB, licensed under Apache 2.0 📜
Language: Python - Size: 90.8 KB - Last synced at: 5 days ago - Pushed at: 6 days ago - Stars: 0 - Forks: 0
Portkey-AI/gateway
A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
Language: TypeScript - Size: 60.4 MB - Last synced at: 6 days ago - Pushed at: 7 days ago - Stars: 7,776 - Forks: 578
NeuralTrust/TrustGate
A powerful, AI Gateway designed from scratch for AI
Language: Go - Size: 1.23 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 22 - Forks: 0
Portkey-AI/portkey-node-sdk
Build reliable, secure, and production-ready AI apps easily.
Language: TypeScript - Size: 6.16 MB - Last synced at: 4 days ago - Pushed at: 5 days ago - Stars: 37 - Forks: 12
Kong/kong
🦍 The Cloud-Native API Gateway and AI Gateway.
Language: Lua - Size: 90.6 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 40,752 - Forks: 4,916
NeuralTrust/trustgate-helm-charts
A Production-Grade TrustGate for Kubernetes
Language: Shell - Size: 152 KB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 0 - Forks: 0
TU-Wien-dataLAB/aqueduct
Aqueduct is a simple yet fully-featured AI gateway.
Language: Python - Size: 695 KB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 0 - Forks: 0
APIParkLab/APIPark
🦄云原生、超高性能 AI&API网关，LLM API 管理、分发系统、开放平台，支持所有AI API，不限于OpenAI、Azure、Anthropic Claude、Google Gemini、DeepSeek、字节豆包、ChatGLM、文心一言、讯飞星火、通义千问、360 智脑、腾讯混元等主流模型，统一 API 请求和返回，API申请与审批，调用统计、负载均衡、多模型灾备。一键部署，开箱即用。Cloud native, ultra-high performance AI&API gateway, LLM API management, distribution system, open platform, supporting all AI APIs.
Language: TypeScript - Size: 59.4 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 1,088 - Forks: 153
LianjiaTech/bella-openapi
Bella OpenAPI是一个提供了丰富的AI调用能力的API网关，可类比openrouter，与之不同的是除了提供聊天补全(chat-completion)能力外，还提供了文本向量化(text-embedding)、语音识别(ASR)、语音合成(TTS)、文生图、图生图等多种AI能力，同时集成了计费、限流和资源管理功能。且集成的所有能力都经过了大规模生产环境的验证。
Language: Java - Size: 2.63 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 13 - Forks: 4
envoyproxy/ai-gateway
Envoy AI Gateway is an open source project for using Envoy Gateway to handle request traffic from application clients to Generative AI services.
Language: Go - Size: 3.14 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 238 - Forks: 46
stacklok/codegate
CodeGate: Security, Workspaces and Multiplexing for AI Agentic Frameworks
Language: Python - Size: 43.7 MB - Last synced at: 9 days ago - Pushed at: 9 days ago - Stars: 643 - Forks: 76
Portkey-AI/helm
Portkey AI official helm charts for kubernetes deployment
Language: Smarty - Size: 1.03 MB - Last synced at: about 9 hours ago - Pushed at: about 9 hours ago - Stars: 4 - Forks: 2
claymore666/MCPConnectionHub
MCP Connection Hub - A unified Model Context Protocol Gateway
Language: Python - Size: 37.1 KB - Last synced at: 12 days ago - Pushed at: 12 days ago - Stars: 0 - Forks: 0
katanemo/archgw
The AI-native proxy server for agents. Arch handles the pesky low-level work in building agentic apps like calling specific tools, routing prompts to the right agents, clarifying vague inputs, unifying access and observability to any LLM, etc.
Language: Rust - Size: 18.5 MB - Last synced at: 13 days ago - Pushed at: 13 days ago - Stars: 2,486 - Forks: 135
kasuboski/openai-gateway
An OpenAI compatible API as a Cloudflare worker. Proxies to various providers through Cloudflare AI Gateway
Language: TypeScript - Size: 1.59 MB - Last synced at: 13 days ago - Pushed at: 14 days ago - Stars: 0 - Forks: 0
Portkey-AI/portkey-python-sdk
Build reliable, secure, and production-ready AI apps easily.
Language: Python - Size: 6.96 MB - Last synced at: 3 days ago - Pushed at: 3 days ago - Stars: 72 - Forks: 20
Portkey-AI/openapi Fork of openai/openai-openapi
OpenAPI specification for the Portkey API
Size: 1.21 MB - Last synced at: 18 days ago - Pushed at: 18 days ago - Stars: 4 - Forks: 1
elizabethsiegle/art-vector-search-cloudflare
Find art with AI using Cloudflare's Vector Database Vectorize, LlaVA and LlaMA on Workers AI, and more!
Language: HTML - Size: 119 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0
wx-yz/ai-gateway
Developer focused AI Gateway
Language: Ballerina - Size: 295 KB - Last synced at: about 1 month ago - Pushed at: 2 months ago - Stars: 11 - Forks: 2
Kenza-AI/sagify
LLMs and Machine Learning done easily
Language: Python - Size: 36.1 MB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 439 - Forks: 69
MagicAPI/ai-gateway
Built for demanding AI workflows, this gateway offers low-latency, provider-agnostic access, ensuring your AI applications run smoothly and quickly.
Language: Rust - Size: 129 KB - Last synced at: 2 months ago - Pushed at: 5 months ago - Stars: 34 - Forks: 7
StacklokLabs/spring-ai-codegate-sample
Experimental application that integrates Spring AI and CodeGate
Size: 753 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 3 - Forks: 0
Portkey-AI/helm-chart
Kubernetes Configs for Portkey Gateway deployment
Language: Smarty - Size: 189 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 3 - Forks: 3
langfuse/oss-llmops-stack
Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-availability, while Langfuse focuses on detailed observability, prompt versioning, and performance evaluations.
Size: 316 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 5 - Forks: 0
kdhrubo/hermes-ai-platform
Enterprise Platform for Open Source AI - Gateway, Inference, Hosting, Fine Tuning in a single box.
Language: Java - Size: 20.5 KB - Last synced at: about 2 months ago - Pushed at: 4 months ago - Stars: 1 - Forks: 1
MagicAPI/ai-gateway-ts
Built for demanding AI workflows, this gateway offers low-latency, provider-agnostic access, ensuring your AI applications run smoothly and quickly.
Language: TypeScript - Size: 56.6 KB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 0 - Forks: 0
adrianliechti/llama
Inference Hub for AI at Scale
Language: Go - Size: 1.61 MB - Last synced at: 6 months ago - Pushed at: 7 months ago - Stars: 33 - Forks: 5
lukemurraynz/AI_Gateway_Tests
Tests and Bicep related to API Management AI Gateway testing.
Language: Bicep - Size: 7.81 KB - Last synced at: 9 months ago - Pushed at: 9 months ago - Stars: 0 - Forks: 0
huhuhang/bob-plugin-cloudflare
Azure OpenAI + Cloudflare AI Gateway 进行翻译或润色
Language: JavaScript - Size: 702 KB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 4 - Forks: 0
murataltanbas/openai-assistant-web-app-cloudflare
ai assistant web app using openai assistant api + cloudflare services
Language: JavaScript - Size: 5.86 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0
