GitHub topics: llm-observability
matdev83/llm-accounting
A Python package for tracking and analyzing LLM usage across different models and applications. It is primarily designed as a library for integration into development process of LLM-based agentic workflow tooling, providing robust tracking capabilities.
Language: Python - Size: 780 KB - Last synced at: about 19 hours ago - Pushed at: about 19 hours ago - Stars: 2 - Forks: 1
VoltAgent/voltagent
Open Source TypeScript AI Agent Framework
Language: TypeScript - Size: 3.55 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 1,904 - Forks: 155
lmnr-ai/lmnr
Laminar - open-source all-in-one platform for engineering AI products. Create data flywheel for your AI app. Traces, Evals, Datasets, Labels. YC S24.
Language: TypeScript - Size: 33 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 2,026 - Forks: 123
langfuse/langfuse
🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23
Language: TypeScript - Size: 22.4 MB - Last synced at: 5 days ago - Pushed at: 6 days ago - Stars: 12,014 - Forks: 1,104
comet-ml/opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Language: Python - Size: 241 MB - Last synced at: 5 days ago - Pushed at: 6 days ago - Stars: 9,278 - Forks: 629
Agenta-AI/agenta
The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.
Language: Python - Size: 171 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 2,795 - Forks: 330
Helicone/helicone
🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓
Language: TypeScript - Size: 493 MB - Last synced at: 10 days ago - Pushed at: 10 days ago - Stars: 3,857 - Forks: 381
zairu507/llm_engineering
Repo to accompany my mastering LLM engineering course
Language: Jupyter Notebook - Size: 39.6 MB - Last synced at: 20 days ago - Pushed at: 20 days ago - Stars: 0 - Forks: 0
cyberark/agentwatch
A powerful AI observability framework that provides comprehensive insights into agent interactions across platforms, enabling developers to monitor, analyze, and optimize AI-driven applications with minimal integration effort.
Language: Python - Size: 4.84 MB - Last synced at: 27 days ago - Pushed at: 27 days ago - Stars: 75 - Forks: 2
radicalbit/radicalbit-ai-monitoring
A comprehensive solution for monitoring your AI models in production
Language: Python - Size: 36.5 MB - Last synced at: 12 days ago - Pushed at: 12 days ago - Stars: 81 - Forks: 11
teilomillet/hapax
The reliability layer between your code and LLM providers.
Language: Go - Size: 11 MB - Last synced at: about 1 hour ago - Pushed at: 5 months ago - Stars: 18 - Forks: 2
palico-ai/palico-ai
Build, Improve Performance, and Productionize your LLM Application with an Integrated Framework
Language: TypeScript - Size: 13.7 MB - Last synced at: 14 days ago - Pushed at: 6 months ago - Stars: 339 - Forks: 27
langfuse/langfuse-java
🪢 Auto-generated Java Client for Langfuse API
Language: Java - Size: 308 KB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 7 - Forks: 0
langfuse/oss-llmops-stack
Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-availability, while Langfuse focuses on detailed observability, prompt versioning, and performance evaluations.
Size: 316 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 5 - Forks: 0
modelmetry/modelmetry-sdk-js
The Modelmetry JS/TS SDK allows developers to easily integrate Modelmetry’s advanced guardrails and monitoring capabilities into their LLM-powered applications.
Language: TypeScript - Size: 509 KB - Last synced at: 5 days ago - Pushed at: 4 months ago - Stars: 1 - Forks: 0
AndrMoura/streamlit-chatbot-analytics
Streamlit-based chatbot leveraging Ollama via LangChain and PostHog-LLM for advanced logging and monitoring
Language: Python - Size: 7.82 MB - Last synced at: 7 days ago - Pushed at: about 1 year ago - Stars: 6 - Forks: 0
myscale/myscale_callback
MyScale Callback integrates with LangChain Callbacks, which seamlessly captures trace data from LangChain-based LLM applications and stores it in the MyScale database.
Language: Python - Size: 22.5 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0
