OpenLIT

OpenLIT is an open-source observability platform specifically designed for monitoring, debugging, and improving large language model (LLM) applications. Built on the OpenTelemetry standard, it provides full visibility into the lifecycle of AI interactions by capturing traces, metrics, and logs from LLM calls, vector databases, and the underlying infrastructure. The platform is free to self-host under the Apache 2.0 license, making it accessible for teams of any size to deploy in production environments without licensing costs. At its core, OpenLIT offers distributed tracing that visualizes request flows across multiple components—from the user query through the orchestration layer, the LLM provider request, and any external tool calls or retrieval-augmented generation (RAG) pipelines. Each trace captures timing, token counts, cost estimates, and error details, allowing engineers to pinpoint bottlenecks, identify failed steps, and understand the complete path of each AI interaction. This real-time monitoring helps detect issues like slow response times, unexpected API failures, or abnormal token consumption that could indicate model drift or misconfigured prompts. The platform includes an evaluation module for systematically assessing LLM outputs. Teams can define custom metrics—such as relevance, factual accuracy, or adherence to system instructions—and run evaluations across different models, prompts, or versions. Results are aggregated into dashboards that track performance trends over time, enabling data-driven decisions about model selection, prompt engineering, and iterative improvements. This evaluation capability is critical for production deployments where consistent output quality is required. A Prompt Hub component allows teams to store, version, and manage pr