Langfuse

Langfuse is an open-source platform designed for AI engineering teams that build and maintain applications powered by large language models (LLMs). It provides a suite of integrated tools focused on observability, evaluation, and prompt management, enabling developers to debug, monitor, and improve the behavior of LLM-based systems in production and development environments. At its core, Langfuse offers tracing capabilities that capture detailed logs of every LLM call, including inputs, outputs, token usage, latency, and metadata. These traces can be ingested via native SDKs for OpenAI, LangChain, or custom integrations, and are visualized in a centralized dashboard. Developers can filter, search, and inspect individual traces to identify issues such as hallucinations, incorrect reasoning, or unexpected response formats. Beyond tracing, the platform includes a prompt management module that allows teams to version, test, and roll back prompts across different model configurations. Users can define prompt templates with variables, run side-by-side comparisons of prompt variations, and attach evaluation scores to determine which prompt yields the most accurate or safe outputs. The evaluation system supports both automated metrics (e.g., exact match, ROUGE, semantic similarity) and human feedback via custom scoring rubrics. Metrics are aggregated over time to track regressions or improvements after changes to prompts, models, or retrieval pipelines. Langfuse also provides A/B testing workflows for LLM applications. Users can run controlled experiments by splitting traffic between different model versions, prompt strategies, or retrieval configurations, then compare performance on predefined metrics such as response quality, latency, or cost. The platform integrates with ext