Agenta

Agenta is an open-source platform designed specifically for teams building applications powered by large language models (LLMs). It addresses the common challenges that arise when developing, testing, and deploying LLM-based features, particularly those related to prompt management, evaluation, and observability. The platform provides a centralized workspace where team members—including product managers, developers, and domain experts—can collaboratively manage prompts. Instead of scattering prompts across emails, Slack messages, or spreadsheets, Agenta offers version-controlled prompt storage with a clear history of changes. This allows teams to track which prompt variations were used in different experiments and roll back if needed. A core component is the Playground, an interactive environment for prompt engineering. Users can iterate on prompts in real time, compare multiple variants side by side, and test them against different LLM models or parameters. The Playground supports both manual and automated testing, helping teams quickly find the most effective phrasing, temperature settings, or system instructions for their use case. Agenta’s evaluation module enables systematic comparison of prompt outputs. Users can define custom evaluation criteria—such as correctness, relevance, or safety—and run automated tests across many prompt versions. The system generates detailed reports showing how each prompt performs, making it possible to objectively measure improvements or regressions. This is especially valuable for tasks like classification, summarization, or question answering, where subtle prompt changes can lead to large differences in output quality. Observability is another key feature. When a prompt goes into production, Agenta logs every request and response, a