Observability & evals

Open-source CLI and library for evaluating and red-teaming LLM apps and agents

Open-source LLM evaluation framework, similar to Pytest, for unit testing LLM apps and agents

Open-source platform to trace, test, evaluate and govern every LLM call and AI agent in production

Open-source, OpenTelemetry-native evals and tracing for AI agents

Open-source framework for large language model evaluations, built by the UK AI Security Institute

Toolkit for evaluating LLM applications with objective metrics, test data generation and feedback loops

Test, monitor and self-improve voice agents with simulation, red teaming and production monitoring

Deployment-readiness platform for voice AI agents: simulate calls, evaluate production conversations and add human review

AI governance platform to discover, test, monitor and govern AI systems and enforce controls

Simulation platform for building frontier AI agents, testing them on realistic scenarios with evals

Trust platform for voice and chat agents: pre-launch testing, red teaming and production monitoring

Open-source workspace where teams build AI agents by chatting and run them in Slack, WhatsApp or Telegram

AI quality platform for developers that builds evaluators, guardrails and monitoring for LLM applications

Enterprise agent security and governance platform covering agent discovery, guardrails, evals and traces

Open-source framework to evaluate, test and monitor ML and LLM systems

Open-source Python library for evals, red teaming and test generation for agentic systems

Simulation and evaluation platform that tests voice and text agents with synthetic users

Enterprise service for objective evaluation of generative AI models using adaptive and static rubrics

Platform for agent observability and evals: trace agents, score outputs and find patterns in production

Open-source holistic solution for AI and ML validation: testing, CI and monitoring

Framework and registry of evals for testing LLMs and LLM-based systems, maintained by OpenAI