Observability & evals

Open-source CLI and library for evaluating and red-teaming LLM apps and agents

Open-source LLM evaluation framework, similar to Pytest, for unit testing LLM apps and agents

Open-source platform to trace, test, evaluate and govern every LLM call and AI agent in production

Open-source, OpenTelemetry-native evals and tracing for AI agents

Open-source framework for large language model evaluations, built by the UK AI Security Institute

Toolkit for evaluating LLM applications with objective metrics, test data generation and feedback loops

Open-source workspace where teams build AI agents by chatting and run them in Slack, WhatsApp or Telegram

Open-source framework to evaluate, test and monitor ML and LLM systems

Open-source Python library for evals, red teaming and test generation for agentic systems

Platform for agent observability and evals: trace agents, score outputs and find patterns in production

Open-source holistic solution for AI and ML validation: testing, CI and monitoring