Everything in the catalog

Open-source platform for tracing, evaluating and improving AI agents

Open-source platform for tracing, evaluating and managing prompts for agents, LLMs and ML models

Open-source CLI and library for evaluating and red-teaming LLM apps and agents

Open-source LLM observability and evaluation platform with agent tracing and test suites

Open-source LLM evaluation framework, similar to Pytest, for unit testing LLM apps and agents

Open-source tool to trace, evaluate and improve AI agents, run locally or self-hosted

Open-source observability for LLM applications, built on OpenTelemetry

AI gateway and LLM observability platform for logging, debugging and routing LLM requests

Open-source platform to trace, test, evaluate and govern every LLM call and AI agent in production

Open-source, MIT-licensed platform for observing production AI agents and turning failures into tracked signals

Observability platform for AI apps covering model calls, agent steps, API requests and database queries in one trace

Continuous-improvement stack for agents: monitor behavior, detect failures and validate fixes

Python framework with input and output guards that detect LLM risks, plus structured output generation

Open-source, OpenTelemetry-native evals and tracing for AI agents

Open-source observability platform for AI agents that detects failures and supports evals

Meta's open safeguard models that moderate LLM prompts and responses, including Llama Guard 4.

CLI that discovers agent components on a machine and scans MCP servers and skills for prompt injections.

Open-source agent harness engineering platform with OpenTelemetry tracing, evals and guardrails

Open-source framework for large language model evaluations, built by the UK AI Security Institute

Toolkit for tracing and evaluating AI agents and LLM apps, built by Weights & Biases

Open-source Python library for adding programmable guardrails to LLM-based conversational applications

Observability and dev-tool platform for building, evaluating and monitoring AI agents

Open-source observability for AI apps: traces LLM calls, prompts, retrieval and agent steps via OpenTelemetry

Toolkit for evaluating LLM applications with objective metrics, test data generation and feedback loops