“LLM evaluation”

Open-source LLM evaluation framework, similar to Pytest, for unit testing LLM apps and agents

Toolkit for evaluating LLM applications with objective metrics, test data generation and feedback loops

Platform for monitoring, troubleshooting and evaluating LLM applications and AI agents

Open-source platform for tracing, evaluating and improving AI agents

Open-source LLM observability and evaluation platform with agent tracing and test suites

Open-source CLI and library for evaluating and red-teaming LLM apps and agents

Open-source platform to trace, test, evaluate and govern every LLM call and AI agent in production

Toolkit for tracing and evaluating AI agents and LLM apps, built by Weights & Biases

Open-source platform for tracing, evaluating and managing prompts for agents, LLMs and ML models

Full-stack observability for AI applications, LLMs and agents, covering performance, cost, quality and compliance

Open-source tool to trace, evaluate and improve AI agents, run locally or self-hosted

Enterprise service for objective evaluation of generative AI models using adaptive and static rubrics

Open-source framework to evaluate, test and monitor ML and LLM systems

Tracing, evaluation and monitoring for LLM apps and agents, with SDKs for several languages

AI quality platform for developers that builds evaluators, guardrails and monitoring for LLM applications

AI engineering platform for observing, evaluating and improving AI agents

Open-source tool that runs dynamic attacks against a GenAI app's system prompt to help harden it.

Observability and dev-tool platform for building, evaluating and monitoring AI agents

Open-source LLM red teaming framework that simulates attacks to find vulnerabilities in AI apps and agents.

OpenTelemetry-native observability for LLMs, vector databases, GPUs, MCP servers and AI evaluations in Grafana Cloud

Control plane for enterprise AI agents covering monitoring, evals, enforcement and governance

Open-source Python library for evals, red teaming and test generation for agentic systems

Traces every LLM call in your AI product, with prompt, tokens, cost, latency and the user who triggered it