“LLM evaluation”

Open-source LLM evaluation framework, similar to Pytest, for unit testing LLM apps and agents

Toolkit for evaluating LLM applications with objective metrics, test data generation and feedback loops

Open-source platform for tracing, evaluating and improving AI agents

Open-source LLM observability and evaluation platform with agent tracing and test suites

Open-source CLI and library for evaluating and red-teaming LLM apps and agents

Open-source platform to trace, test, evaluate and govern every LLM call and AI agent in production

Toolkit for tracing and evaluating AI agents and LLM apps, built by Weights & Biases

Open-source platform for tracing, evaluating and managing prompts for agents, LLMs and ML models

Open-source tool to trace, evaluate and improve AI agents, run locally or self-hosted

Open-source framework to evaluate, test and monitor ML and LLM systems

Tracing, evaluation and monitoring for LLM apps and agents, with SDKs for several languages

Open-source tool that runs dynamic attacks against a GenAI app's system prompt to help harden it.

Observability and dev-tool platform for building, evaluating and monitoring AI agents

Open-source LLM red teaming framework that simulates attacks to find vulnerabilities in AI apps and agents.

Open-source Python library for evals, red teaming and test generation for agentic systems

Hobby project for LLM code editing in large codebases, run via Docker with a web UI and a SWE-bench harness.