“Monitoring and observability”

Open-source LLM observability and evaluation platform with agent tracing and test suites

Open-source LLM evaluation framework, similar to Pytest, for unit testing LLM apps and agents

Continuous-improvement stack for agents: monitor behavior, detect failures and validate fixes

Open-source agent harness engineering platform with OpenTelemetry tracing, evals and guardrails

Observability and dev-tool platform for building, evaluating and monitoring AI agents

Open-source framework for large language model evaluations, built by the UK AI Security Institute

AI gateway giving one API to 250+ models, with observability, guardrails and prompt management.

Open-source framework to evaluate, test and monitor ML and LLM systems

Tracing, evaluation and monitoring for LLM apps and agents, with SDKs for several languages

Open-source holistic solution for AI and ML validation: testing, CI and monitoring