“monitoring”

APM-based AI monitoring for end-to-end visibility into the performance, cost and quality of LLM apps

AI observability in Sentry for tracing and debugging agent runs, tool calls, token usage and errors

Test, monitor and self-improve voice agents with simulation, red teaming and production monitoring

Trust platform for voice and chat agents: pre-launch testing, red teaming and production monitoring

Open-source holistic solution for AI and ML validation: testing, CI and monitoring

Platform for developers of AI chatbots and LLM apps, covering monitoring, prompts and analytics

Control plane for enterprise AI agents covering monitoring, evals, enforcement and governance

AI governance platform to discover, test, monitor and govern AI systems and enforce controls

Continuous-improvement stack for agents: monitor behavior, detect failures and validate fixes

Monitor the performance, usage, cost and guardrails of LLM and agentic AI apps in Elastic Observability

AI quality platform for developers that builds evaluators, guardrails and monitoring for LLM applications

Open-source framework to evaluate, test and monitor ML and LLM systems

Tracing, evaluation and monitoring for LLM apps and agents, with SDKs for several languages

Observability and dev-tool platform for building, evaluating and monitoring AI agents

Open-source LLM observability and evaluation platform with agent tracing and test suites

Open-source LLM evaluation framework, similar to Pytest, for unit testing LLM apps and agents

Monitoring platform for AI agents that discovers silent agent failures in production

Agent observability platform for tracing, evaluating, monitoring and improving AI agents

Platform for monitoring, troubleshooting and evaluating LLM applications and AI agents

Full-stack observability for AI applications, LLMs and agents, covering performance, cost, quality and compliance

Open-source agent harness engineering platform with OpenTelemetry tracing, evals and guardrails

Open-source framework for large language model evaluations, built by the UK AI Security Institute

Connects prompts, model calls, tool use, cost and failures to help debug LLM apps and agents in production