New in week 43

Open-source workspace where teams build AI agents by chatting and run them in Slack, WhatsApp or Telegram

Observability and dev-tool platform for building, evaluating and monitoring AI agents

Open-source tool to trace, evaluate and improve AI agents, run locally or self-hosted

Enterprise agent security and governance platform covering agent discovery, guardrails, evals and traces

Platform for agent observability and evals: trace agents, score outputs and find patterns in production

Python tool that scans MCP servers and their tools for security findings using YARA, LLM and AI Defense analysis.

Open-source LLM observability and evaluation platform with agent tracing and test suites

Open-source LLM evaluation framework, similar to Pytest, for unit testing LLM apps and agents

Open-source LLM red teaming framework that simulates attacks to find vulnerabilities in AI apps and agents.

Automated LLM fuzzer that tries jailbreak attacks against models and LLM APIs, including your own REST API.

Open-source holistic solution for AI and ML validation: testing, CI and monitoring

Open-source framework to evaluate, test and monitor ML and LLM systems

Open-source observability for AI apps: traces LLM calls, prompts, retrieval and agent steps via OpenTelemetry

Open-source Python library for evals, red teaming and test generation for agentic systems

Python framework with input and output guards that detect LLM risks, plus structured output generation

AI gateway and LLM observability platform for logging, debugging and routing LLM requests

IBM model family that judges whether LLM inputs and outputs meet criteria such as jailbreak, harm and hallucination checks.

Continuous-improvement stack for agents: monitor behavior, detect failures and validate fixes

Tracing, evaluation and monitoring for LLM apps and agents, with SDKs for several languages

Open-source platform for tracing, evaluating and improving AI agents

Open-source platform to trace, test, evaluate and govern every LLM call and AI agent in production

Open-source, MIT-licensed platform for observing production AI agents and turning failures into tracked signals

Open-source observability platform for AI agents that detects failures and supports evals

Meta's open safeguard models that moderate LLM prompts and responses, including Llama Guard 4.

Open-source Python framework from Microsoft for finding risks in generative AI systems through red teaming.

Open-source platform for tracing, evaluating and managing prompts for agents, LLMs and ML models

Open-source Python library for adding programmable guardrails to LLM-based conversational applications

Open-source LLM vulnerability scanner that probes models and apps for prompt injection, data leakage and jailbreaks.

Framework and registry of evals for testing LLMs and LLM-based systems, maintained by OpenAI

Open-source agent harness engineering platform with OpenTelemetry tracing, evals and guardrails

Open-source tool that runs dynamic attacks against a GenAI app's system prompt to help harden it.

Open-source CLI and library for evaluating and red-teaming LLM apps and agents

Observability platform for AI apps covering model calls, agent steps, API requests and database queries in one trace

CLI that discovers agent components on a machine and scans MCP servers and skills for prompt injections.

Security scanner for agentic workflows that maps tools, MCP servers and known vulnerabilities into an HTML report.

Open-source observability for LLM applications, built on OpenTelemetry

Open-source, OpenTelemetry-native evals and tracing for AI agents

Open-source framework for large language model evaluations, built by the UK AI Security Institute

Automated prompt injection scanner for custom LLM applications, with white-box and black-box testing modes.

Toolkit for evaluating LLM applications with objective metrics, test data generation and feedback loops

Toolkit for tracing and evaluating AI agents and LLM apps, built by Weights & Biases

Coding agent security platform that watches agent actions on developer machines and blocks dangerous chains.

Configurable safeguards for generative AI apps that filter harmful content, redact PII and detect hallucinations.

AI engineering platform for observing, evaluating and improving AI agents

Microsoft AI service that detects harmful text and image content and screens user input for prompt attacks.

Test, monitor and self-improve voice agents with simulation, red teaming and production monitoring

Finds AI assets and risks, red teams models and agents, and blocks threats to AI apps in real time.

AI observability with real-time evaluators, dashboards, session tracing and cost tracking for AI agents

Deployment-readiness platform for voice AI agents: simulate calls, evaluate production conversations and add human review

Platform for monitoring, troubleshooting and evaluating LLM applications and AI agents