“monitoring”

Open-source holistic solution for AI and ML validation: testing, CI and monitoring

Open-source persistent development workspace with unattended tasks, schedules and heartbeat monitoring.

Continuous-improvement stack for agents: monitor behavior, detect failures and validate fixes

Open-source framework to evaluate, test and monitor ML and LLM systems

Tracing, evaluation and monitoring for LLM apps and agents, with SDKs for several languages

Observability and dev-tool platform for building, evaluating and monitoring AI agents

API to search, scrape and interact with the web for AI agents, open source and available as a hosted service.

Open-source LLM observability and evaluation platform with agent tracing and test suites

Open-source LLM evaluation framework, similar to Pytest, for unit testing LLM apps and agents

Orchestration platform for AI agents, background tasks and durable workflows.

Terminal coding agent with per-request model routing across six cost tiers, a code graph and spend monitoring, served by a hosted router.

Open-source agent harness engineering platform with OpenTelemetry tracing, evals and guardrails

Open-source framework for large language model evaluations, built by the UK AI Security Institute

Desktop and mobile app that runs several coding agents side by side, each in its own git worktree

AI gateway giving one API to 250+ models, with observability, guardrails and prompt management.

Open-source personal assistant with a RAG knowledge base, agent loops and MCP tool access.

Desktop app that runs Claude Code, Codex and other coding agents in parallel, each in its own git worktree

Self-hosted web UI for the Pi Coding Agent that keeps agent sessions running in real workspaces

Background agent that picks up Linear, GitHub, GitLab and Slack issues and runs Claude Code, Codex or other coding agents on them.

Desktop GUI for the Pi coding agent that runs the same engine as the Pi CLI, with multi-session chat and tool approvals.

Agent that runs deep learning experiments around the clock on local, SSH or Slurm backends and records each outcome