online-evals
Attach judges to config variations for automatic LLM-as-a-judge evaluation.
Skill by launchdarklyOfficial
Install
Works with Claude Code, Claude apps, Codex, Cursor, Gemini CLI, GitHub Copilot, VS Code, OpenCode, Goose, Windsurf. Read the repository before you give any tool access to your data.
npx agentmarket install skill/launchdarkly-online-evals
Asks which of your tools to add it to (Claude Code, Codex, Cursor, VS Code, Windsurf, Gemini CLI, Claude Desktop) and shows the change first. See https://agentmart.dev/cli.
Get the weekly diff
The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.
About
Use cases: ml
Reviews
Write a reviewNo reviews yet. Installed it? Reviews come from verified installers only. How reviews work
Permissions
What this skill can do and what it asks for, from its SKILL.md. Checked Oct 25, 2026.
- Tools
- No tools found
- Secrets
- None asked for
- Install scripts
- None found
Network hosts it talks to 1
codeload.github.com· found in install command
Files it touches
~/.claude/skills/online-evals
What's inside
- Skill name
online-evals- Description
- Attach judges to config variations for automatic LLM-as-a-judge evaluation. Create custom judges, configure sampling rates, and monitor quality scores.
- License
- NOASSERTION
- Compatibility
- Requires LaunchDarkly API access token with ai-configs:write permission. SDK versions Python v0.20.0+ or Node.js v0.20.0+ for automatic metric recording and the consolidated `track_judge_result` / `trackJudgeResult` API.
- Metadata
- author: launchdarkly · version: 0.1.0
- Instructions
- 446 lines in SKILL.md · view on GitHub
Before installing
- Instructions call the network
Agent Skills spec issues
- body: contains an absolute path from the author's machine
Files (2)
- README.md
- SKILL.md
The license isn't open source, so the instructions aren't shown here. Read SKILL.md on GitHub
More from launchdarkly/ai-tooling
- 0.752score
launchdarkly-flag-drift
launchdarklyOfficial
Detect and reconcile drift between a feature flag's in-code SDK fallback default and its LaunchDarkly default rule (fallthrough).
SkillAFreshness: A★ 28Claude Code +9 - 0.749score
built-in-metrics
launchdarklyOfficial
Instrument an existing codebase with LaunchDarkly config tracking.
SkillAFreshness: A★ 28Claude Code +9 - 0.748score
launchdarkly-flag-qualitative-feedback-setup
launchdarklyOfficial
Integrate LaunchDarkly qualitative user feedback into a JavaScript/TypeScript codebase.
SkillAFreshness: A★ 28Claude Code +9 - 0.747score
launchdarkly-release-autopilot-settings
launchdarklyOfficial
Configure LaunchDarkly Release Autopilot settings (GitHub App auto-flagging and auto-releasing) via the hosted MCP, or diagnose why a pull request was not…
SkillAFreshness: A★ 28Claude Code +9 - 0.743score
projects
launchdarklyOfficial
Guide for setting up LaunchDarkly projects in your codebase.
SkillAFreshness: A★ 28Claude Code +9 - 0.743score
custom-metrics
launchdarklyOfficial
Create, track, retrieve, update, and delete custom business metrics for configs.
SkillAFreshness: A★ 28Claude Code +9 - 0.742score
migrate
launchdarklyOfficial
Migrate an application with hardcoded LLM prompts to a full LaunchDarkly AgentControl implementation in five stages: audit the code, wrap the call, move the…
SkillAFreshness: A★ 28Claude Code +9 - 0.739score
configs-create
launchdarklyOfficial
Create and configure configs in LaunchDarkly.
SkillAFreshness: A★ 28Claude Code +9 - 0.733score
configs-targeting
launchdarklyOfficial
Configure config targeting rules to control which variations serve to different users.
SkillAFreshness: A★ 28Claude Code +9 - 0.707score
flag-release
launchdarklyOfficial
Record an automated rollout for an existing LaunchDarkly flag that guards a pull request's change, so the change releases safely when the PR merges.
SkillBFreshness: B★ 28Claude Code +9 - 0.707score
onboarding
launchdarklyOfficial
Scripted onboarding for LaunchDarkly: quiet execution, fixed sequence, SDK install, first flag with a live reveal, MCP offered afterwards.
SkillBFreshness: B★ 28Claude Code +9 - 0.701score
should-flag-change
launchdarklyOfficial
Decide whether a given code change should be placed behind a LaunchDarkly feature flag.
SkillBFreshness: B★ 28Claude Code +9
Signals
- Stars
- 28
- Forks
- 11
- Open issues
- 7
- Last commit (this folder)
- Sep 28, 2026 (1 month ago)
From GitHub (fetched Oct 24, 2026).
Details
- Version
- 0.1.0
- License
- NOASSERTION
- Repository
- github.com/launchdarkly/ai-tooling
- First seen
- Oct 24, 2026
- Last checked
- Oct 24, 2026
- Source
- GitHubgithub-graphql
More from launchdarkly
- 0.752score
launchdarkly-flag-drift
launchdarklyOfficial
Detect and reconcile drift between a feature flag's in-code SDK fallback default and its LaunchDarkly default rule (fallthrough).
SkillAFreshness: A★ 28Claude Code +9 - 0.749score
built-in-metrics
launchdarklyOfficial
Instrument an existing codebase with LaunchDarkly config tracking.
SkillAFreshness: A★ 28Claude Code +9 - 0.748score
launchdarkly-flag-qualitative-feedback-setup
launchdarklyOfficial
Integrate LaunchDarkly qualitative user feedback into a JavaScript/TypeScript codebase.
SkillAFreshness: A★ 28Claude Code +9 - 0.747score
launchdarkly-release-autopilot-settings
launchdarklyOfficial
Configure LaunchDarkly Release Autopilot settings (GitHub App auto-flagging and auto-releasing) via the hosted MCP, or diagnose why a pull request was not…
SkillAFreshness: A★ 28Claude Code +9 - 0.743score
custom-metrics
launchdarklyOfficial
Create, track, retrieve, update, and delete custom business metrics for configs.
SkillAFreshness: A★ 28Claude Code +9 - 0.743score
projects
launchdarklyOfficial
Guide for setting up LaunchDarkly projects in your codebase.
SkillAFreshness: A★ 28Claude Code +9
Cite this page
online-evals skill. Agent Mart. https://agentmart.dev/skills/launchdarkly-online-evals (last updated 2026-10-24)