Skip to content
Agentmart
Submit a tool

diagnosing-experiment-health

Runs diagnostics and a health check on one PostHog experiment: its setup, exposures, results and changes during the run.

Skill by PostHogOfficial

IndexedLicense unknown★ 95
Write a review

Install

Works with Claude Code, Claude apps, Codex, Cursor, Gemini CLI, GitHub Copilot, VS Code, OpenCode, Goose, Windsurf. Read the repository before you give any tool access to your data.

npx agentmarket install skill/posthog-diagnosing-experiment-health

Asks which of your tools to add it to (Claude Code, Codex, Cursor, VS Code, Windsurf, Gemini CLI, Claude Desktop) and shows the change first. See https://agentmart.dev/cli.

Get the weekly diff

The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.

We use your email only for this newsletter. Privacy

About

Runs diagnostics and a health check on one PostHog experiment: its setup, exposures, results and changes during the run. Covers 0 exposures, sample ratio mismatch, lost or uneven exposures, users in multiple variants, identity faults, who the experiment counts, significance traps (peeking, A/A, 'target reached', Bayesian vs Frequentist, sequential, CUPED), mid-run edits, pause and freeze, and survey follow-up. TRIGGER when: user asks 'is my experiment healthy / set up right / biased?' or 'why 0 exposures?', asks why it counts different people than their insight, mentions the bias or mismatch warning, an uneven split, significance that flips, numbers that disagree with their SQL, results that changed after an edit, or wants user feedback on an experiment. DO NOT TRIGGER when: creating an experiment (use creating-experiments), only configuring rollout (use configuring-experiment-rollout) or metrics (use configuring-experiment-analytics), or only asking lifecycle questions (use managing-experiment-lifecycle).

Use cases: backend, data, writing, research

Reviews

Write a review

No reviews yet. Installed it? Reviews come from verified installers only. How reviews work

Permissions

What this skill can do and what it asks for, from its SKILL.md. Checked Oct 25, 2026.

Tools
No tools found
Secrets
None asked for
Install scripts
None found

Network hosts it talks to 1

  • codeload.github.com · found in install command

Files it touches

  • ~/.claude/skills/diagnosing-experiment-health

What's inside

Skill name
diagnosing-experiment-health
Description
Runs diagnostics and a health check on one PostHog experiment: its setup, exposures, results and changes during the run. Covers 0 exposures, sample ratio mismatch, lost or uneven exposures, users in multiple variants, identity faults, who the experiment counts, significance traps (peeking, A/A, 'target reached', Bayesian vs Frequentist, sequential, CUPED), mid-run edits, pause and freeze, and survey follow-up. TRIGGER when: user asks 'is my experiment healthy / set up right / biased?' or 'why 0 exposures?', asks why it counts different people than their insight, mentions the bias or mismatch warning, an uneven split, significance that flips, numbers that disagree with their SQL, results that changed after an edit, or wants user feedback on an experiment. DO NOT TRIGGER when: creating an experiment (use creating-experiments), only configuring rollout (use configuring-experiment-rollout) or metrics (use configuring-experiment-analytics), or only asking lifecycle questions (use managing-experiment-lifecycle).
License
unknown
Instructions
257 lines in SKILL.md · view on GitHub

Agent Skills spec issues

  • files: referenced but missing: references/changing-distribution-after-launch.md

Files (8)

  • SKILL.md
  • bias-and-skew.md · references/
  • diagnostic-snapshot.md · references/
  • empty-experiment.md · references/
  • interpretation.md · references/
  • mid-run-changes.md · references/
  • numbers-vs-sql.md · references/
  • qualitative-feedback.md · references/

The license isn't open source, so the instructions aren't shown here. Read SKILL.md on GitHub

More from PostHog/ai-plugin

Signals

Stars
95
Forks
20
Open issues
12
Last commit (this folder)
Oct 21, 2026 (9 days ago)

From GitHub (fetched Oct 24, 2026).

Details

License
License unknown
Repository
github.com/PostHog/ai-plugin
First seen
Oct 24, 2026
Last checked
Oct 24, 2026
Source
GitHubgithub-graphql

Same skill elsewhere

Copies of this SKILL.md published by others; search shows only one of them.

More from PostHog

Cite this page

diagnosing-experiment-health skill. Agent Mart. https://agentmart.dev/skills/posthog-diagnosing-experiment-health (last updated 2026-10-24)