diagnosing-bugs

Files (1)

• SKILL.md

SKILL.md (first lines)

<!-- Adapted from mattpocock/skills diagnosing-bugs at 6654f6b60cd9d5be8b54c6fafe44346dabeb3b76 (MIT). -->

# Diagnosing Bugs

A discipline for hard bugs. Skip phases only when explicitly justified.

When exploring the codebase, from the repository root read `dydo/glossary.md` and
`dydo/understand/architecture.md` to get a clear mental model of the relevant
modules, and check the Decision Records in the area you're touching.

**Run as crew**, you ask and post only through your Ticket Captain: each "ask the user" and each post
"on the ticket" below becomes a hand-raise to the captain carrying what you tried, never a post of your
own. A Bug fix needs its red reproduction, so a missing correct seam also returns to the captain instead of excusing the regression test. Run directly by the user, the steps stand
as written.

## Redact

This skill has you show commands, outputs and captured artifacts. **Redact every secret first**: write
`<REDACTED>` in its place. Build loops against env vars, so the credential stays in the environment
rather than in what you show. Captured artifacts carry auth headers: quote only the lines that carry
the signal.

If the redacted output is not enough to diagnose the bug, say so on the ticket and ask the user.

## Phase 1: Build a feedback loop

**This is the skill.** Everything else is mechanical. If you have a **tight** pass/fail signal for the
bug (one that goes red on _this_ bug), you will find the cause; bisection, hypothesis-testing, and
instrumentation all just consume it. If you don't have one, no amount of staring at code will save you.

Spend disproportionate effort here. **Be aggressive. Be creative. Refuse to give up.**

### Ways to construct one, in roughly this order

1. **Failing test** at whatever seam reaches the bug: unit, integration, e2e.
2. **Curl / HTTP script** against a running dev server.
3. **CLI invocation** with a fixture input, diffing stdout against a known-good snapshot.
4. **Headless browser script** (Playwright / Puppeteer) that drives the UI and asserts on
   DOM/console/network.
5. **Replay a captured trace.** Save a real network request / payload / event log to disk; replay it
   through the code path in isolation.
6. **Throwaway harness.** Spin up a minimal subset of the system (one service, mocked deps) that
   exercises the bug code path with a single function call.
7. **Property / fuzz loop.** If the bug is "sometimes wrong output", run 1000 random inputs and look
   for the failure mode.
8. **Bisection harness.** If the bug appeared between two known states (commit, dataset, version),
   automate "boot at state X, check, repeat" so you can `git bisect run` it.
9. **Differential loop.** Run the same input through old-version vs new-version (or two configs) and
   diff outputs.
10. **HITL script.** Last resort. If a human must click, drive _them_ with a numbered script of steps
    and paste-back points so the loop is still structured. Captured output feeds back to you.

Build the right feedback loop, and the bug is 90% fixed.

### Tighten the loop

Treat the loop as a product. Once you have _a_ loop, **tighten** it:

- Can I make it faster? (Cache setup, skip unrelated init, narrow the test scope.)
- Can I make the signal sharper? (Assert on the specific symptom, not "didn't crash".)
- Can I make it more deterministic? (Pin time, seed RNG, isolate filesystem, freeze network.)

A 30-second flaky loop is barely better than no loop; a 2-second deterministic one is tight, a
debugging superpower.

### Non-deterministic bugs

The goal is not a clean repro but a **higher reproduction rate**. Loop the trigger 100×, narrow
timing windows, force timing with fake timers or explicit interleaving. A 50%-flake bug is debuggable; 1% is not, so keep raising the rate
until it's debuggable. Shared machine: never synthetic CPU load; any load stays small and cleans up.

### When you genuinely cannot build a loop

Stop and say so on the ticket. List what you tried. Ask the user for: (a) access to whatever
environment reproduces it, (b) a redacted captured artifact (HAR file, log dump, core dump, screen
recording with timestamps), or (c) permission to add temporary production instrumentation. Do **not**
proceed to hypothesise without a loop.

### Completion criterion: a tight loop that goes red

Phase 1 is done when the loop is **tight** and **red-capable**: you can name **one command** (a script
path, a test invocation, a curl) that you have **already run at least once** (show the invocation and
its output, redacted), and that is:

- [ ] **Red-capable**: it drives the actual bug code path and asserts the **ticket's exact symptom**,
      so it can go red on this bug and green once fixed. Not "runs without erroring"; it must be able
      to _catch this specific bug_.
- [ ] **Deterministic**: same verdict every run (flaky bugs: a pinned, high reproduction rate, per
      above).
- [ ] **Fast**: seconds, not minutes.
- [ ] **Agent-runnable**: you can run it unattended; a human in the loop only via a HITL script.

If you catch yourself reading code to build a theory before this command exists, **stop: jumping
straight to a hypothesis is the exact failure this skill prevents.** No red-capable command, no
Phase 2.

## Phase 2: Reproduce + minimise

Run the loop. Watch it go red as the bug appears.

Confirm:

- [ ] The loop produces the failure mode the **ticket** described, not a different failure that
      happens to be nearby. Wrong bug = wrong fix.
- [ ] The failure is reproducible across multiple runs (or, for non-deterministic bugs, reproducible
      at a high enough rate to debug against).
- [ ] You have captured the exact symptom (error message, wrong output, slow timing) so later phases
      can verify the fix actually addresses it.

### Minimise

Once it's red, shrink the repro to the **smallest scenario that still goes red**. Cut inputs, callers,
config, data, and steps **one at a time**, re-running the loop after each cut, and keep only what's
load-bearing for the failure.

Why bother: a minimal repro shrinks the hypothesis space in Phase 3 (fewer moving parts left to
suspect) and becomes the clean regression test in Phase 5.

Done when **every remaining element is load-bearing**: removing any one of them makes the loop go

From bodnarbalazs/dydo, MIT.

More from bodnarbalazs/dydo