bmad-run-inspector

Files (15)

• README.md
• README.vi.md
• README.zh.md
• SKILL.md
• environment.md · assets/templates/
• environment.toml · assets/templates/
• project-readme.md · assets/templates/
• adapter-bootstrap.md · references/
• anomaly-triage.md · references/
• log-forensics.md · references/
• run-anatomy.md · references/
• bootstrap_adapter.py · scripts/
• extract_transcript.py · scripts/
• run_probe.py · scripts/
• test_scripts.py · tests/

SKILL.md (first lines)

# BMAD Run Inspector

Verified against [bmad-loop 0.13.2](https://github.com/bmad-code-org/bmad-loop/releases/tag/v0.13.2).
The bundled scripts run on Python 3.9+. `bmad-loop` itself needs 3.11+, but it often lives in
an isolated environment while the host's `python3` is still stock macOS 3.9.

A run leaves a directory of evidence. This skill is about reading it honestly: saying what it
proves, and refusing to say what it doesn't.

## The one thing to get right

**The `.log` file is a terminal redraw capture, not a transcript.** The coding CLI paints a
live TUI into a tmux or psmux pane. The capture records every repaint.

One adapter differs. `opencode-http` writes a real transcript instead. Read
`references/log-forensics.md` before applying any of this to it.

Three consequences:

- Each logical line appears hundreds of times, growing character by character. `tail -50`
  returns fragments of one frame, not the last 50 things that happened.
- Whitespace is dropped unpredictably. `Done(10tooluses·99.2ktokens·1m0s)` is one real line.
- **Long tool output is collapsed into `… +N lines (ctrl+o to expand)` and never painted.**
  Those lines do not exist in the file.

The third one decides what you may claim. Test summaries sit inside collapsed blocks. Your test
runner's pass/fail line is long output, so the TUI hides it. One measured 4.35 MB log had 58
collapse markers hiding roughly 2,463 lines. Its largest single block was 484 lines.

So **never report that tests passed or failed based on the log.** Report which test *commands*
ran. The verdict comes from "Deciding whether verify passed" below.

**The heuristics are stack-specific. They fail quietly.** `scripts/extract_transcript.py`
matches one coding CLI's TUI vocabulary: its spinner frames, its `Done(N tool uses · T tokens ·
Ns)` footer, its `… +N lines (ctrl+o to expand)` marker. Its `results` and `errors` sections
match one test runner's banner, one type checker's error codes, one runtime's errno names.

Change the CLI or change the test stack and the script does not error. It matches fewer
patterns and still returns a result. An empty `--section errors` or `--section results` on an
unfamiliar CLI or stack is a miss, not good news.

So establish which CLI and which stack the repo runs before you read anything. The next section
covers that. Run `scripts/extract_transcript.py --collapsed` to print how much is hidden, then
quote that number instead of hedging.

## Know the project before you read it

This skill ships generic. Everything that varies between repositories lives in a project
adapter at `_project/bmad-loop/`. Read it before the first inspection. Bootstrap it when it is
not there.

| File | Holds |
|---|---|
| `_project/bmad-loop/environment.toml` | the values: which coding CLI, which test runner and type checker, which multiplexer, whether the dev skill writes `result.json`, which verify steps are non-fatal, what feeds the backlog |
| `_project/bmad-loop/environment.md` | the knowledge: a dated current-state snapshot, which of the extractor's constants match here and which don't, and the judgment calls a bare value can't carry |

When `environment.toml` is missing:

```bash
python3 <skill>/scripts/bootstrap_adapter.py --repo-root /path/to/repo
```

It writes skeletons, never overwrites, and prints every value still marked `TODO(confirm: …)`.
Those TODOs are the point. Research each one from a real file or a real `bmad-loop` command.
Then show the user the drafted values and where each came from, before you inspect anything.

An adapter of plausible guesses is worse than no adapter. A wrong `coding_cli` makes the
extractor match nothing and return a result that reads clean.
`references/adapter-bootstrap.md` sources each field, including the ones no one can answer
until a run exists.

When the adapter and a live run disagree, the run wins. The disagreement is itself a finding
about the adapter. Report it. Do not quietly override either one.

## Check the CLI before reading the disk

`bmad-loop` answers some questions faster and more reliably than parsing artifacts. `list`,
`status`, `diagnose`, plain `validate`, `adapters` and bare `mux` are safe on a live run. Try them
first.

| Command | What it gives you |
|---|---|
| `list`, `status` | Run state, but only for `.bmad-loop/runs/`. An archived run is invisible to both and returns `no such run`. Extract the tarball and read the artifacts by hand, starting at `## Workflow` |
| `adapters` | Which coding-CLI adapter each profile selects. This is where the adapter's `orchestrator.coding_cli` comes from. Re-run it when an extraction comes back suspiciously empty |
| `validate` | Live host facts no run directory holds: multiplexer availability and version, whether the coding CLI is on PATH, hook registration and staleness, worktree cleanliness. Run it when a story fails for reasons that look environmental |

**Never mutate what you are observing.** These commands read as harmless and are not:

- `mux set` writes `policy.toml`.
- `confirm` and `decisions` act on their target unless called with `--list`.
- `attach` joins a live session. Any keystroke sent to it acts.
- `probe-adapter --probe` launches a real CLI turn.
- `validate --probes` runs the project's `[environment] probes` shell. `validate --render-probe` runs
  each dev skill's render command and needs network.
- `resolve` (any flag) and `resume` change the run. `resolve --reverify` re-runs verify and commits.
- `run`, `sweep`, `clean` and `cleanup` act unless given `--dry-run`.
- `tui` is not confirmed inert in every view. Treat it as not scriptable.

`references/anomaly-triage.md` has the full table.

## Workflow

Live watch and post-hoc forensics use the same three steps. Only the leading question differs.

### 1. Probe the state

```bash
python3 <skill>/scripts/run_probe.py --project /path/to/repo
```

Prints health flags, per-task phase/attempt/review_cycle, heartbeats, log sizes, journal tail,
ATTENTION metadata, any pending hard or graceful stop request, and a findings list.

It also writes `.probe-snapshot.json` into the run directory. The next probe reads that file
and reports what changed. The delta is what separates "working" from "hung". It also separates
a new ATTENTION notice from an unchanged append-only file. One reading alone answers neither.

Read thresholds from `state.json`'s `policy_snapshot`, never from memory. Every project tunes
`max_dev_attempts` and `session_timeout_min` differently. A remembered 2 becomes a false alarm

From tronghieu/agent-skills, MIT.

More from tronghieu/agent-skills