verify

Files (1)

• SKILL.md

SKILL.md (first lines)

# verify — exercise it, prove it, root-cause failures

Two jobs: prove the change works with fresh evidence, and when it doesn't, find *why* before
changing anything. "Should pass", a previous run, or someone's self-report are never enough —
you run it now and read what actually happened.

Read [CONVENTIONS.md](../../CONVENTIONS.md) for the workspace (§1), the ledger (§2), skip rules (§7), the feature changelog (§13), memory (§4), the resume
sweep (§5), multi-agent rules (§8), platform adapters (§9), closing output (§18 — the
evidence is the result; don't pad it with suggestions), data-driven decisions (§19), and workspace integrity (§20). Load the **discipline pack** for the detected stack
from `references/disciplines/` — its "In VERIFY" section tells you how to exercise *this*
surface for real (real endpoints and error/auth paths for backend, offline and
permission-denied paths for mobile, loading/error/empty states for frontend, a fresh eval-set
run for AI/ML). Load any **concern pack** the change touches the same way — e.g.
`database.md`'s VERIFY section (run migrations up *and* down, realistic data volume).

**Journal every run (§2.1).** The first write of the run, once the task folder is resolved
(§20.1), is a START entry in the task's `log.md`; then set this phase `in_progress` in
`state.json`. Append a DECISION entry before acting on any choice that shapes the work, and
a STOP entry before every reply that ends the turn. Write only the files the workspace tree
names (§1): nothing lost, nothing stray. Before asking a question or deciding, check the
user's judgment (§4.1) for a confirmed rule that covers it, and journal who decided.

## Part A — Prove it works (the evidence gate)

Walk **§5.1's evidence gate** for every claim you are about to make — identify the proving
command, run it now in full, read the real output, compare it to the claim, and only then
speak. This skill is where that gate is exercised hardest: everything downstream treats your
verdict as fact.

**Then reconcile against the code (§5.2).** Walk every spec criterion, plan task, and locked
decision against the code as it is now, and put the reconciliation table in `verify.md`:
each row with its file and line, or with the searches that found nothing. Never take status
from `state.json`, the journal, or memory. A `missing` or `done differently` row is not a pass.

```
1. Identify the proving command(s): the full test suite, plus how to exercise the actual
   behavior (run the endpoint, click the flow, invoke the function with real input).
2. Run them FRESH and in FULL — not a cached result, not a subset you assume covers it.
3. Read the exit code and the output. Actually read it.
4. Only now state the result, WITH the evidence: "18/18 passing" and the command, or the
   exact failure.
```

**For a web UI, load `references/disciplines/browser-verify.md`** — the console-clean gate,
screenshot evidence, and a11y-tree reads are how "exercise it for real" is proven in a
browser. **Never re-run an unchanged test command "to be sure"** — the first full read is the
evidence; re-rolling adds nothing but doubt. **Deterministic scanners must not regress:** if
the repo has linters/analyzers, run them on the base commit and on the change, over the
changed scope, and save both counts in `evidence/`; the change must not report more
findings. "No regression" without the two counts is a claim, not a result.

**On the small route there is no `inspect` or `release`** (§7): when `verify` is green there,
it closes the task. It sets the task's `status` to `done` in `state.json` and the `index.md`
row from it (§2), and appends the feature's dated changelog entry (§13.1). On the full route,
it hands to `inspect` instead.

**A quarantined test is logged** in the task's `summary.md` under `Follow-ups:` as a defect to
investigate; marking it skipped in its own module is part of what the user approved.

**A pass-on-rerun is not green.** A test that fails then passes on retry is an intermittent
failure: name it, **propose** quarantining it to the user, and log it as a defect to
investigate — never re-roll a flaky suite until it happens to pass. A quarantine takes
effect only on the user's yes. If the quarantined test is **outside the changed scope**, it
does not block this task: `verify.md` names the exception, and everything else must pass. If
it is **inside** the changed scope, `verify` is not green until it is fixed, or root-caused
per Part B (the cause at its file:line) and the user accepts shipping with it. That
acceptance is recorded in `verify.md` and in `decisions.md`, with the cause's file:line and
`By: user`, so a resume and `release` can tell an accepted defect from an unexamined one.

**Exercise it, not just the unit tests.** Tests passing is necessary, not sufficient — run
the real thing the way a user would and confirm the observable success criteria from the
spec are met. A feature with green units that was never actually run is not verified.

**Independent test pass (multi-agent mode).** The builder verifying its own build shares the
builder's blind spots. When worker agents are available (§8), dispatch an independent tester
who **never saw the build session**, working only from `spec.md`'s success criteria: negative
cases, boundary abuse, and exploratory paths beyond the author's tests. Single-agent: run
that pass yourself from the spec alone, deliberately trying to break it.

If everything passes: mark `verify` done+validated in `state.json` and hand to `inspect`.
If anything fails: go to Part B.

**No execution capability (no shell / can't run the suite)?** There is no inline substitute
for actually running the tests, so do **not** claim a pass. Report plainly: *"Cannot verify
— no execution capability in this runtime; N tests written but not run."* Leave `verify`
un-validated and hand the user the exact command to run themselves. Never let "proceed with
the inline equivalent" (CONVENTIONS §9) become a fabricated green — for verify, the honest
degrade is to stop, not to guess.

## Part B — Root-cause before fix (when something breaks)

**The iron law: no fix before the root cause is found.** Steps 1–3 below are not advice —
until they are done, you do not propose a fix, and "I think it's probably…" is not a root
cause. This applies *especially* under time pressure: guessing feels faster and is the thing
that turns one bug into an afternoon.

**These thoughts mean stop — you are about to guess:**

| The thought | The reality |
|---|---|
| "This one's obvious, I'll just fix it" | Obvious fixes to unlocated bugs mask symptoms. |
| "Let me try changing this and see" | That's a search, not a diagnosis. Each try adds noise. |
| "It's urgent, no time for process" | Systematic is faster than thrashing. Always. |
| "It works now, close it" | If you can't say *why* it broke, you can't say it's fixed. |
| "The error message says X, so X is the bug" | The message is data, not a verdict (below). |
| "Can't reproduce it — I'll fix what looks wrong" | Unreproducible means gather more data, never guess. |

Stop. Do not start trying fixes. A guessed fix that happens to mask a symptom leaves the real
bug in place and adds noise. **If the failure comes from production, start from the
production evidence (§19)** — error-tracker entries, logs, and metrics narrow when it
started, who it hits, and what correlates, which is what makes a minimal repro findable.

```
1. Reproduce   Get a reliable, minimal reproduction. If you can't reproduce it, you can't
               fix it — narrow the input until it triggers every time. STILL not
               reproducible? Classify it: timing-dependent (add timestamps, widen the race
               with artificial delay, run under concurrency) · environment-dependent
               (diff versions/config/data vs. where it fails) · state-dependent (leaked
               globals/singletons — run in isolation vs. after other tests). For any

From saleh-alhaddad/itqan-engineering, MIT.

More from saleh-alhaddad/itqan-engineering