qp-run

Files (1)

• SKILL.md

SKILL.md (first lines)

# qp-run — Automation (execute → verdict → mask)

Input: `qp/plan/tcs.tsv` + the adapter/engine lines from `qp/plan/inventory.md`. Output dir: `qp/run/<YYYYMMDD-HHMM>/`.

## 0. MTM state — decide once, write it first
`#mtm:` is the first line of `verdicts.tsv`:
| value | when |
|---|---|
| `ACTIVE` | a plan document was given **and** you opened it (path recorded) |
| `UNAVAILABLE` | no plan document — expected values come from the inventory; the branch *differs from plan* is **unreachable** and the vocabulary collapses to AS_PLANNED / CODE_DIFFERS |
| `FAILED` | a plan document was given but could not be read — say so; do not fall back silently |
| `DISABLED` | the user turned it off (reason recorded) |
`UNAVAILABLE` and `FAILED` are different events; never merge them.

## 1. Precondition first — attribution, not blocking
Before step 1 of each TC, establish its `precondition`. Navigation you can do (`at:/about`) you do; state you cannot create (`logged-in`, `has-an-order`) you mark `UNMET_UNEXECUTABLE`. A step-1 failure under an unmet precondition is **`BLOCKED`**, not `FAIL` — you never reached the screen, so it is not a claim about the app. Only step 1 is re-attributed; later steps failed after arrival.

## 2. Execute each step, observe before and after
web: accessibility snapshot before → action → snapshot after. desktop: screenshot before/after. Save each observation as `<tc_id>_step<N>_{before,after}.{md,png}` and append its stable id from `bash plugins/fh-qp/scripts/qp_tools.sh screen-id <file>` (a content hash — Playwright MCP ref tokens change every navigation, so a raw file hash is NOT stable; measured 2026-09-05) to `evidence.tsv` (`tc_id<TAB>step<TAB>screen_hash`) — this file is what `qp-regress` reads.

## 3. Verdict row per step — `verdicts.tsv` (7 tab-separated fields)
```
tc_id	status	branch	closure	verb	assertion	expected_source
```
- `status` ∈ `PASS FAIL BLOCKED AMBIGUOUS` (AMBIGUOUS = could not map the step to one element: 0 candidates or a tie).
- `branch` ∈ `AS_PLANNED DIFFERS_FROM_PLAN CODE_DIFFERS NONE`. `DIFFERS_FROM_PLAN` only under `#mtm: ACTIVE` and only with `expected_source=PLAN_DOC`. `CODE_DIFFERS` needs `expected_source` `CODE` or `PLAN_DOC`. `NONE` only for BLOCKED/AMBIGUOUS.
- `closure` = `MACHINE` **only if** `assertion` names the check that closed it (`text-visible:About`, `url==…`, `element-count==3`). An empty assertion with `MACHINE` is invalid — that is the muscle-not-skeleton hole this field exists to close. Everything you decided by reading a screenshot is `JUDGMENT`.
- Validate before you finish: `bash plugins/fh-qp/scripts/qp_tools.sh mtm-check verdicts.tsv` must print `OK`; `run-verbs verdicts.tsv` must print `OK` (≥1 click/input actually executed).

## 4. Mask, then save — original 0
Every text artifact goes through `qp_tools.sh mask <in> <out>` and only `<out>` is kept; delete `<in>`. `RESIDUE` (rc 5) → do not keep the file; report which artifact. Typed secret values (from the profile's `*_env`) are read from the environment at type-time and never written. Screenshots that show a filled password field are kept only if the field is masked in the image (crop or overlay); otherwise drop the screenshot and say so.

## Done When
1. `verdicts.tsv` passes `mtm-check` and `run-verbs` · **[mandatory-pass]** — rc 0 both.
2. `evidence.tsv` has ≥1 row per executed TC · **[measured]**.
3. Masking: `grep -cE '@|Bearer|eyJ' <saved artifacts>` is 0 outside `__REDACTED_*__` markers · **[measured]** — the lane reproduces it on a known-dirty fixture.
4. Every BLOCKED verdict names its unmet precondition; every JUDGMENT closure says what was read · **[judged — pairing: `qp-regress` re-runs the same TCs and flags verdicts that flip without a DOM change; a second session reviews BLOCKED rows against the before-snapshots]**.

## Independently executable
Yes, given a `tcs.tsv` in the documented shape (hand-written is fine) and an adapter.

From chrono-meta/forge-harness, MIT.

More from chrono-meta/forge-harness