roi-verify

Files (1)

• SKILL.md

SKILL.md (first lines)

# roi:verify — verdict at the verify gate

This skill records a verification verdict for a run paused at `verify_gate`.
It owns one stage: **read evidence → judge against criteria → record verdict
→ next-step pointer**.

The verdict is one of:

- `pass` — evidence satisfies the brief's success criteria. Mission may
  proceed to `roi:publish`.
- `partial` — some criteria met, others outstanding. Run does not advance to
  publish; operator must `roi:edit` or `roi:go` for the missing slice.
- `fail` — evidence contradicts criteria, or critical oracles failed.
- `inconclusive` — evidence is incomplete or unverifiable from this session.

**Boundary:** this skill does not modify plans or evidence. It only records
a verdict on what already exists. To re-record evidence, use `roi:go`.

**Identity binding:** every recorded verdict snapshots the mission-brief
revision and content digest it was judged against. If the brief is later
revised with different semantic content, the verdict appears in
`status_get`'s `summary.stale_verdicts` and no longer stands as a current
approval — re-run `roi:verify` against the revised brief.

**Agentic missions:** Verdicts must cite **falsified or satisfied properties**
(acceptance criteria / `verification_targets`), not "touched every file in
the plan." Read
[`references/agentic-plan-strength.md`](../references/agentic-plan-strength.md).

## Inputs

1. **Mission ID** required.
2. **Run ID** required — the run may be paused at `verify_gate` or at an
   orientation-gated spec/quality review task.
3. **Optional flags** — see Flags table below.
4. **Current verification orientation checkpoint** — required for the exact
   semantic scope being judged, bound to current plan revisions, owner seams,
   proof obligations, source-contract requirements, and live-state identity.

## Procedure

1. Read mission state and the run's evidence:

   ```bash
   node roi/scripts/lifecycle.mjs status_get '{"mission_id":"<id>"}'
   node roi/scripts/lifecycle.mjs evidence_list '{"mission_id":"<id>","run_id":"<run_id>"}'
   node roi/scripts/lifecycle.mjs task_list '{"run_id":"<run_id>"}'
   node roi/scripts/lifecycle.mjs orientation_list '{"mission_id":"<id>"}'
   ```

   Treat `partial_verification_eligible`, substantive/open counts, percentages,
   scores, and ContextPack TTL as telemetry only. None selects the verification
   scope or proves that it is sufficient.

2. For each plan in scope, judge the **latest** `source: roi:go`
   verification row (newest `captured_at`; `created_at` only on legacy rows);
   ignore stale pass/fail from earlier runs. Ignore stub-only evidence whose
   `LOCAL_EXECUTION_COMPLETED` from local `implement` tasks — that is not
   implementation proof.

3. For each `roi:go` verification with `result: pass`, confirm
   `content.implementation_proof` exists (diff or `paths_touched`) and that
   oracles were not vacuous (no `[no tests to run]`). Treat pass without
   proof as **not substantive** — the verdict should be `partial` or `fail`,
   not `pass`. Likewise, an oracle that is **under-fit** for its declared
   behavior class — one that cannot falsify the class it claims to prove (a
   smoke oracle for a refactor-equivalence or regression claim; an exact-match
   oracle for a relation-only claim), or one whose behavior class cannot be
   identified at all (on surfaces carrying structured target rows, a `proves`
   that names no class; on ROI's bare command strings, where **no `proves` field
   exists** per decision 055 D1, a plan whose prose states no falsifiable claim)
   — is **not substantive** even when it exits 0:
   downgrade to `partial` or `fail`, not `pass`. Oracle fitness is
   reviewer-judged against the behavior-class → minimum-fit map in
   [`references/oracle-patterns.md`](../references/oracle-patterns.md); it is a
   refinement of the vacuous-proof rule above, not a separate gate.

4. For each source-derived plan (`requires_source_contract_check: true` or
   non-empty `source_contract_refs`), confirm the latest passing `roi:go`
   row includes `implementation_proof.source_contract.source_refs` and
   `coverage[]`. Each coverage row must name the source requirement and map
   it to `verification_target`, `manual_review`, or `not_applicable` with
   proof or reason. `manual_review` proof must cite an inspectable evidence
   artifact; local repo-relative evidence paths must exist when the helper can
   resolve them. `source_refs` must include the plan's `source_contract_refs`,
   and `verification_target` rows must cite an actual plan target. Treat missing
   or mismatched coverage as **not substantive** even when oracles passed.

5. For high-stakes source-derived missions, pass
   `require_independent_source_contract_review: true`. That gate blocks `pass`
   unless source-contract proof confidence is `independent_reviewed`, which
   requires explicit independent-review metadata in the `roi:go` proof bundle.

6. Before each verifier, including every target run through `run_oracles` and
   each manual review obligation, call `orientation_refresh` with
   `action_class: verifier_execution`. Bind the exact verifier, proof
   obligation/targets, current plan revision and identity, run, and the
   matching spec-review, quality-review, or verify-gate `task_id`. Refresh each
   open verifier task separately. Include live-state identity, observed owner-seam ids, and
   checked preconditions. Re-read with `orientation_get`; do not issue any
   verdict from a
   stale, blocked, missing, differently scoped, or pre-`roi:go` checkpoint.

7. Form an explicit verdict and write reasoning into `notes`. The notes
   field is the durable record of why this verdict was recorded.

8. Persist:

   ```bash
   node roi/scripts/lifecycle.mjs verify_evaluate '<json>'
   ```

   Where `<json>` includes:

   ```json
   {
     "run_id": "<run_id>",
     "verdict": "pass",
     "notes": "All in-scope plans show substantive roi:go evidence; success criteria 1-3 satisfied per oracle output.",

From instagrim-dev/roir, MIT.

More from instagrim-dev/roir