review-verification

Files (1)

• SKILL.md

SKILL.md (first lines)

# Review Verification

Check an existing review file finding by finding, so that whoever acts on it next works
only from claims that hold. A wrong finding costs more than a missing one: it becomes
an issue, then a pull request, before anyone notices.

## Scope

- The only file you change is the review file itself. Do not create branches, commits,
  issues, or pull requests: the review file is a scratch input, not a deliverable, so a
  general rule to commit finished work does not cover it. Instruction files that say
  what to do with review files do.
- Throwaway probes and environments go in a `probes/` subdirectory of the scratch
  directory the agent instruction files name, otherwise `.scratch/probes/` at the
  repository root, added to `.git/info/exclude` when git does not already ignore it.
  Remove them at the end.
- Record what you verified against at the top of the file: the commit SHA, or the
  dependency version, installed in an isolated environment rather than the project's
  own one.

## 1. Inventory

List every item the file asserts: numbered findings, quick wins, table rows, rejected
proposals, migration mappings. Keep the list, so you can check at the end that every
item got a verdict.

## 2. Verify

- **P0 and P1:** read the code path and prove or disprove the claim with a throwaway
  script or test. Name the triggering input and the observed result.
- **P2 and P3:** read the code. Probe when it is cheap or the claim is surprising.
- **Proposed fixes:** check them too. A correct finding with a fix that would break
  something else needs a corrected fix, not just a verdict.
- With more than about 40 items, split them by area across parallel reviewers that all
  use the verdict format below, then spot-check their false positives yourself.

## 3. Annotate

Under each item, add one line with the verdict (Confirmed, Partially confirmed, or
False positive) and its evidence: `file:line`, or the probe you ran and its output.

```markdown
**Verdict:** False positive. `cache.py:88` closes only caches the client created.
```

Choose the verdict for the claims, not just the headline. If a finding combines a real
validation gap with a consequence that does not occur, use Partially confirmed and
name the failed sub-claim. Use False positive when the alleged defect is absent,
including behavior the project documents and tests as intentional. Keep the original
text so the reader can see what changed.

For an older review, separate a claim that never held from one that the project has
since fixed. Use False positive for a finding that no longer holds at the requested
revision, but add "fixed since the reviewed version" and cite the change or old and
current code. Do not imply the original reviewer was wrong.

## 4. Add omissions

Search for omissions before concluding there are none. Within the review's scope,
follow the paths you inspected from input validation through side effects and error
handling. Check failure paths the findings did not cover: invalid inputs, exceptions
from external components, and failures after an operation has changed persistent
state. A successful happy-path probe does not establish that these paths are safe.

Add important problems you confirm as N1, N2, … in the same format as the findings,
with evidence of the same standard. Do not invent an item to fill the list. If you
find none, state which adjacent paths you checked.

## 5. Report

Reply in chat with:

- a table of counts by verdict;
- each false positive with a one-line reason;
- the added items;
- what you did not run again, such as the baseline checks.

Reconcile the counts with the inventory and the verdicts in the file before replying;
do not count a fixed finding as still holding.

From jmfontaine/agent-skills, Apache-2.0.

More from jmfontaine/agent-skills