evidence-gates

Files (6)

• SKILL.md
• architecture-checks.md
• coverage-and-anti-patterns.md
• evidence-kinds.md
• review-dimensions.md
• traceability.md

SKILL.md (first lines)

# Evidence Gates

**The evidence-not-assertion guardrail** says a guardrail clears with artifacts
and command output, never a claim. "The tests pass" is not a
gate-passing statement on any delivery approach; the *recorded test run* is. This skill
covers what counts as evidence, the checklists for each review dimension, and -
just as important - the line between what is *checked* and what is *assessed*.

## Guardrails are checked; strategies are assessed

This is the distinction the whole gate model rests on. The two kinds of
governance are checked two different ways, and conflating them is the failure
mode the evidence-not-assertion guardrail exists to prevent.

- **Guardrails are *checkable*.** They are cleared with evidence - a test ran, a
  scan passed, a human approved, the artifact exists. A guardrail clears or it
  does not, and "clears" means there is recorded output a skeptic could re-run. A
  failed guardrail is a no-pass. A guardrail beats any strategy.
- **Strategies are *assessed*.** They are the reviewer's judgment - is this the
  simplest thing that works, does it follow the team's engineering or voice
  strategies, was a departure recorded. There is no pass/fail artifact for a
  strategy; there is an honest opinion, and it must be **labeled as judgment**,
  not dressed up as evidence. A strategy not followed is a note and a
  conversation, not an automatic gate failure.

Naming them differently is what stops a judgment call being presented as a
hard gate, and stops a hard gate being waved through on an opinion. When you
write the `governance` dimension into `verification-report.md`, the guardrail
findings and the strategy assessment go in clearly separated - evidence on one
side, judgment on the other.

## Evidence vs. assertion

| Assertion (does not clear a guardrail) | Evidence (clears a guardrail) |
|---|---|
| "The tests pass." | The recorded test-runner output: counts, the green summary, the command that produced it. |
| "Coverage is fine." | The coverage report, with the number, against any project coverage-floor guardrail. |
| "It's fast enough." | The benchmark output against any project performance-budget guardrail. |
| "No regressions." | The regression run, before-and-after, showing nothing green before is red now. |
| "It's secure." | The scan output; the dependency-CVE result where a project security guardrail needs it. |
| "Every claim is backed." | `launch-readiness.md` with each claim's backing scenario and that scenario's passing status. |

The test is simple: **could someone who does not trust you check it from what
you recorded?** If yes, it is evidence. If they would have to take your word, it
is assertion.

## Records that meet a Definition of Done item

An item of the Definition of Done is met by a record in the manifest, never by a
tick on the page: `compass gate pass ... --meets <item>` or `compass evidence
add ... --meets <item>` records evidence, `compass follow-up add FU-<id>
--defers <item> --summary "..."` carries the item to ship, and a person ticks
an item by typing `/compass:approve` for a `compass evidence request --check
<item>`, or with `compass evidence approve --check <item>` at a terminal. `dod-evidence-typed`
fails an item with no record. `compass issue render` writes the list and each
item's state into the verification report.

## Records that also move a document

`compass tdd-red` and `compass acceptance record` are the first records of the
implement stage. Each one moves the documents of earlier stages that are still
`draft` and have a path: to `approved` (with `approved_by` naming the command),
or to `awaiting-approval` when the document has a human check. A person
approves that document by typing `/compass:approve` for a `compass evidence
request --artifact <kind>`, or with `compass evidence approve --artifact <kind>`
at a terminal. That approval is recorded as `artifact-approval`, which no gate accepts and the
sign-off check of `G5` never reads.

`compass acceptance record --release-note "<text>"` also stores the issue's
release line in `release_note`, once the command passes. A failed run stores
nothing, and the flag is refused on a completed or closed issue
(`docs/release-line.md`).

## Properties of real evidence

- **Reproducible** - it includes the command, so it can be re-run.
- **Current** - it is from this change, this run, not a remembered earlier one.
- **Complete** - it shows the whole relevant output, not a hand-picked green
  line. A run with skipped tests is not a green run; show the skips.
- **Honest about gaps** - where evidence is missing or a scenario could not be
  run, that absence is itself reported. A gap surfaced is a finding; a gap
  hidden is a lie the gate cannot catch.

## The kinds of evidence

A vocabulary - commit, acceptance, and the rest - in
`skills/evidence-gates/evidence-kinds.md`.

## How the two halves of the verify stage split

- The **Verifier** does the mechanical half: runs the scenarios as the
  acceptance suite, runs the TDD suite, runs regression, and **writes each
  capture to an evidence file and links it** from
  `verification-report.md`. It establishes what is true.
  **Link, do not paste.** A report that reproduces its evidence stops being
  something a person reads - it becomes a transcript with a summary at the top.
  The record belongs in `evidence/`; the report cites it and states what it
  shows.
- The **Reviewer** does the judgment half: applies the review dimensions to
  that evidence and the change, and renders pass / no-pass. It decides whether
  what is true is good enough.

Judgment rests on evidence. The Reviewer never passes a gate without the
Verifier's artifacts in hand.

## The review-dimension checklists

One checklist per dimension, in `skills/evidence-gates/review-dimensions.md`. Read the one for the dimension you are applying - the delivery approach says which the issue carries.

## Passing a gate - the procedure

1. Read `delivery-approach.md` for the gate set and the dimensions in play.
2. Verifier: run everything the dimensions need, write the raw output to an
   evidence record and link it from `verification-report.md`, flag every gap.
3. Reviewer: walk each dimension's checklist against the evidence and the
   change. Record per-dimension **pass** or **no-pass with the specific reason**.
4. Before the verification report is complete, make sure that every required
   review dimension passed with evidence attached, that every gate in the
   delivery-approach record and every immovable gate is green, and that no

From ayeo-io/compass, Apache-2.0.

More from ayeo-io/compass