compliance

Files (1)

• SKILL.md

SKILL.md (first lines)

# Compliance — Adaptive Post-QA Hardening

Compliance is the final quality gate before archiving. It verifies that the work meets all stated requirements and standards. The checklist adapts to the task type.

## Deferral vs Inline-Ship Decision Heuristic

When a finding (Low / Medium severity) surfaces during compliance and a clean deferral target exists (a successor task that already owns the same surface), the default reflex is Path A: «document as caller-side contract, hand off to successor». This is the safe choice for high-blast-radius findings or those without a clean inline reduction.

But Path A carries a hidden cost: **caller-side contract is invisible closure debt**. The successor task carries the obligation, but the obligation is no longer linked to the original finding's context — the operator has to re-derive why the caller-side handoff exists when the successor task starts. If the successor expands into a multi-finding scope, the original handoff item silently grows in importance.

**Rule of thumb — prefer Path B (inline-ship) when ALL of the following hold:**

1. The fix is **≤100 LoC** of production code + tests (no architectural displacement).
2. The fix adds **≥2 behavioural tests** that exercise the previously-uncovered path.
3. The successor task has **no other depends-on** beyond this single obligation (so closing the handoff also closes a real downstream dependency, not just one of many).
4. The fix is **reversible by single-commit revert** (FB-6) and **does not change public API contracts** that other consumers depend on.

When any condition fails, Path A is correct. When all four hold, Path B is the lower-friction path — the operator gets one fewer cross-task handoff to track, and the successor task's scope is provably narrower at archive time.

**Operator override is the canonical escape hatch.** If compliance default-routes to Path A but the operator wants Path B (or vice versa), the override carries via `--focus-items` invocation of `/dr-do` — see Layer 3b FAIL-Routing CTA. Each override round MUST land in the init-task append-log (FB-2 / FB-5 traceability).

This distinction is operationally useful: operator overrides can move concrete
deferred findings from Path A to Path B in later `/dr-do` rounds, shrinking a
successor task, while small method-size findings can be closed inline without
creating an accepted-risk entry.

---

## Step 0: Detect Task Type

Read `datarim/tasks.md` and `datarim/activeContext.md` to determine task type:

| Type | Indicators | Checklist |
|------|-----------|-----------|
| **Code** | Modified source code files (.js, .ts, .py, etc.) | Software Checklist (7 steps) |
| **Documentation** | Modified .md files in docs, guides, README | Documentation Checklist |
| **Research** | PRD mentions research, analysis, literature | Research Checklist |
| **Legal** | PRD mentions legal, compliance, terms, policy | Legal Checklist |
| **Content** | Modified posts, articles, blog content | Content Checklist |
| **Infrastructure** | Modified Docker, CI/CD, IaC, deploy configs | Infrastructure Checklist |
| **Mixed** | Multiple types | Apply relevant checklists from each type |

---

## Software Checklist (7 steps)

### 1. Change Set & PRD/Task Alignment
- Compare diff against PRD requirements and task acceptance criteria
- Flag: unimplemented requirements, out-of-scope changes, missing edge cases
- When a task's backlog entry states a root cause explicitly (e.g. "fixed-port collision", "missing index", "race on shutdown"), verify the stated cause against a live repro **before** accepting the implemented fix. A misidentified root cause in the backlog leads to misdirected rounds: the fix targets the wrong axis, passes a weak gate, and the real defect survives. A short manual repro at re-validate time surfaces the mismatch cheaply.
- When an acceptance contract is superseded, inventory and compare the full text of every case in both the original and replacement versions, including cases expected to remain unchanged. In the Return-to-Source record, name every added, removed, or changed case and describe each material semantic difference (for example, changed pass criteria, scope, evidence, command, or threshold); use redacted excerpts if case text contains sensitive material. Preserve both contract versions. An unaccounted change or a relaxation without an explicit basis is a finding routed to the stage that changed the contract. This comparison does not authorize rewriting the contract, and static wiring tests do not perform the semantic review.

### 2. Code Simplification
- Check: functions >50 lines, deeply nested logic, duplicate code
- Apply Code Simplifier principles to recently modified code only

### 3. References and Dead Code
- Flag unused imports, variables, functions in changed files
- Verify no debug statements left in production code

### 4. Test Coverage
- Verify tests exist for all changed code paths
- Check: edge cases, error paths, boundary conditions
- **Quantitative-threshold AC enforcement.** When an Acceptance Criterion declares a numeric threshold (coverage %, latency budget, RPS, error rate, etc.), Compliance Step 4 (or Step 6 for runtime metrics) MUST execute the measurement tool and record the actual number. «Presumed met» is not an acceptable verdict for thresholded AC. If the measurement tool is absent on the host, install it (typically a one-line install via the project's package-manager-native tooling) before claiming PASS, or escalate the AC as BLOCKED back to `/dr-do`. When the AC genuinely cannot be measured yet (blocked on an external dependency, e.g. tooling not yet installed on the target host), defer it explicitly using the four-field waiver shape in `${DATARIM_RUNTIME:?}/templates/coverage-deferral-clause.md` (status, gating dependency, follow-up condition, timestamp + owner) rather than carrying it forward as unstructured prose. Source: prior incident — AC-7 (≥80% line coverage) was carried as «presumed met» through `/dr-qa` PASS_WITH_NOTES; Compliance had to install the coverage tool and measure (actual 91.5%) to close the gap.
- **Agentic Entrypoint Wiring + Live-Run gate.** When the task ships a service/daemon/cron/agent whose declared purpose is to invoke an external CLI/LLM/subprocess (`claude -p`, `gh`, `aws`, …) and act on its output, Compliance MUST NOT pass it on a mock-only suite. Verify per `${DATARIM_RUNTIME:?}/skills/testing/live-smoke-gates.md` § Gate 7: (a) the *real* entrypoint (`__main__`/systemd `ExecStart`/cron) actually reaches the declared function — call-graph grep + runtime probe, an orchestrator/lane reachable only from tests is dead-code-in-prod ⇒ **NON-COMPLIANT** → `/dr-do`; (b) one **live** run of the agent against the real tool was performed, with the captured real output + observed side-effect recorded in the report. A kill-switch-OFF exit-0 probe proves the agent does *nothing* and does NOT satisfy this gate; an `evidence_type: empirical` wish marked met on mocks alone is a hard finding. Source: prior incident — orchestrator + repair lanes fully unit-tested but never wired to `cli.main`; `claude -p` never ran in prod; QA passed on mocks and proposed archive; operator caught the unwired entrypoint.
- **Shell-harness Gate 9.** When a status-transforming shell layer supplies QA or compliance evidence, require the positive child control, negative child control, and any expected-red exact named assertion from `${DATARIM_RUNTIME:?}/skills/testing/live-smoke-gates.md` § Gate 9. An unrelated setup error must be classified `HARNESS_INVALID`, not caught. Missing attribution evidence is **NON-COMPLIANT** and cannot be carried as notes.
- **Unique-violation domain normalization.** When a change maps `23505`, `P2002`, or another unique-error family to a domain result, require an exact named constraint or exact normalized column signature plus runtime-shaped unit, real boundary, intended-positive, and unrelated-negative evidence per `${DATARIM_RUNTIME:?}/skills/testing/SKILL.md` § Unique-Violation Domain Normalization. Unknown explicit names fail loud without column fallback. Require real-database evidence when driver/schema metadata determines the classification. A mock-only or broad-family mapping is **NON-COMPLIANT**, never `PASS_WITH_NOTES`.
- **Ambient-reaper masking (timing-sensitive mutation gates).** When the change ships a timing-sensitive guard verified by a mutation-gate test (break the guard → assert the outcome changes), confirm the test isolates the guard from any background reaper/watchdog/supervisor that can independently produce the same effect. A mutation masked by an ambient reaper leaves the gate green on a dead guard while the reaper silently holds the invariant. Verify per `${DATARIM_RUNTIME:?}/skills/testing/SKILL.md` § Ambient-Reaper Masking in Timing-Sensitive Mutation Gates: the test must disable the reaper, assert on a guard-unique signal, or prove the effect preceded the reaper's fire window. Source: prior incident — a shutdown-guard mutation gate passed because a background reaper produced the same termination regardless of whether the guard fired.
- **Invariant-by-construction guard.** When a security or correctness invariant is enforced *structurally* rather than by an explicit runtime check — a type that makes the illegal state unrepresentable, a constructor/factory that rejects bad input, a schema or DB constraint, a guard clause on the only write path — but no test exercises it, the "it cannot happen by construction" claim is unverified: a later refactor can silently weaken the structure with no failing test to catch it. Require a mutation-checked regression test that proves the guard actually fires — break the structure (delete the constraint, invert the check, widen the type) and confirm a test goes RED. A structural invariant with no test that fails when the structure is broken is coverage theatre, not coverage. Source: prior incident — an invariant assumed safe "by construction" was silently weakened by an unrelated refactor because no regression test pinned the guard firing.

### 5. Linters and Formatters
- Run project linters and formatters
- Flag: lint errors, formatting inconsistencies
- **CI linter-scope check.** When the project's CI scopes the linter narrower than the full repo (e.g. `ruff check src/ scripts/` but not `tests/`), and compliance finds the unscoped directories dirty, recommend widening the CI scope to the full tree (`ruff check .` or explicit `src/ tests/ scripts/`) as part of the cleanup. Compliance is the wrong place to discover a narrowly-scoped linter — a narrow CI scope produces avoidable cosmetic notes in every QA cycle. Fix the scope at the source, not just the dirty files.

### 6. Test Execution
- Run the full test suite, report pass/fail counts
- **Pre-existing branch failure discrimination.** When tests fail in a CI-faithful run, check whether each failure was present before the task's commit: `git log --oneline <base>..HEAD -- <failing-test-file>`. If the violation predates the task (none of the task's commits touched the failing file), document it as "pre-existing branch issue, not introduced by task commit" and do NOT treat it as a task-scope blocking verdict. The task's own diff must be clean — failures from unrelated branch history or parallel-session dirty files are advisory, not verdicts.
- **Workspace-hygiene auto-classification (regression-invariant tests).** A common failure shape in a shared workspace is a *regression-invariant* test — a test asserting that some scope directory is gate-clean (e.g. "`skills/` scope is English-only", "`agents/` scope has no private IDs"). When such a test fails, do not hand-write a discrimination note each time. Run the deterministic git-based classifier with the failing test's scope directory:
  ```sh
  bash "${DATARIM_RUNTIME:?}/dev-tools/classify-bats-failure-scope.sh" \
       --repo <framework-repo> --base <base-ref> --scope <scope-dir> [--scope <scope-dir> ...]
  ```
  The classifier runs `git log <merge-base>..HEAD -- <scope-dir>` per scope. A scope with **zero** task commits in the range is labelled `pre-existing` (exit 0) — the failure is foreign / parallel-session noise; record the classifier's verdict verbatim and do NOT treat it as a task-scope block. A scope that **did** receive task commits is labelled `regression` (exit 1) — it stays a real, blocking regression and a real regression is never masked. The helper fails **closed** (exit 2) when a scope's git range cannot be evaluated, so an undeterminable scope is never auto-classified as foreign. This replaces the per-archive manual discrimination note for regression-invariant failures; the per-file manual check above still applies to failures that are not scope-shaped.
<!-- gate:history-allowed -->
Prior incident: two test failures traced to pre-task commits and foreign dirty hunks — neither introduced by the task under compliance review. Recurrence: the same hand-written discrimination note for a regression-invariant test whose scope had no task commits surfaced repeatedly across tasks until it was made deterministic via the classifier above.
<!-- /gate:history-allowed -->
- **Report-cited-SHA resolution probe (fabrication vs stale-clone).** When a QA or compliance report cites commit SHAs or merge-request numbers, do not trust them on faith nor reject them as fabricated without a probe. Run `git cat-file -t <sha>`; if the object is absent locally, run `git ls-remote origin` and `git fetch origin <branch>` read-only, then verify the diff against `FETCH_HEAD`. A SHA absent **both** locally and on the remote is a genuine fabrication finding — **NON-COMPLIANT**. A SHA absent locally but present on the remote branch is the expected stale-clone case in a remote-first project (the work was pushed from another machine and the local clone never fetched): fetch read-only and verify the actual diff — never block, never reject. Verify against `FETCH_HEAD`, do not check out, so a shared clone parked on another task's branch stays undisturbed.
<!-- gate:history-allowed -->
Prior incident: a QA report cited two commit SHAs invalid on the local clone (`git cat-file` reported them as unknown objects); both resolved on the remote feature branch and the implemented work was real — the local clone was simply stale in a remote-first workflow. Compliance fetched the branches read-only and verified the diffs against `FETCH_HEAD`.
<!-- /gate:history-allowed -->

### 7. CI/CD Impact Analysis
- Detect: new dependencies, changed env vars, new build steps
- Flag: breaking changes to build pipeline, missing migration steps
- **Stale-runtime advisory (non-blocking).** When the task touched a shipped script (`scripts/lib/*.sh`) or skill (`skills/*/SKILL.md`), the change is live only where it was committed. Run the shared detector and surface its output verbatim so multi-machine consumers are reminded to update each install: `bash "${DATARIM_RUNTIME:?}/dev-tools/check-stale-runtime.sh" --repo <framework-repo> --range <base>..HEAD`. Single source of truth (same script as `/dr-archive` Step 0.48); advisory only — it never gates the compliance verdict.
- **Stale-base merge-result gate (`git`-only).** Before reporting any apparent change in PR diff vs `origin/<base>` as a "regression", check whether the diff is a side-effect of `origin/<base>` advancing past the branch's merge-base rather than a branch-side edit. If `git diff <merge-base>..HEAD -- <file>` is empty for a file that nonetheless appears in `git diff origin/<base>..HEAD -- <file>`, simulate the actual 3-way merge: `git merge-tree $(git merge-base HEAD origin/<base>) HEAD origin/<base>`. If the simulated tree preserves the upstream change in question, the apparent diff is a no-op on merge — record this in the report and do NOT block the archive on a rebase requirement. Source: prior incident — a feature PR appeared to revert an upstream baseline-hardening fix that landed mid-flight; merge-tree simulation confirmed the fix was preserved by 3-way merge, deflating a needless rebase cycle.
- **Stale test-count classification (non-blocking).** When a commit message claims N/N tests pass but a live re-run of the test suite reports M/M with M > N, and the additional tests are verifiably present in the commit (grep confirms the test definitions exist in the committed files), classify the discrepancy as informational non-blocking — a normal polish-test addition committed after the run that produced the message. Do NOT treat it as a re-commit trigger or a compliance failure; record the live count (M/M) as the authoritative figure and note the original message count for traceability.
- **Plan-header base-SHA drift (non-blocking, annotate — do not rewrite history).** When a task was rebased *after* its plan was authored, the plan header's recorded base-SHA no longer matches the branch's actual merge-base. This is expected post-authoring drift, not a defect — do NOT raise it as a compliance finding and do NOT rewrite the plan header in place to "correct" the value. Reconcile inline non-destructively: append a short annotation recording the new base-SHA and the rebase (e.g. `base-SHA reconciled <old>→<new> after rebase on <date>`), leaving the originally-authored value intact for provenance. Rewriting the header destroys the record of what the plan was actually written against; an inline annotation preserves it while making the current base explicit. Source: prior incident — a rebased task's stale plan-header base-SHA was raised as a defect when it was benign drift; the correct move was a non-destructive inline reconciliation note, not a history rewrite.

### Loop-guard pre-emptive operator handoff (attempt 2 vs attempt 3)

`cta-format` § Loop guard escalates after **3 same-layer fails**. The default automatic re-run between attempts 1 and 2 is appropriate only when the verdict could plausibly change without external action — for example, when a flaky test was re-run, when an upstream timing dependency might have caught up, or when the operator may have acted between attempts.

When a probe set is **deterministic** (`gh pr view` PR states, `git rev-parse origin/main` HEAD, live `/health` endpoints, validator stdout) AND **state delta vs the previous attempt is empty across all probes**, attempt 2 carries zero information. The verdict cannot change without operator action.

In that case Compliance MUST, on attempt 2:

1. Capture the empty-delta finding in the v2 report (one-line delta table is sufficient).
2. Pre-emptively formulate a handoff question to the operator (FB-8 — surface only the question that actually blocks safe forward progress), **rather than** running attempt 3 with the same probe set.
3. Loop-guard counter remains at 2/3 until external action (operator or autonomous-by-delegation) produces a real state delta.

Anti-pattern caught by this rule: identical NON-COMPLIANT verdicts at v1 + v2 produced by re-running the same `gh pr view` / `curl /health` set 23 minutes apart with no merge in between. The runtime probes are cheap, but the compliance turn (context, narration, validator append-log) is not — and the operator gets a noisy «attempt 2 of 3» banner that obscures the actual blocker.

Source: prior incident — a multi-repo task ran compliance v1+v2 within 23 minutes; both returned identical Layer-4 NON-COMPLIANT verdicts with state delta ∅ on all 5 runtime probes (PR states, main HEAD, live `/health`, expectations validator). Attempt 3 resolved only when the operator explicitly delegated the blocking mechanical action under FB-1..FB-5 + autonomous-ops authorization, not via further probe re-runs. Provenance: see `documentation/how-to/evolution-log.md` for the archive entry that motivated this rule.

---

## Documentation Checklist

From Arcanada-one/datarim, MIT.

More from Arcanada-one/datarim