quiz-diff

Files (1)

• SKILL.md

SKILL.md (first lines)

# Quiz Diff

Test the **human's** understanding of a diff — the inverse of code review.
Code review asks "is the code right?"; quiz-diff asks "does the human hold a
correct mental model of it?" AI-assisted work erodes that model silently
("comprehension debt"): tests stay green while understanding decays.

Evidence base: retrieval practice beats passive explanation; pretesting (answer
first, corrective feedback after) improves encoding; real comprehension spans a
program model (control/data flow) AND a situation model (rationale/goals)
(Pennington). Developers who only delegate score far lower on comprehension
than those who engage conceptually.

Prior art acknowledged: `Ciucky/no-numb` (session-gating quiz plugin, last-turn
scope, MCQ), `RonaldSit/comprehension-debt` (own-words repay/confirm CLI).
This skill differs deliberately: finished branch/PR scope, advisory not gated,
free-response with evidence-validated grading, fingerprinted gap records.

## Hard rules

1. **Quiz before explaining.** Never summarize or walk through the diff before
   the questions — pretesting only works if the attempt comes first.
2. **Free response by default.** No multiple choice unless the user asks for
   "quick mode" — MCQ tests recognition and cues answers. Short typed answers
   reveal real gaps.
3. **Validate every question before asking it** (see Item validation). A
   question you cannot ground in evidence is a question you must not ask.
4. **Lock all answers before teaching.** Collect answers to every question
   first; explanations for Q1 must not leak answers to Q2.
5. **"I don't know" is a first-class answer** — route straight to teaching,
   never shame.
6. **Advisory, never blocking.** Report the outcome; do not gate commits,
   pushes, or PRs on it.
7. **Honest claims only.** A pass does not prove understanding; say "no gaps
   surfaced", not "understanding verified". Call it "the selected diff", not
   "AI-written code" (provenance is usually mixed).

## Flow

### 1. Resolve scope — announce before asking

Default (pre-PR ritual): the merge-base of the real target branch through the
current worktree — committed + staged + unstaged + *selected* untracked files.
Explicit user scope always wins (staged-only, last N commits, a phase's
recorded range, named files).

Use the scope recipe in `code-review` → **Inputs** as-is. It resolves the
actual base (an explicit PR or user-named target, then `origin/HEAD`, then
`main`/`master`, then fails rather than guessing — and never the branch's own
`@{upstream}`, which on a pushed feature branch hides every pushed commit),
makes you choose untracked files deliberately, and fingerprints the result with
`git hash-object` rather than `sha256sum`, which does not exist on macOS.

Two of its rules bind harder here, because this fingerprint gets persisted and
read back later:

- **The fingerprint must cover everything you quizzed on.** Hashing only
  `git diff base...head` ignores staged, unstaged and untracked work, so a gap
  record can read as current long after the code it describes moved. Emit every
  part your resolved scope actually covers.
- **Never hash or enumerate the whole untracked set.** It holds credentials,
  unrelated experiments and other agents' in-flight work. Only the files you
  deliberately selected enter the fingerprint, and the record keeps their paths,
  never their contents.

Announce: the resolved base and which step resolved it, endpoints (SHAs), file
count. Capture `base SHA`, `head SHA` and the fingerprint for the gap record.

Skip only mechanically-recognizable noise: whitespace/formatting-only,
typo-only, generated artifacts, pure version bumps. Do NOT auto-exempt tests,
docs, or config — tests encode contracts and mocks; config carries
consequential decisions. A zero-question result is legitimate when no valid
item exists — say so and stop.

### 2. Select at most 3 concepts by semantic risk

Read the changed code PLUS relevant unchanged context (callers, types, config,
tests) — never reason from the patch alone. Rank candidate concepts by
semantic impact: blast radius, boundary behavior, security/data effects,
novelty, undocumented rationale. Complexity is one signal, not the ranking —
a one-line auth default or timeout change can outrank a dense helper.

### 3. Build an internal item spec per question (do not show it)

For each question, record before asking:
- the construct tested (which concept, which model — program / situation / transfer)
- expected answer + accepted variants
- evidence that uniquely determines the answer (file:line at CURRENT revision)
- the likely misconception the question probes

**Item validation gate:**
- Behavior questions: evidence must uniquely determine the answer.
- Rationale questions ONLY when the rationale is on record (request, ADR,
  comment, test name, session context). Otherwise ask "what trade-off does
  this implementation make?" and accept multiple defensible answers.
- Reject ambiguous items; replace or drop them.

### 4. Ask — one at a time, three-question default shape

1. **Program model**: predict/trace actual control or data flow
   ("Given X empty, what executes and what is returned?")
2. **Situation model**: explain a supported trade-off or design constraint
   ("What does this approach trade away vs the obvious alternative?")
3. **Transfer**: predict a failure, boundary, or maintenance consequence
   ("What silently breaks if Y is removed/changed?")

2 questions for a single small concept; 4 for a genuinely multi-concept diff;
5 only on explicit "deep mode". Never pad. No hints, no citations in the
question stem (they leak answers — cite in the teaching phase).

### 5. Grade against the stored rubric

Four outcomes per answer: `correct | partial | incorrect | invalid`.
`invalid` = the user challenged the question and the challenge holds (evidence
ambiguous, question wrong) — treat as a defect in the quiz, not a miss.
Re-check challenged items against the code before ruling.

### 6. Teach only the gaps

From clemens33/dotfiles, MIT.

More from clemens33/dotfiles