knowledge-check

Files (1)

• SKILL.md

SKILL.md (first lines)

# Knowledge Check Skill

Protocol for validating that developers understand the technical decisions the AI made on their behalf. The AI acts as a tutor — it quizzes, evaluates, explains, and logs results.

## When This Skill Is Loaded

- **After `/virtual-team:plan` approval** — quiz on architectural decisions and technical approach
- **Before `/virtual-team:pr` submission** — quiz on implementation patterns and code decisions
- **Standalone `/virtual-team:check`** — developer-initiated quiz on any current work

## Developer Settings

The knowledge check triggers automatically based on the developer's home settings.

**File:** `~/.claude/settings.json`

**Setting:**
```json
{
  "knowledgeCheck": "on"
}
```

**Values:**
- `"on"` — run knowledge checks at plan and PR checkpoints (soft block: warn on gaps, always proceed)
- `"strict"` — run knowledge checks as hard blocks (must pass to proceed)
- `"off"` or absent — skip knowledge checks entirely

**How commands use this setting:**
1. Read `~/.claude/settings.json` at the start of the check step
2. If `knowledgeCheck` is `"on"` or `"strict"`, run the check
3. If `knowledgeCheck` is `"off"`, absent, or the file doesn't exist, skip silently
4. The standalone `/virtual-team:check` command always runs regardless of settings (developer chose to invoke it)
5. `--auto` mode on `/virtual-team:plan` and `/virtual-team:pr` always skips the check (nobody is there to answer)

---

## Question Generation

### Post-Plan Questions (Architectural Focus)

Read the approved plan and generate questions targeting:

1. **Tech choices** — Why was this library/pattern/approach chosen?
2. **Tradeoffs** — What are the downsides of the chosen approach? What was considered and rejected?
3. **Ordering** — Why are the phases in this sequence? What breaks if you reorder them?
4. **Integration** — How does this change fit into the existing system architecture?
5. **Risks** — What are the identified risks and how does the plan mitigate them?

**Source material:** The plan document, the feature spec, `stack.md`, any referenced decision records.

### Pre-PR Questions (Implementation Focus)

Read the branch diff and the implementation plan, then generate questions targeting:

1. **Patterns** — Why was this code pattern used? What existing code does it follow?
2. **Data flow** — How does data move through the new/modified code?
3. **Edge cases** — What edge cases are handled? How?
4. **Error handling** — What happens when things fail? What's the recovery path?
5. **Testing** — Why were these specific test scenarios chosen? What's covered and what isn't?

**Source material:** `git diff <base>...HEAD`, the implementation plan, the feature spec, the test files.

### Standalone /virtual-team:check Questions

Determine what the developer is currently working on:
1. Load the backlog skill and call **`list(status=doing)`** and **`list(status=implemented)`** to find items in progress
2. Find the associated plan and feature spec
3. Check the current branch diff
4. Generate questions appropriate to the current stage:
   - If pre-implementation (plan exists, no code yet) → architectural questions
   - If mid-implementation (code in progress) → mix of both
   - If post-implementation (code done, pre-PR) → implementation questions

---

## Question Format (Mix)

Generate **3-5 questions** per checkpoint.

### Multiple Choice (2-3 per checkpoint)
For factual/recall questions where there's a clear correct answer.

**Structure:**
```
**Q1:** [Question about a specific decision from the plan/code]

A) [Correct answer]
B) [Plausible but incorrect alternative]
C) [Plausible but incorrect alternative]
D) [Plausible but incorrect alternative]
```

**Guidelines:**
- The correct answer should come from the actual plan/code decisions
- Wrong answers should be plausible alternatives that a developer might consider
- Don't make wrong answers obviously absurd — they should test real understanding
- Shuffle the position of the correct answer (don't always put it as A)

**Examples of good multiple choice questions:**
- "Which existing pattern does the plan reference for the new API endpoint?"
- "What database approach was chosen for the notification preferences?"
- "Which phase must complete before the frontend work can begin?"

### Open-Ended (1-2 per checkpoint)
For critical decisions where the developer needs to articulate reasoning.

**Structure:**
```
**Q4 (open):** [Question requiring explanation]

*Key concepts your answer should touch on:*
(hidden until after the developer answers)
```

**Guidelines:**
- Target the single most important architectural or implementation decision
- The question should require understanding the WHY, not just the WHAT
- Prepare 3-5 key concepts that a good answer should cover (used for evaluation)

From ovargas/virtual-team, MIT.

More from ovargas/virtual-team