distill

Files (1)

• SKILL.md

SKILL.md (first lines)

# /distill — Extract Rules from Observed Corrections

When you and an agent collaborate on an artifact, the diff between the agent's draft and the version you actually shipped is a learning corpus. This skill takes that diff, classifies the patterns of correction, and proposes candidate rules for the relevant skill file.

**This is the manual half of the self-improving-skill loop.** The agent-watching-edits-automatically half (auto-watcher) is deferred pending Anthropic's native skill-learning features. The *doctrine* — that corrections are a learning corpus — is provider-neutral and ships now.

**Promotion gate applies.** Candidate rules go to `~/.claude/candidate-rules.md` on first sighting. Promotion to permanent skill doctrine happens on the second sighting per the rule promotion gate (see `~/.claude/skills/wrapup/health-check-guide.md`).

---

## Input

`$ARGUMENTS` should be one of:

1. **Two file paths**: `<draft-path> <final-path>` — explicit before/after files
2. **Git refs**: `<commit-or-branch>..<commit-or-branch>` <file-path> — use the diff between commits
3. **"last /ship"**: when invoked from `/ship` or right after a commit, automatically diff `HEAD~1..HEAD` for the staged files
4. **A description**: "Claude drafted X; I edited it to Y" — agent reconstructs the relevant artifacts and diffs them

Optional flag:
- `--target-skill <skill-name>` — propose rules for a specific skill (default: agent infers which skill produced the draft)

---

## Process

### Step 1: Establish the diff

Render the diff between draft and final in a structured way:
- Lines/blocks removed (what the agent *did* that the user un-did)
- Lines/blocks added (what the agent *missed* that the user filled in)
- Lines/blocks modified (what the agent *got partly right*)

If the diff is tiny (< 5 lines changed) or uniform (only whitespace/typos), report "no signal" and exit. Don't manufacture patterns from noise.

### Step 2: Classify correction patterns

For each non-trivial removal/addition/modification, classify it. Common pattern types:

- **Voice/tone:** Agent used X phrasing; user replaced with Y. (e.g. AI-tell phrases caught by the de-AI-ism pass — "load-bearing," "let me gently push back")
- **Specificity:** Agent abstracted; user got concrete. (e.g. "users" → a specific handle/name, "improve performance" → "reduce p95 latency below 200ms")
- **Hedging:** Agent qualified; user committed. (e.g. "this should work" → "this works")
- **Structure:** Agent used wrong format. (e.g. paragraph → numbered list; missing headers; wrong code-fence language)
- **Scope:** Agent did more than asked. (e.g. unsolicited refactor; defensive error handling for impossible cases)
- **Specificity-of-attribution:** Agent generic-attributed; user named the source. (e.g. "research suggests" → a named, dated citation)
- **Workflow-specific:** Agent missed a project convention. (e.g. forgot a known data-file encoding gotcha; used `cd <dir>` before git commands)

Pattern-count threshold: a pattern needs to appear **at least twice in this single diff** to be worth proposing as a rule. Once is noise; twice is signal.

### Step 3: Identify the target skill (if not specified)

Without `--target-skill`, infer:
- If the draft was a CLAUDE.md addition → propose rule for CLAUDE.md
- If the draft was a commit message → /ship
- If the draft was a code review comment → /review or /bug-hunt
- If the draft was prose for external publication → the pre-publish-critical-response guide
- If the draft was a plan → /plan-task
- If unclear, ask the user.

### Step 4: Propose candidate rules

For each pattern that fired ≥2 times in the diff, write a candidate rule entry. Format:

```
### YYYY-MM-DD — <short rule>
- **Pattern observed:** <how it showed up in the diff — give 2-3 concrete examples from the actual edits>
- **Proposed rule:** <one-sentence behavioral rule that would prevent the correction>
- **Proposed home:** <specific skill file / CLAUDE.md / guide>
- **First sighting context:** <which artifact, which project>
- **What would promote:** <what specific second sighting would justify codifying>
```

Append all candidates to `~/.claude/candidate-rules.md`.

### Step 5: Check for second-sighting matches

Before exiting, scan `~/.claude/candidate-rules.md` for prior entries whose **proposed rule** matches what was just observed. If a prior candidate matches, this is the **second sighting** — promote it now:

1. Read the prior candidate's "Proposed home"
2. Append the rule to that target file (with a "Promoted from candidate-rules.md after second sighting on YYYY-MM-DD" note)
3. Remove the original entry from candidate-rules.md
4. Report the promotion to the user

If no second sighting fires, exit with: "N candidate rules logged. Promotion happens when these patterns appear again."

---

## Output

Always print a concise summary to chat:

```
DISTILL SUMMARY
- Artifact: <what was diffed>
- Patterns detected: <N>
- Candidates logged: <N> (appended to candidate-rules.md)
- Promotions this run: <N> (matched second sightings)
- Top pattern: <one-line description of the most-fired pattern>
```

---

## When This Skill Triggers

- **Automatic at /ship** (intended): after a commit lands, /ship can call /distill on the agent's last draft vs. the committed version. (Currently /ship doesn't auto-call /distill — that's a future wiring.)
- **Manual after a session** where you noticeably edited Claude's output: invoke `/distill` with two file paths
- **From /wrapup** when reflecting on what got edited and what didn't
- **On demand** when you suspect a recurring correction pattern but haven't named it

**Don't use this skill when:**
- The diff is trivial (typos, whitespace, single-word swaps) — no learning signal
- The corrections were the user's own change of mind (not agent error) — wrong direction of learning
- The artifact was experimental / one-off — corrections won't generalize

---

## Latent / Deterministic Split (per CLAUDE.md rule)

| Step | Latent / Deterministic |

From griffinhilly/claude-code-synthesis, MIT.

More from griffinhilly/claude-code-synthesis