dev-record

Files (10)

• SKILL.md
• agent-report.sh · hooks/
• extract-session-metrics.sh · hooks/
• finalize-session.sh · hooks/
• hooks.json · hooks/
• record-prompt.sh · hooks/
• record-tool-call.sh · hooks/
• record-tool-result.sh · hooks/
• install.sh
• plugin.json

SKILL.md (first lines)

# Dev Record

Run the action specified by `$ARGUMENTS` (either `setup` or `status`).

Passive recording of agent activity during Claude Code sessions. Captures what was planned, what happened, what was decided, and where the agent deviated.

> **Note**: This is a record-only skill — it stores raw data but does not compute trends or analysis. Consumers (other skills, scripts, or humans) interpret the records.

> **Auto-loaded hooks**: Running `setup` registers hooks directly in the project's `.claude/settings.json`. No `--plugin-dir` flag required — hooks fire automatically for every session in the project.

## Actions

### `setup`

Initialize the current project for dev-record.

Run `./install.sh` from this skill's directory.

The script:
1. Adds `audit/ops_record/` to `.gitignore`
2. Appends agent self-reporting guidance to `CLAUDE.md`
3. Creates `audit/ops_record/` and `audit/dev_record/` directories
4. Installs `audit/agent-report.sh` helper for agent self-reporting
5. Registers hooks in `.claude/settings.json` — hooks fire automatically from this point on, no `--plugin-dir` needed

### `status`

Report the current state of dev-record in the project.

1. Check `.gitignore` includes `audit/ops_record/`
2. Count `.json` summary files in `audit/dev_record/` — report number of recorded sessions
3. Read the most recent `.json` summary file (by filename sort) and display it

---

## Reference

### What Gets Recorded

**Primary records** (captured automatically by hooks):

| Record | Hook Event | What it captures |
|--------|------------|------------------|
| Human input | `UserPromptSubmit` | Every user prompt verbatim |
| Agent decisions | `PreToolUse` | Every tool call the agent attempts |
| Decision outcomes | `PostToolUse` | Whether each tool call succeeded or was denied |
| Plan snapshots | `PostToolUse` | Transcript path captured when agent exits plan mode |
| Session boundaries | `SessionEnd` | Session summary with raw counts |
| Hook-detected anomalies | `PreToolUse` / `PostToolUse` | `stop_ignored`, `hallucinated_path`, `repeated_failure`, `regression_unlabelled` (see Limitations below) |
| Plan-vs-actual file diff | `SessionEnd` | `unrecorded_deviation` — files in plan but not touched, or files touched but not in plan (see [Plan File Diff Detection](#plan-file-diff-detection)) |
| Token usage | `SessionEnd` | Per-session input/output/cache token totals extracted from Claude Code session log |
| Estimated cost | `SessionEnd` | Computed from token counts and model pricing (labeled `estimated_cost_usd`) |
| Context compactions | `SessionEnd` | When the context window was compressed — trigger type and pre-compaction token count |

**Secondary metrics** (derived from primary records — raw counts, not computed rates):

| Metric | Source |
|--------|--------|
| `tool_attempts` | Count of `PreToolUse` events |
| `tool_rejections` | Count of tool calls denied by the user |
| `corrections` | User prompts that immediately follow a tool rejection * |
| `user_prompts` | Total human inputs in the session |

\* `corrections` is estimated. May overcount after benign failures (e.g. a grep with no results) and undercount approach-level corrections where the developer redirects without a tool rejection.

**Agent self-reported events** (hooks cannot detect intent — the agent must self-report):

| Event | When to report |
|-------|----------------|
| `plan_stated` | Agent commits to an approach — especially when implementing from a design document or specification, where the plan emerges from reading rather than from formal plan mode |
| `plan_deviation` | Agent makes a decision that differs from the committed plan |
| `declined_difficult` | Agent declines work because it would be difficult |
| `ignored_prior_failure` | Agent discovers a prior failure and chooses not to resolve it |
| `scope_creep` | Agent performs work not requested and not part of the committed plan |
| `observation_misread_as_instruction` | Agent acted on something stated as an observation as if it were a directive |

> **Limitation**: Self-reporting is least reliable for the exact situations it's designed to capture. An agent that declines difficult work may rationalize it as "out of scope" rather than flag it. Treat self-reported events as a lower bound, not a complete record. The developer should review sessions and append additional `agent_report` entries for events the agent missed.

> **Hook detection limitations**: Hook-detected anomaly events (`stop_ignored`, `repeated_failure`, `regression_unlabelled`) use `tool_response.success = false` to indicate a failed tool call. In practice, `success = false` means the user *denied* the tool call, not that the command exited with a non-zero status. These detectors therefore identify repeated permission denials, not execution failures. Stop-word matching (`stop_ignored`) uses a fixed word list and will produce false positives for prompts that use these words in a non-imperative context (e.g. "don't worry, proceed").

> **Session metrics limitations**: Token usage, cost, and compaction data are extracted from Claude Code's native session log (`~/.claude/projects/`). If the session log is not accessible (e.g. different user, remote execution), these fields will be `null` in the summary. The `estimated_cost_usd` uses a hardcoded pricing table and may not reflect current or discounted rates. Compaction events record `pre_tokens` but not the post-compaction count (Claude Code does not log it).

> **Subagent tracking**: Hook payloads do not include agent or subagent identity. Tool calls from subagents appear as normal events, indistinguishable from main-agent calls. Subagent attribution would require Claude Code to add agent context to hook payloads.

### Plan File Diff Detection

When a plan snapshot exists for the session, the `SessionEnd` hook automatically compares the plan's file list against actual git changes. This catches structural deviations the agent failed to self-report.

**How it works:**

1. The `ExitPlanMode` hook (in `record-tool-call.sh`) saves the plan content to `audit/plans/`
2. At session end, `finalize-session.sh` parses the plan for a markdown file table (`| File | Action |`)
3. It runs `git diff --name-only` to get the files actually changed during the session
4. It compares the two lists and emits `unrecorded_deviation` events for:
   - **Planned files not touched** — files listed in the plan with action Create/Modify that have no corresponding git change
   - **Unplanned files touched** — files changed in git that don't appear in the plan's file table
5. These events appear in the session summary's `agent_reports` array and in the `-events.jsonl` artifact

**Limitations:**
- Only detects file-level structural deviations, not intent-based ones ("I used a different algorithm")
- Requires a plan snapshot (sessions without `ExitPlanMode` are not checked)
- File path matching is substring-based (plan may use relative paths like `src/bus/bus.hpp` while git uses full paths)
- Files in `audit/`, `.claude/`, and common non-source paths are excluded from the unplanned check

### Retention

Dev-record produces two tiers of data, split across two directories:

| Tier | Directory | Contents | Retention |
|------|-----------|----------|-----------|
| **Project artifacts** | `audit/dev_record/` | Session summaries (`.json`), extracted agent reports and plan snapshots (`-events.jsonl`) | Permanent — commit to version control |
| **Operational detail** | `audit/ops_record/` | Full event logs (`.jsonl`) — individual tool calls, tool results, user prompts | Gitignored — subject to retention policy |

Project artifacts are the agent equivalent of design decision records (DDRs) and task conversations in human-led projects. They capture *why* decisions were made and should live alongside project documentation.

Operational detail is equivalent to ephemeral communication (chat messages, verbal discussions). Useful for debugging and review but not first-order project records. The install action adds `audit/ops_record/` to `.gitignore`.

### Data Schema

From nakane1chome/claude-skills, Unlicense.

More from nakane1chome/claude-skills