Agent Cost Report

Files (54)

• .gitignore
• CHECKSUMS.txt
• SKILL.md
• VERIFICATION.md
• acr.py · scripts/
• __init__.py · scripts/acr/
• behavior.py · scripts/acr/
• classify.py · scripts/acr/
• costs.py · scripts/acr/
• devices.py · scripts/acr/
• evidence.py · scripts/acr/
• labels.py · scripts/acr/
• measure.py · scripts/acr/
• mistakes.py · scripts/acr/
• patterns.py · scripts/acr/
• period.py · scripts/acr/
• prices.py · scripts/acr/
• render.py · scripts/acr/
• rollup.py · scripts/acr/
• rules.json · scripts/acr/
• rules.py · scripts/acr/
• sample.py · scripts/acr/
• snapshot.py · scripts/acr/
• sync.py · scripts/acr/
• transcripts.py · scripts/acr/
• wins.py · scripts/acr/
• _db.py · scripts/tests/
• _paths.py · scripts/tests/
• rollout-codex-1.jsonl · scripts/tests/fixtures/codex/sessions/2026/09/18/
• make_report_fixtures.py · scripts/tests/fixtures/
• prices_sample.json · scripts/tests/fixtures/
• sess-aaaa.jsonl · scripts/tests/fixtures/projects/-workspace-proj-a/
• sess-bbbb.jsonl · scripts/tests/fixtures/projects/-workspace-proj-b/
• report-1day.json · scripts/tests/fixtures/
• report-30day.json · scripts/tests/fixtures/
• report-7day.json · scripts/tests/fixtures/
• report-empty.json · scripts/tests/fixtures/
• report-session.json · scripts/tests/fixtures/
• test_behavior.py · scripts/tests/
• test_devices.py · scripts/tests/
• test_labels.py · scripts/tests/
• test_measure.py · scripts/tests/
• test_pdf_windows.py · scripts/tests/
• test_period.py · scripts/tests/
• test_prices.py · scripts/tests/
• test_render.py · scripts/tests/
• test_render_encoding.py · scripts/tests/
• test_rollup.py · scripts/tests/
• test_snapshot.py · scripts/tests/
• test_sync.py · scripts/tests/
• test_transcript_encoding.py · scripts/tests/
• test_transcripts.py · scripts/tests/
• test_wins.py · scripts/tests/
• precision-2026-09-26.json · verification/

SKILL.md (first lines)

# Agent Cost Report

**Claude-Mem / Claude Code skill.** Runtime is the `scripts/` pipeline (transcripts → tokens → dollars → Timing-style report) plus a progressive Mem Search review pass that confirms the drafted labels. The Notion draft is SPEC history only — never the product, never the runtime, never the ship vehicle.

Resolve the absolute directory containing this `SKILL.md`; all helper paths are relative to that directory. `${CLAUDE_SKILL_DIR}` is the shortcut: `python3 "${CLAUDE_SKILL_DIR}/scripts/acr.py" …`. Python 3.9+ standard library only, with IANA timezone data for `America/Los_Angeles`; no pip installs. `tzdata` is the only exception to the no-pip-installs rule, and only when `acr.py` exits saying that timezone data is unavailable (stock Windows Python ships none): show the user the lines it printed and ask (AskUserQuestion) before installing it. On a yes, run the line for your shell (the plain quoted line in Bash or cmd, the `PowerShell:` line in PowerShell), then rerun the failed command. Setting `PYTHONTZPATH` instead needs no install. The look lives in `scripts/acr/render.py`, never here.

## Purpose

Turn Claude-Mem activity into a **manager-readable cost and failure report**.

Product idea: a reusable skill that searches Claude-Mem via Mem Search, reconstructs real units of work, assigns cost and failure categories, and renders a printable report.

## Insight north star

The headline is dollars, to two decimals, labeled. The dollars come from Claude Code transcripts (exact per-reply token usage) priced at OpenRouter public list prices, so they are **ESTIMATED**. Measured provider spend appears only when a sanctioned source gives it. Directly under the dollars: what the mistakes cost, what shipped, and both on one time axis.

## Questions the report must answer

1. What work was completed?
2. What did each outcome cost?
3. What was wasted through looping, hedging, wrong turns, rework, or poor routing?
4. Were any unauthorized actions attempted?
5. What should the manager change next?

**Primary unit = cost per completed outcome** (not cost per observation).

## When to use

- "Agent cost report" / "cost per outcome" / "failure economics" / "was this session worth it" / "what did the agents cost this week"
- After a real Mem session dig when leadership needs outcome economics
- Sample / ship packs that need self-contained HTML + JSON + CSV + evidence

Memory dig mechanics: the claude-mem `mem-search` skill (progressive recall).

## Default scope (ALWAYS)

- Unless the user names a **specific session / range / project**, the window is the **last 7 full days in PT, not counting today**: `end` = the PT midnight that started today (exclusive), `start` = `end − 7 days`. The default window never contains a partial day (G3, Alex 2026-09-25).
- Explicit windows: `--start YYYY-MM-DD --end YYYY-MM-DD` (PT calendar days, end exclusive). An explicit `--end` later than today marks the last day "partial, generated HH:MM PT".
- One session: `--session <content_session_id>`. One project plus a period: `--project <name> --start … --end …` (worktrees of the project are included).
- The report **Scope** strip shows the PT range, and Details list every session id in scope.

## Progressive Mem Search (ALWAYS)

Follow the claude-mem `mem-search` three layers. Keep spend light.

1. **Search** — get an index of IDs (titles, types, token hints).
2. **Timeline** — only around anchors you care about.
3. **Observations** — `get_observations` for the filtered IDs you will cite as evidence.

Recipe:

1. Resolve scope (default: the last 7 full PT days).
2. Search → collect IDs.
3. Timeline for thin context only.
4. Observations for intended / actual / outcome / waste / rework / blocked / unauthorized / status.
5. Group into named work items + failure events.
6. Calculate line-item costs.
7. Render HTML + optional PDF (+ json/csv/evidence).
8. Keep evidence IDs in the appendix — do not dump entire timelines into the main report.

ADHD process bullets:

- Search first → pick IDs → timeline only if context is thin → fetch only needed obs.
- Work-item titles are invented for managers ("Restore search after Chroma crash-loop"); observation titles stay evidence-only.
- Evidence appendix lists obs IDs + short titles; main sections stay outcome-first.

In this skill the search pass is the **review step** (see Recipe step 4): the pipeline drafts categories and failure types from keywords (`label_source: keyword`); the orchestrator confirms or changes each line item's `category` and `failure_type` from its cited evidence IDs and applies the result with `review --apply`. Items left unreviewed keep the "draft label" mark and the footer counts them.

## Work categories (ALWAYS)

`Feature` · `Bug fix` · `Incident` · `Maintenance` · `Investigation` · `Experiment`

## Failure / waste types (ALWAYS)

`Looping` · `Hedging` · `Wrong turn` · `Rework` · `Regression` · `Premature completion` · `Unauthorized action` · `Suboptimal path` · `Duplicate work` · `Blocked work` · `Missed requirement` · `Unnecessary escalation` · `Context re-read` · `Model thrash` · `Fan-out waste` · `Recovery after miss`

**Rework lock (ALWAYS):** Rework lives **only** under `failure_type` — never as a work category. Keep category as the intended job type; set `failure_type: Rework` when rework occurred.

A line item can have a work category **and** a failure_type (e.g. Maintenance + Looping).

## Cost model

Measured tokens come from Claude Code transcripts (`~/.claude/projects/**/*.jsonl`, assistant replies deduped on `(message.id, requestId)`); Codex transcripts are read the same way. Prices are OpenRouter public list prices per million tokens, fetched at run time and saved with the report.

```
agent_cost_i (per reply, micro-dollars) =
      input × price.input + output × price.output
    + cache_write_5m × price.cache_write + cache_write_1h × price.cache_write_1h
    + cache_read × price.cache_read                    # cache_write_1h = listed rate, else 2 × input

agent_estimated_usd        = Σ agent_cost_i over every reply in the window (matched or not)   # ESTIMATED, the headline
extrapolated_unmeasured    = observer tokens of sessions with no transcript here × (measured $ per observer token)
                           # EXTRAPOLATED (low confidence); "all sessions measured" when nothing remains
observer_note_taker_est    = note-taker (observer) tokens, deduped per reply, × its input list price
                           # priced separately, never agent cost, never in the headline
mistakes_estimated_usd     = Σ agent_cost_i over the same-session union of wasted turns, each turn once   # low figure
cost_per_completed_outcome = Σ attributed $ for status ∈ {shipped, completed} / count(those work items)
waste_rate                 = Σ wasted_cost / Σ attributed $     recovery_share = Σ recovery_cost / Σ attributed $
```

Line items are sessions: `attributed_usd` = estimated (transcript on this box) or extrapolated (no transcript). `wasted_cost` and `recovery_cost` come from the behavior pass (below), one union set for the ribbon, the line items and the mistakes line. The upper bound (redo windows plus project-wide fallback) stays in Details.

**Unauthorized blocked:** `direct_cost` $0, `risk_exposure` high, `action_status` blocked. Always keep risk_exposure non-dollar unless real cash/remediation is at stake — never invent risk dollars.

`confidence` — `high` when tokens + model + outcome are clear; `medium` when allocation across obs is judgmental; `low` when evidence is thin.

Unpriced models (not in the price list, or a negative "variable" price) are listed by name with their tokens and add nothing; they are never priced at zero silently.

## Behavior metrics (heuristic until reviewed)

A second pass over the same transcripts tags every user turn human / bot / unknown (relayed agent prompts are never Alex's words), finds frustration episodes, and runs the pattern detectors from the Frustration Arc study: invented human gates, broke working things, wrong or expensive model, over-engineering, did something not asked, fake output, false "done", wrong tool or contact, bad outbound (incidents × recipients, never dollars), memory or rule loss, jargon, unclear cause, plus tool errors and hedging (Alex's definition: a caveat given when the answer was already available). Four summary tiles; everything else in Details. Every count is labeled `heuristic` until reviewed or classified. The optional classifier (`--classify`) is off by default, capped at $2.00 per run, uses only a regular inference `OPENROUTER_API_KEY`, and its spend is shown separately.

## Money labeling (ALWAYS)

| Label | Meaning |
|-------|---------|
| **Measured** | Provider-reported spend from a sanctioned source (today: the OpenRouter per-key snapshot, shown with its UTC bucket label) |
| **Estimated** | Measured transcript tokens × OpenRouter list price per MTok (the formula above) |
| **Extrapolated** | Sessions with no transcript on this machine, from the observer-token ratio; always "low confidence" |

From thedotmack/claude-mem, Apache-2.0.

More from thedotmack/claude-mem