assess

Files (200)

• SKILL.md
• pyproject.toml
• actions-schema.md · references/
• consent-lifecycle.md · references/
• hotspot-cochange-experiment.md · references/
• monorepo-scoping.md · references/
• uninstall.md · references/
• assess_core.py · scripts/
• assess_emit_workflow.py · scripts/
• assess_finalize.py · scripts/
• assess_gate.py · scripts/
• assess_report.py · scripts/
• complexity-treemap.py · scripts/
• doc-graph-svg.py · scripts/
• README.md · scripts/lib/
• __init__.py · scripts/lib/
• accretion_ratchet.py · scripts/lib/
• agent_environment.py · scripts/lib/
• agent_harness.py · scripts/lib/
• agent_instructions_grader.py · scripts/lib/
• agent_ops.py · scripts/lib/
• anomaly_detector.py · scripts/lib/
• archetype.py · scripts/lib/
• artifact_schema.py · scripts/lib/
• assess_config.py · scripts/lib/
• badge.py · scripts/lib/
• change_coupling.py · scripts/lib/
• ci_workflow.py · scripts/lib/
• claude_config.py · scripts/lib/
• claude_config_fields.py · scripts/lib/
• command_index.py · scripts/lib/
• command_resolver.py · scripts/lib/
• config_drift.py · scripts/lib/
• context_blocks.py · scripts/lib/
• coupling_analysis.py · scripts/lib/
• coverage_gate.py · scripts/lib/
• coverage_report.py · scripts/lib/
• dart_capabilities.py · scripts/lib/
• dart_complexity.py · scripts/lib/
• decline_markers.py · scripts/lib/
• diff_reliability.py · scripts/lib/
• doc_complexity_join.py · scripts/lib/
• doc_graph.py · scripts/lib/
• doc_graph_layout.py · scripts/lib/
• doc_links.py · scripts/lib/
• doc_provenance.py · scripts/lib/
• doc_staleness.py · scripts/lib/
• evidence_check.py · scripts/lib/
• gap_actions.py · scripts/lib/
• gate_cost.py · scripts/lib/
• generated_files.py · scripts/lib/
• gh_cli.py · scripts/lib/
• git_churn.py · scripts/lib/
• import_credit.py · scripts/lib/
• instruction_budget.py · scripts/lib/
• instruction_claims.py · scripts/lib/
• instruction_content.py · scripts/lib/
• instruction_discovery.py · scripts/lib/
• instruction_files.py · scripts/lib/
• instruction_globs.py · scripts/lib/
• …and 140 more

SKILL.md (first lines)

# AI Readiness Assessment + Complexity Hotspot

Three artefacts in one pass against a target repo:

1. **Layered contract assessment** - 0-8 score across navigability, runtime liveness, code design, linters, architecture tests, CI, coverage, review bots, and AI project management.
2. **Complexity hotspot SVG** - Codecov-style treemap of the code. Size = LOC. Colour = cyclomatic complexity. Saturation = recent git churn. Vivid red = complex AND active = riskiest to change.
3. **Doc navigability SVG** - a node-graph of the docs. Structure = connectivity (centre = entry, rim = unreachable, dashed ring = orphan, solid edge = link, dotted edge = reference); colour = staleness (vivid red = a frozen doc beside churning code = a *lying map*); size = file length. Folds navigability and the decaying-map signal into one artifact.

Both SVGs are colour-blind-safe by default (OrRd ramp, no red-green).

All land as files inside the target repo. The skill always writes them locally; after writing, **ask the user** whether to open a PR in the target repo with the artefacts.

## The model: truth-pressure, not presence

Read this before scoring - it changes how you score. Across every layer, the real signal is never **presence**. It is whether a thing is under **active pressure to stay true**:

- Tests keep **behaviour** honest (CI fails when it's wrong).
- Retros / feedback loops keep the **process** honest (Layer 8 scores whether retros are *carried out*, not merely present).
- Maintenance keeps **docs** honest (a wiki tracked against code churn).
- Telemetry / liveness keeps **relevance** honest (is this code actually exercised).

So **AI-readiness is the degree to which a codebase's self-descriptions are kept honest, not the degree to which scaffolding exists.** Score artefacts on *maintenance pressure*, not existence. A stale-but-present doc scores **at or below absent**: missing makes the agent go look; confidently-stale makes it navigate fast to a wrong, current-looking conclusion.

The 9 layers (0-8) fall into three bands, ordered by dependency - what must hold for the next band to mean anything:

- **Read-side foundation** (L0 navigability, L1 liveness) - can the agent form a *true picture* before it acts?
- **Write-side enforcement** (L2-L7) - can the agent be trusted to produce good output? Only means something once you can trust that what you're reading is real and current.
- **Meta** (L8 feedback) - does the system keep itself honest over time? Depends on a working enforced system to improve, so it stays last.

### The three write-side tendencies the layers guard against

The write-side scores aren't abstract good practice - each traces to a known tendency of an AI contributor, observed across models. All three are the same defect: a self-description (the file's shape, a comment's promise, a gate's verdict) under no pressure to stay true. The deterministic core turns each into a cross-layer finding so the report names the specific files, not just the category:

- **Accretion** - an agent does what is asked, and what is asked is feature after feature; nothing in that loop asks for a refactor, so files only grow. **Now fully instrumented** via the `accretion_ratchet` finding: a file whose accumulated line count ratcheted monotonically upward across multiple commits with almost no deletion pressure (deletions below ~15% of total churn). Only top complexity/size-band files are flagged, never documentation, so growth-but-simple is never noise; `archive/`, `archived/` and `attic/` paths stay out of the attention list, disclosed in `excluded_as_archive`. It surfaces on three surfaces - the `accretion_ratchet` block in `run-context.json`, the `accretion_ratchet` cross-layer finding (with its files in the attention list), and a *growth-profile* line on each flagged hotspot page (`hotspots/*.md`). The signal disclaims itself (rather than dropping the result) when the git history is degenerate - a shallow clone or squashed import has no meaningful net-delta sequence, so the block carries `reliable: false` and the hotspot line is marked as possibly incomplete.
- **Unactioned intent** - an agent records promises it never returns to keep (`TODO` / `FIXME` / "remove after migration"). Instrumented via the `unactioned_intent` finding: markers aged by the edits they survived without being kept - a lying map of intent.
- **Guardrail erosion** - under pressure to make red go green, an agent loosens the check instead of fixing the root (a suppression, a skipped test, a widened threshold), hollowing out the layers meant to protect it while they still read as Present.

### Repository archetype (not every repo is software)

The 0-8 model assumes a software repo. A **knowledge / document base** - markdown sources, an LLM-maintained wiki, a `CLAUDE.md` schema, and no application code or runtime - has no code surface for the write-side layers (L2-L7). Scoring them Missing is itself a lying score: a well-run KB reads ~2.5/8 ("Not Ready") when it is in fact well-run, penalised for not testing code it doesn't contain.

The deterministic core (`lib/archetype.py`) classifies the repo and writes an `archetype` block to `run-context.json`:

- **Detection** is a heuristic - the code-file ratio (code vs markdown) and the absence of a runtime surface (`package.json`, `pyproject.toml`, `go.mod`, `Dockerfile`, ...). A documentation-heavy *application* (lots of markdown but a real build) stays software because of the runtime-surface gate.
- **Override marker.** An `assess-archetype: knowledge-base` (or `software`) marker in any instruction file (`CLAUDE.md`/`AGENTS.md`/...) **forces or suppresses** detection, so a maintainer is never trapped by a misfire. Write it as an HTML comment, e.g. `<!-- assess-archetype: knowledge-base -->`.
- **Scoring.** For a detected knowledge base the write-side layers (2-7) are scored **N/A** (not Missing) and **excluded from the denominator**; the headline renormalises over the applicable layers (L0, L1, L8 → denominator 3) and the maturity label names the archetype and the applicable-layer count (e.g. `Knowledge Base · Solid (3 applicable layers)`). A software repo is unaffected - all 0-8 layers, denominator 8.
- **KB-maintenance signal.** `archetype.kb_maintenance` flags whether the repo documents *how the AI maintains the KB* - the [Karpathy LLM-wiki pattern](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) (immutable raw sources, the schema file as the product, an ingest workflow, query-as-filing, periodic lint/consolidation). It is both a detection signal and a scored read-side (Layer 0) quality signal; the gist is cited in the report as the best-practice pointer whether or not the workflow is documented.

This is intentionally **one** archetype (knowledge base), structured as an extensible dispatch so more are cheap to add later - not a general archetype framework (YAGNI). The `assess-layer-scorer` agent reads the block (its Step 0) and the `assess-findings` skill renders N/A layers and the renormalised headline.

<!-- chat-skip:start -->
**$ARGUMENTS**
<!-- chat-skip:end -->

## Step 1: Determine Repo Root and Output Directory

```bash
git rev-parse --show-toplevel   # from the arg path if given, else pwd
```

Set `$REPO_ROOT` to the result. All scanning happens from here.

**Scoping a subtree (`/assess <path>`).** When the argument is a directory under the repo root, scope the whole run to it - metrics, score, badge, wiki, and gate all computed for and labelled with the scope, artifacts under `.assess/<scope-slug>/`, no signal from a sibling. Pass `--scope "$SCOPE"` to `complexity-treemap.py` and `assess_core.py` and swap `.assess/` for `.assess/<slug>/` throughout. Full recipe: `references/monorepo-scoping.md` (relative to this skill dir). A no-path run is whole-repo, unchanged.

Decide the output directory (default: `$REPO_ROOT/.assess/`). Create it if needed:

```bash
mkdir -p "$REPO_ROOT/.assess"
```

Artefacts will land at:
- `$REPO_ROOT/.assess/complexity-heatmap.svg`
- `$REPO_ROOT/.assess/complexity-stats.json`
- `$REPO_ROOT/.assess/doc-graph.svg`
- `$REPO_ROOT/.assess/assess-report.md`

> **Write-protected repo root?** `/assess` writes `.assess/` into `$REPO_ROOT`, and the treemap/core run as `uv` subprocesses that write there too. If your workflow keeps the repo root pristine and read-only (e.g. a `<repo>-main` clone that teammates branch from, with a hook blocking direct edits), a guard on *your* writes won't stop the subprocess - it just makes the run write into the directory you meant to protect. **Create a worktree first and run `/assess` from there.**

## Step 2: Generate the Code Heatmap + Doc Graph

This step produces **two** views of the codebase, both colour-blind-safe (OrRd ramp, no red-green):

- **Complexity heatmap** (`complexity-heatmap.svg`) - a treemap of the *code*. Size = LOC, colour = cyclomatic complexity, saturation = recent churn. Vivid red = complex AND active = "hard to change safely".
- **Doc navigability graph** (`doc-graph.svg`) - a node-graph of the *docs*. Structure shows connectivity (centre = entry point, rings = link-distance, rim = unreachable; orphans carry a dashed ring; solid edges are links, dotted edges references); colour shows staleness in the same grammar as the code heatmap (vivid red = a frozen doc beside churning code = a lying map); size = file length. It folds both Layer 0 doc signals - navigability and the decaying-map - into one artifact. Beyond static wikilinks and CommonMark links, it counts a backticked path to an existing doc as a reference edge (a cited `.claude/` file included) and recognises Obsidian vault-native navigation - `.base` view hubs and `dataview` query blocks - as edges (resolved statically by folder / tag / frontmatter predicate), so a vault navigated by dynamic queries isn't mis-scored as orphaned. The SVG and the scored signal compute over the identical doc set: both honour the same excludes (`.assess/config.toml`).

Feed the complexity stats into the linter/complexity layer (Layer 3) and the `doc_graph` / `doc_staleness` blocks of `run-context.json` into **Layer 0** (the graph SVG is the visual; the score reads the structured blocks). The core also reads the repo's Claude Code configuration - `.claude/skills/`, `.claude/commands/`, `.claude/agents/`, and for a plugin repo `skills/`, `commands/`, `agents/` or the paths `.claude-plugin/plugin.json` names - and writes the `claude_config` block: legacy command files, and frontmatter keys or values Claude Code silently ignores, checked against documented field sets pinned in `scripts/lib/claude_config_fields.py` with their snapshot date. It is Layer 0 evidence, not a badge finding; `available: false` (no `.claude/` and no plugin manifest) means not checked, never clean. Beside it, the `agent_harness` block lists what the committed agent configuration runs on every session (settings and frontmatter hooks, `env` names, `apiKeyHelper`, `.mcp.json`, `.cursor/hooks.json`) and flags a tracked `.env` or `settings.local.json`, a missing `.env` deny rule, `disableAllHooks: false`, a broad `Bash` allow, a devcontainer credential mount, hidden Unicode in instruction files and comment-triggered deploys; it is informational and never prints a secret value. The `agent_environment` block feeds **Layer 5**: committed agent setup (`copilot-setup-steps.yml`, a dev container, `.cursor/environment.json`, a `SessionStart` hook, a setup script or target), the check command an instruction file names, lockfiles per manifest, and `layer5_cap`; its `ci_duration` is the one `gh` read, optional and `available: false` with a reason when `gh` cannot answer.

### The consent lifecycle (read `references/consent-lifecycle.md`)

Steps 2a/2b/2d and the assess-pr end-of-run offers share one consent model, **specified in full in `references/consent-lifecycle.md`** (relative to this skill dir) - load it before running the offers. Load-bearing hooks the steps rely on: decline markers carry provenance (write `.no-<tool>` JSON via the reference's `write_decline_marker` helper, never a bare `touch`; a mutation decline under an older plugin *major* sets `reoffer_mutation: true` so Step 2d re-asks once); three phases each a single batched question (Phase 1 tool installs 2a+2b, Phase 3 the separate mutation pass 2d, Phase 2 the assess-pr write-back offers); and the non-interactive contract - a headless/CI run makes **no AskUserQuestion calls in any phase**, Phase 1 as orchestration from your runtime context and Phases 2/3 from the core's `run-context.json .interactive` flag, which pre-records every skipped offer in `.offers`.

### 2a: Detect `scc` need (feeds Phase 1)

The bundled treemap uses [`lizard`](https://github.com/terryyin/lizard) (Python, Go, JS, Java, C/C++, etc.) by default. Optional `scc` extends coverage to 200+ languages including markdown, JSON, YAML, SQL, and shell - useful when the repo's surface is more than just traditional source code.

Before scanning, check three signals:

```bash
# 1. Is scc already on PATH?
command -v scc >/dev/null 2>&1 && SCC_PRESENT=1 || SCC_PRESENT=0

# 2. Has the user previously declined for this repo?
[ -f "$REPO_ROOT/.assess/.no-scc" ] && SCC_DECLINED=1 || SCC_DECLINED=0

# 3. Is the repo mostly markdown/data/config (where lizard alone will be sparse)?
#    Cheap heuristic: count non-code files vs code files. The `.` argument is
#    the regex pattern (matches every path) and "$REPO_ROOT" is the search
#    path - without `.`, fd treats $REPO_ROOT as the pattern itself, matches
#    nothing, and silently returns 0.
CODE_FILES=$(fd -t f -e py -e js -e ts -e tsx -e jsx -e go -e java -e kt -e rs -e rb -e cs -e swift -e dart -e cpp -e c -e h -e php . "$REPO_ROOT" 2>/dev/null | wc -l | tr -d ' ')
NONCODE_FILES=$(fd -t f -e md -e json -e yaml -e yml -e toml -e sh -e sql . "$REPO_ROOT" 2>/dev/null | wc -l | tr -d ' ')
```

**Add `scc` to the Phase 1 offer list only if all three are true:** `SCC_PRESENT=0`, `SCC_DECLINED=0`, and the repo looks lizard-sparse (`CODE_FILES < NONCODE_FILES` or `CODE_FILES < 10`). Otherwise it contributes nothing to Phase 1. Do **not** ask here - `scc` is batched with the dead-code tools into the single Phase 1 question in Step 2b. Its trade-off phrasing, for when the batched question is presented:

> "This repo has <N> code files and <M> non-code files (markdown/JSON/YAML). `scc` would include the non-code files in the treemap; without it the treemap may be sparse. Install `scc`?"

The three options are the shared Phase 1 shape (**Install** / **Skip for now** / **Skip permanently** via `write_decline_marker scc`). If the user accepts, run the platform-appropriate command (do **not** auto-install - `brew install` is a system mutation):

From bjcoombs/ai-native-toolkit, Apache-2.0.

More from bjcoombs/ai-native-toolkit