eng-harness-0-harnessability-assessment

Files (11)

• AUTHORING.md
• README.md
• SKILL.md
• assessment-latest.json · templates/
• assessment-latest.md · templates/
• assessment-report.md · templates/
• assessment-report.schema.json · templates/
• canonical-boundary.txt · templates/
• codebase-affordance-record.json · templates/
• evidence-log.jsonl · templates/
• summary.md · templates/

SKILL.md (first lines)

# eng-harness-0-harnessability-assessment

Assess how easy a repository is for a human or agent to **enter, operate, modify, observe, prove, and improve** through an engineering harness.

Cold-start onboarding difficulty is one part of harnessability, not the whole assessment.

The agent harness drives. The engineering harness proves.

## Core idea

Harnessability is the degree to which a codebase can be worked through a project-side engineering harness by a fresh human or agent.

A harnessable repo lets an agent move through this loop:

```text
cold start -> boot/setup -> scoped change -> supported interaction -> observable consequence -> deterministic verdict -> clean rerun -> encoded improvement
```

The important object is **back pressure**: deterministic feedback that pushes reality back into the agent's loop. Back pressure can come from many surfaces:

- build, compile, typecheck, lint, unit test, schema, security, dependency, or architecture checks;
- boot, health, smoke, E2E, API, UI, CLI, worker, queue, event, MCP, or system interactions;
- observable consequences such as state, files, database rows, messages, events, cache entries, search indexes, generated artifacts, rendered UI, screenshots, logs, traces, metrics, or structured diagnostics;
- external-effect sinks for email, SMS, payments, webhooks, object storage, third-party APIs, analytics, notifications, or background jobs;
- production/customer evidence such as telemetry, SLOs, incident-free release, usage outcomes, or human acceptance supported by captured evidence.

Do **not** overfit harnessability to databases. A database migration/seed/reset loop is only one easy-to-understand example of the broader **change-to-evidence loop**.

A **Backpressure Check** is advisory. It surveys whether enough deterministic sensors exist for the scoped work. It is not itself proof. Proof comes from the sensors, commands, evidence artifacts, and observed consequences.

## Read the tests as harness reconnaissance

The test suite is the strongest evidence of how a codebase already solves the hard harness problems. Whatever the tests do to make behavior real is a proven, in-repo mechanism the engineering harness can reuse instead of inventing.

When inspecting tests, ask:

- How do they isolate dependencies — mocks, fakes, stubs, in-memory implementations, contract tests?
- How do they inject or substitute behavior — dependency injection, ports/adapters, provider overrides, test-only configuration?
- How do they handle datastores and state — seed, snapshot, restore, reset, factories, fixtures, Testcontainers, ephemeral schemas?
- How do they make behavior real — spin up real or containerized services, exercise true interactions, and assert observable consequences?
- How do they stay deterministic — fixed clocks, IDs, ports, ordered teardown, idempotent setup?

Every mechanism the tests already use is a candidate harness affordance. An existing integration or E2E suite that provisions a real or containerized datastore, seeds it, exercises behavior, verifies the consequence, and tears down is itself a reusable change-to-evidence surface — often the cheapest existing path to L4 proof, because the harness can wrap it rather than build new scaffolding. Treat "expose what the tests already do as a supported harness command" as a first-class, high-leverage remediation.

## When to use

Use this skill when the user asks to:

- onboard a repo;
- assess repo onboarding difficulty;
- compare repositories for agent readiness;
- inspect whether an agent can get a repo running;
- determine how much remote infrastructure, secrets, services, or env setup blocks local operation;
- assess whether changes can be made and proven locally or safely;
- evaluate engineering harness quality;
- identify missing back-pressure surfaces;
- decide what harness or product-code affordances would make future agent work safer, faster, or more deterministic.

If the user says "onboard this repo", treat it as the broad form of the request: harnessability assessment covers cold-start orientation as one part of a wider readiness picture.

## Inputs

```text
$ARGUMENTS

# Common flags:
# --repo <path>                         Repository root. Defaults to current working directory.
# --markdown                            Emphasize the Markdown report path in the final answer.
# --json                                Emphasize the JSON report path in the final answer.
# --output-dir <path>                   Defaults to .harness/reports/harnessability/.
# --execute-safe-probes                 Allow safe local read-only probes such as help/list/version/dry-run commands.
# --deep                                Allow slower read-only analysis such as dependency graph inspection and git-history mining.
# --compare <previous.json>             Compare against a previous assessment JSON file.
# --weight <operate>:<adaptability>     Override default Harnessability Index weights. Default: 0.5:0.5.
# --scenario <name>                     Add or prioritize a scenario probe. May be repeated.
# --no-index                            Omit the single blended index; still emit the two-axis tuple.
# --apply-safe-harness-patches          Allow low-risk harness-only patches. Never applies product-code changes.
# --propose-codebase-affordances        Include product-code affordance recommendations. Default: on.
# --target-mode clean-start             Future mode: evaluate the clean-start experience in a disposable context.
```

No flags means read-only static assessment plus report generation.

## Output contract

Each run writes a per-run history directory plus stable root "latest" files:

```text
.harness/reports/harnessability/<ordinal>-<slug>/report.md
.harness/reports/harnessability/<ordinal>-<slug>/report.json
.harness/reports/harnessability/<ordinal>-<slug>/summary.md
.harness/reports/harnessability/<ordinal>-<slug>/evidence.jsonl
.harness/reports/harnessability/latest.md
.harness/reports/harnessability/latest.json
.harness/reports/harnessability/schema.json
```

`<ordinal>` is the next free 3-digit number (`001`, `002`, …) found by scanning existing `.harness/reports/harnessability/<NNN>-*/` directories; `<slug>` is a short kebab-case label for the run (e.g. the repo name, or `assessment`).

**Every run overwrites the three root files** (`latest.md`, `latest.json`, `schema.json`) so they mirror the newest run. The root `latest.json` is a load-bearing **sentinel**: the adoption flow (`/eng-harness-flow`) detects an existing assessment with `test -f .harness/reports/harnessability/latest.json || ls .harness/reports/harnessability/*` and then reads `latest.json` for recommendations. Keep it present **and** readable on every run — never write only the history directory (that would satisfy detection but leave nothing stable to read).

The Markdown report follows `templates/assessment-report.md`. The JSON report follows `templates/assessment-report.schema.json`, the authoritative v0.2 contract; write a copy of it to `.harness/reports/harnessability/schema.json`. The terminal-sized `summary.md` follows `templates/summary.md`; keep detailed evidence in `report.md`. See `templates/assessment-latest.md` and `templates/assessment-latest.json` for sanitized examples. If the repo already contains earlier assessment reports or onboarding docs, read them as evidence, but write this run's reports under `.harness/reports/harnessability/`.

Keep the JSON schema version at `harnessability-assessment.v0.2` until the core contract changes.

## Safety defaults

Default mode is static and conservative.

Allowed by default:

- read repository files;
- inspect Git metadata with read-only commands;
- inspect harness surfaces, command maps, package manifests, CI configs, local docs, and examples;
- inspect non-secret environment variable names and example values from safe example files;
- identify required secrets by name only;
- classify evidence, inference, human-supplied facts, and unknowns;
- write assessment reports under `.harness/reports/harnessability/`.

Not allowed by default:

From AI-Substrate/harness-engineering, MIT.

More from AI-Substrate/harness-engineering