agentic-readiness-assessment

Files (6)

• SKILL.md
• agent-execution-contract.md · references/
• assessment-decision-record.md · references/
• report-contract.md · references/
• validate_report.py · scripts/
• test_validate_report.py · tests/

SKILL.md (first lines)

# Agentic Readiness Assessment

Read the `version` field of the plugin's `plugin.json` and record it as `prompt_version` in the report run block, so reports produced by different revisions stay comparable. Write `unknown` if the manifest cannot be read.

## Objective

Assess whether an AI coding agent can independently understand this repository, find the correct change points, establish a reproducible environment, validate work, and prepare a change for delivery. Produce an evidence-backed improvement backlog for maintainers.

This is an audit. Do not implement features or refactor production code. Create only one file: `reports/agentic-readiness.md`, a single document holding both the decision content and the supporting evidence.

The only permitted change to any other tracked file is the single reversible probe edit described in *Probe*, which you revert before writing the report. Preserve all pre-existing user changes.

The deliverable of this task is that report. Any standing instruction to output only code, or to treat an analysis document as a failed task, applies to feature work and not to this audit.

## Host Execution Contract

Read `references/agent-execution-contract.md` before collecting the baseline. Under the explicit constraints in Run and Scope, record the host's model identity, effective permission profile, command boundary, run stop condition, and context telemetry availability. Write `unavailable` for any field the host cannot evidence, classify it as an `agent-environment` limitation, and never invent a tool restriction, model pin, or telemetry control. After roughly five attempts to obtain unavailable evidence, stop and record the limitation.

## Evidence and Safety Rules

1. Cite repository-relative paths and line numbers for repository claims.
2. Cover every applicable workspace or component; do not silently sample a monorepo.
3. Prefer commands from agent instructions, documentation, task runners, package scripts, and CI. Label each command `documented` or `inferred`.
4. Run relevant safe commands. Record exact commands, working directories, results, durations when measured, and artifacts.
5. A command you did not execute is a claim, not evidence. Do not mark a capability verified because configuration exists when a safe executable check was available. A reachable health endpoint does not prove the system behind it performs meaningful work.
6. Do not deploy, publish, push, run production migrations, use real credentials or customer data, call paid services, or execute destructive or privileged commands.
7. Never bulk-restore. Do not run `git restore`, `git checkout -- .`, `git clean`, or `git reset` across the tree. Revert the probe by rewriting the exact path you edited.
8. Stop only services you started, and delete only artifacts you created. Never create, update, or commit a lockfile or equivalent resolution data.
9. Run linters and formatters in check or diff mode. Never auto-fix.
10. Use reasonable timeouts. Leave no background processes running.
11. Write `Not found` when evidence is absent. Never infer a capability, and never reinterpret a failure as a pass.

**Never let your own tooling become the finding.** If a command fails because of a flag you chose, a wrapper you lack, a missing timing utility, or a typo, retry it correctly and log only the meaningful attempt. Offline, cached-only, or sandbox-restricted flags are your constraints, not the repository's — a failure they cause is never evidence against the repository.

Classify commands before execution:

- **local-read:** Reads repository or machine state. Run.
- **local-build:** Produces local build, cache, test, or ignored artifacts. Run when documented and safe.
- **external-read:** Reads from a registry or source already required by the repository. Run only when permitted; record the endpoint.
- **external-write:** Pushes, publishes, deploys, or mutates remote state. Do not run.
- **privileged:** Requires root, `sudo`, or machine-wide changes. Do not run.
- **paid:** Calls a billed external service. Do not run.

Run `python3 <skill-directory>/scripts/validate_report.py reports/agentic-readiness.md` from the audited repository after the provisional report write; `<skill-directory>` is the directory containing this `SKILL.md`, not a path inside the audited repository. This is a `local-read` command that validates the report contract without changing the repository.

Being documented does not make a command safe. A refused command is not a skipped category: record it, name its class, and score on the safe evidence that remains.

### Dependency Installation

Attempt the repository's frozen or locked installation first, in the ecosystem's own idiom. If it fails because resolution data is missing, incomplete, or incorrect, that is a repository finding — record it, then retry **once** with the ecosystem's non-frozen install using flags that cannot create or modify resolution files, for example `npm install --package-lock=false`.

Label that result **unfrozen fallback**. Evidence obtained after a successful fallback is valid for every downstream area: proving that typecheck, tests, and the build actually pass is more useful than reporting them unproven. The setup area itself scores at most half when the fallback was required. If even the fallback fails, or if the ecosystem offers no non-mutating equivalent, record the blocker and move on.

## Blockers and Fix Ownership

Keep repository readiness separate from limitations of the audit environment:

- **Repository blocker:** Repository-owned instructions, dependencies, commands, fixtures, or controls are missing, incorrect, unsafe, contradictory, or nondeterministic.
- **Environment blocker:** The repository documents a sufficient requirement, but the audit machine lacks the required tool, service, network access, or permission.
- **External prerequisite:** A credential or external service is required; assess whether a documented local or test substitute exists.
- **N/A:** The capability is genuinely irrelevant to the repository archetype.

Route each fix to one owner:

- **repo:** A file or control in the audited repository.
- **agent-environment:** The image, hook, runtime, or toolchain that hosts the coding agent.
- **platform:** Organization or infrastructure configuration outside both the repository and agent environment.
- **external-service:** A third-party service, credential, or policy outside local control.

A missing build tool is never a documentation fix. If `make`, a container runtime, a package manager, or a browser is absent, the owner is `agent-environment`; documenting the prerequisite instead is a deferred repair. Missing prerequisite documentation is itself a separate repository defect.

Do not invent an external file path or configuration key. If the exact target cannot be established, name the required artifact generically and mark the target unverified.

## Audit Method

### Delegation

Read `references/assessment-decision-record.md` before baseline collection. Use the audit agent only to interpret repository evidence; use deterministic commands for score arithmetic and `scripts/validate_report.py` for report validation. The requesting human owns scope constraints and must approve an irreversible operation before it runs; otherwise stop and record the command as refused. Keep the delegation map's Cost / Complexity / Risk discipline in the evidence and report the named detection mechanism for any limitation.

### 1. Baseline and Scope

Capture the baseline before anything else: tracked and untracked worktree state, stash list, commit, and branch. A dirty tree or pre-existing untracked directory is the baseline, not an error. Record the platform, explicit constraints, and relevant installed tools.

Classify the repository as an application, service, frontend, library, CLI, infrastructure, documentation, data pipeline, monorepo, or an applicable combination. List all components and workspaces in scope.

### 2. Resolve the Archetype's Surfaces

Before scoring, name the concrete command filling each role, or `N/A` with justification.

| Role | Application / service | Library / SDK | Infrastructure | Documentation | Data / model |
|---|---|---|---|---|---|
| Deliverable | Build the app or image | Package or compile | Validate or lint the plan | Render the site | Build the pipeline or compile the schema |
| Primary verification | Unit or component tests | Unit tests, API compatibility | Policy or conformance tests | Link, schema, front-matter checks | Schema validation, golden-record tests |

If a role is `N/A`, name the nearest executable substitute and say what it does not prove. A repository with no executable verification surface at all cannot be Ready.

### 3. Reconstruct the Golden Path

Determine the intended route from a clean checkout to a delivered change:

`setup -> targeted feedback -> full validation -> review or PR`

Inspect:

- `AGENTS.md`, `CLAUDE.md`, repository-local rules, commands, prompts, and skills;
- README, architecture, contribution, ownership, and delivery documentation;
- manifests, dependency-resolution files, runtime versions, environment examples, containers, and fixtures;
- task runners, build scripts, test configuration, hooks, permission and policy files, and CI.

Trace one representative feature from its entry point through business logic, persistence or an external boundary, UI when applicable, and tests. Assess whether a newcomer can discover where a similar change and its tests belong. Do not edit the feature.

### 4. Exercise Applicable Validation

For every component, find and safely exercise the applicable surfaces: dependency bootstrap; build, packaging, rendering, or schema validation; static analysis; unit tests; integration or contract tests; runtime, API, CLI, or local-preview validation; browser validation; coverage evidence.

Start services only through a documented local or disposable workflow. If execution is unsafe or blocked, record why and classify the blocker; do not invent a substitute that changes the intended workflow.

### 5. Check Operability and Enforcement

From exadel-inc/agentic-readiness-assessment, Apache-2.0.

More from exadel-inc/agentic-readiness-assessment