skill-gauntlet-runner

Files (4)

• SKILL.md
• contracts.md · references/
• extract_bundle.py · scripts/
• package_results.py · scripts/

SKILL.md (first lines)

# skill-gauntlet Cowork runner

Run exactly one prepared suite. Do not grade it and do not expose your reasoning in artifacts.

## Inputs

The human supplies a prepared ZIP, `RUN_ID`, and asks you to use this skill. Invoke this skill's
bundled `scripts/extract_bundle.py` with the archive, `--working-root working/skill-gauntlet`, and
`--run-id`. The validator derives a fresh destination below that root; use its returned workspace.
Never accept a prepared folder, extract into `output/`, reuse an existing destination, or extract
manually after validation fails.

Read `run-request.json`, `evals.json`, and `grading-envelope.json` from the extracted workspace.
Refuse the run if their `schema_version` major is not `0`, the run ID differs, or a case path contains
a separator or `..`. Read `references/contracts.md` before writing artifacts; its JSON shapes are
mandatory.

## Output root

Use the session's synced `output/` surface. Create:

`skill-gauntlet/<RUN_ID>/`

Write all paths below relative to that root. Never write durable artifacts under `working/`.

## Execution

1. Copy `grading-envelope.json` to the run root unchanged.
2. Write `run.json` with schema version `0.1`, skill name, run ID, mode, status `running`, ordered
   `case_ids`, adapter capabilities, and `skill_gauntlet.deploy_mechanism: plugin-package`.
3. Process `case_order` sequentially. Never run cases in parallel.
4. For each non-trigger case:
   - Create `outputs/case-<id>/workspace/inputs/` and `artifacts/`.
   - Copy only that case's prepared inputs into its `inputs/`.
   - Spawn a **fresh subagent** with no prior case conversation.
   - In `with_skill` mode, require the subagent to resolve the subject skill from
     `run-request.json` **by name** and execute the case prompt. Do not paste skill instructions or
     point to a `SKILL.md` path as fallback.
   - In `without_skill` mode, execute the prompt without invoking the subject skill.
   - Tell the subagent it may read only its case workspace and must put every deliverable under its
     `artifacts/` directory.
   - Write `case.json` with schema version, case ID, execution status, null token counts, and
     `worker_dispatch` set honestly to `by_name`, `not_applicable`, or `unresolved`.
   - If by-name resolution fails, record `unresolved`; never substitute a fallback.
5. For trigger cases, write `case.json` with status `skipped` and reason
   `Cowork does not support trigger evaluations`; do not spawn a worker.
6. After every case manifest exists, update `run.json` status to `complete`.
7. Invoke this skill's bundled `scripts/package_results.py` with
   `--output-root output/skill-gauntlet --run-id <RUN_ID>`. It validates the completed tree and writes
   the single durable `output/skill-gauntlet/<RUN_ID>.results.zip` atomically. A run is not deliverable
   until that archive exists. Never create the archive manually or report success if packaging fails.

Report the absolute result-archive path, run-root path, and any unresolved/error case IDs. Do not
invoke a grader. Local grading deliberately runs after OneDrive collection so the grader cannot infer
mode from Cowork's installed skill registry.

From andreas-deruiter/skill-gauntlet, Apache-2.0.

More from andreas-deruiter/skill-gauntlet