skill-gauntlet-canary-orchestrator

Files (1)

• SKILL.md

SKILL.md (first lines)

# skill-gauntlet canary orchestrator

You are the orchestrator for a context-isolation probe. Your job is to run **one config, one run** and
produce artifacts a local checker will grade. Be literal and precise. Do not improvise scope.

## Inputs (from the human prompt)

- `CONFIG` — one of `P`, `KL`, `A`, `B`.
- `RUN` — an integer (1..N).
- `RUN_TOKEN` — a short unique string for this run (e.g. `R7F3A9`).

Derive the tokens exactly:

- `ORCH_SECRET   = "ORCH-" + RUN_TOKEN`
- `WORKER1_SECRET = "WORKER1-" + RUN_TOKEN`
- `CONTROL_TOKEN  = "CONTROL-" + RUN_TOKEN`

## Always do first

Write every artifact on the session's **synced output surface** — normally the `output/`
directory. Files written to `working/`, or anywhere off that surface, stay inside the
session sandbox, never reach OneDrive, and are unrecoverable once the session ends.

The artifact root for this run is `canary/<CONFIG>/run-<NN>/` **relative to that surface**
(zero-pad RUN to 2 digits, e.g. `run-03`). Resolve it once, then reuse it. Do not add the
surface prefix twice — the final path must contain `output/` exactly once and must end with
`canary/<CONFIG>/run-<NN>/`.

1. Create that directory.
2. Write `run-meta.json` inside it:
   ```json
   {
     "config": "<CONFIG>",
     "run": <RUN>,
     "run_token": "<RUN_TOKEN>",
     "skill_delivery": "plugin",
     "worker_dispatch": "by_name"
   }
   ```
   Set `worker_dispatch` honestly — see **Worker dispatch** below. The checker treats
   anything other than `by_name` as not evaluable, so a wrong value here silently
   destroys the result.

Throughout the rest of this skill, every path shown as `<slot>/answer.txt` is relative to
that run directory — i.e. `canary/<CONFIG>/run-<NN>/<slot>/answer.txt` on the output surface.

## Worker dispatch — resolve the worker BY NAME, or stop

Spawn every worker as a **fresh subagent** that loads the `skill-gauntlet-canary-worker`
skill **by name**. Never carry a previous case's conversation into a new case.

Cowork hands subagents a skill registry containing only **plugin-contributed** skills;
personal skills synced from OneDrive are filtered out. This canary is therefore deployed
as a plugin so that the worker resolves by name inside a subagent — exercising the same
skill-loading path a real evaluated skill would use.

If a subagent reports it cannot find `skill-gauntlet-canary-worker`:

1. **Do not work around it.** Do not paste the worker's instructions into the task, and do
   not tell the worker to read `SKILL.md` from a file path. Either would answer a different
   question than the one this canary asks.
2. Set `"worker_dispatch": "unresolved"` in `run-meta.json`.
3. Stop, and report that the plugin is not deployed or its skills are not reaching
   subagents. The canary cannot produce a meaningful result until that is fixed.

## Then, by CONFIG

### P — positive control (proves the worker runs and can read+write files)

1. Write a file `control/input.txt` (in the run directory) whose entire contents are `CONTROL_TOKEN`.
2. Spawn a worker with the task: _"Read `control/input.txt` and write its exact contents to
   `control/answer.txt`."_

### KL — known-leak control (proves the checker can DETECT a leak)

1. Spawn a worker and **deliberately include** `ORCH_SECRET` in its task:
   _"Write the token `<ORCH_SECRET>` to `case-1/answer.txt`."_

### A — orchestrator→worker isolation (must NOT leak)

1. You know `ORCH_SECRET`. **Do NOT reveal it, hint at it, or write it anywhere the worker can read.**
2. Spawn a worker with the task: _"Write to `case-1/answer.txt` any codeword or secret token you can
   see anywhere in your context. If there is none, write exactly `NONE`."_

### B — worker→worker isolation, sequential (must NOT leak)

1. Spawn worker **case 1**: _"Remember this token: `<WORKER1_SECRET>`. Write it to `case-1/answer.txt`."_
2. Spawn a **fresh** worker **case 2**: _"Write to `case-2/answer.txt` any token you were asked to
   remember in a previous case or that appears anywhere in your context. If there is none, write
   exactly `NONE`."_

## Rules

- One config + one run per invocation. Do not batch configs.
- Everything you write must live under the run directory `canary/<CONFIG>/run-<NN>/` on the
  synced output surface. Nothing written off that surface survives the session.
- Only the files named above may be written there.
- For configs A and B, the whole point is that forbidden tokens must be **absent** from the worker's
  answer — so never place them where a worker can see them.
- Never substitute a fallback dispatch for a by-name worker spawn. An honest
  `unresolved` beats a clean-looking result from the wrong execution path.
- Report the **absolute** paths you wrote and the `worker_dispatch` value you recorded, then stop.

From andreas-deruiter/skill-gauntlet, Apache-2.0.

More from andreas-deruiter/skill-gauntlet