authoring-skills

Files (31)

• SKILL.md
• evals.json · evals/
• pending-task.md · evals/fixtures/build-baseline/.opencode/plans/
• AGENTS.md · evals/fixtures/build-baseline/
• SKILL.md · evals/fixtures/build-baseline/
• package.json · evals/fixtures/build-baseline/
• SKILL.md · evals/fixtures/event-streams/authoring-skills__1/
• evals.json · evals/fixtures/event-streams/authoring-skills__1/evals/
• events.jsonl · evals/fixtures/event-streams/authoring-skills__1/
• SKILL.md · evals/fixtures/event-streams/authoring-skills__2/
• evals.json · evals/fixtures/event-streams/authoring-skills__2/evals/
• events.jsonl · evals/fixtures/event-streams/authoring-skills__2/
• events.jsonl · evals/fixtures/event-streams/designing-architecture__1/
• feat-offline-queue.md · evals/fixtures/event-streams/designing-architecture__1/
• events.jsonl · evals/fixtures/event-streams/implementing-features__1/
• offline-queue.ts · evals/fixtures/event-streams/implementing-features__1/src/lib/
• events.jsonl · evals/fixtures/event-streams/observing-runs__1/
• REVIEW.md · evals/fixtures/event-streams/reviewing-code__1/
• events.jsonl · evals/fixtures/event-streams/reviewing-code__1/
• ROADMAP.md · evals/fixtures/event-streams/triaging-requirements__1/
• events.jsonl · evals/fixtures/event-streams/triaging-requirements__1/
• events.jsonl · evals/fixtures/event-streams/writing-unit-tests__1/
• duration.test.js · evals/fixtures/event-streams/writing-unit-tests__1/tests/
• anthropic-best-practices.md · references/
• eval-assertions.md · references/
• eval-harness.md · references/
• opencode-spec.md · references/
• run_behavioral_eval.py · scripts/
• test_runner.py · scripts/
• test_validate_markers.py · scripts/
• validate_skill.py · scripts/

SKILL.md (first lines)

# Authoring Skills

Author skills that a mid-tier model can execute reliably. The context window
is a public good: every line must justify its token cost once loaded.

Detailed guidance lives in two references — consult them at the steps below:

- [references/anthropic-best-practices.md](references/anthropic-best-practices.md)
  — distilled authoring guidance: progressive disclosure, descriptions,
  degrees of freedom, patterns, anti-patterns, quality checklist
- [references/opencode-spec.md](references/opencode-spec.md)
  — frontmatter rules, name validation, discovery paths, permissions

## The authoring loop

Copy this checklist and check off items as you complete them:

```
Authoring Progress:
- [ ] Step 1: Capture intent
- [ ] Step 2: Interview for specifics
- [ ] Step 3: Draft evals BEFORE the skill body
- [ ] Step 4: Draft the skill
- [ ] Step 5: Self-review against the checklist
- [ ] Step 6: Test with a fresh agent
- [ ] Step 7: Iterate on observed behavior
- [ ] Step 8: Finalize and install
```

### Step 1: Capture intent

Determine what the skill should enable and when it should trigger. If the
current conversation already contains the workflow (the user says "turn this
into a skill"), extract from history first: tools used, step sequence,
corrections the user made, input/output formats. Confirm the extraction with
the user before proceeding.

### Step 2: Interview for specifics

Ask about edge cases, input/output formats, success criteria, and
dependencies. Do not guess — the cost of one clarifying question is far
below the cost of a skill built on a wrong assumption. Establish:

1. What should the skill enable?
2. What user phrases/contexts should trigger it?
3. What is the expected output format?
4. Is the output objectively verifiable (favors eval assertions) or
   subjective (favors human review)?

### Step 3: Draft evals before the skill body

Write 2–3 realistic test prompts to `evals/evals.json` in the skill
directory — the kind of thing a real user would actually type, concrete and
specific (file paths, situational backstory, casual phrasing), not abstract
requests. Share them with the user for approval. Evals-first keeps the skill
solving real problems instead of documenting imagined ones.

```json
{
  "skill_name": "example-skill",
  "evals": [
    {
      "id": 1,
      "prompt": "Realistic user request here",
      "expected_behavior": [
        "Observable outcome 1",
        "Observable outcome 2"
      ],
      "files": ["fixtures/sample-input.json"]
    }
  ]
}
```

**Evals must be self-contained.** Any file an eval depends on lives in
`evals/fixtures/` inside the skill directory and is listed in `files[]`
(paths relative to `evals/`). Fixtures left in `/tmp` or pointing at a
live repo make the eval unreproducible after the environment resets —
which means no regression testing on the next edit. Embedding fixture
content directly in the prompt is equally valid for text sources.

**Verifiers in fixtures must be able to fail.** When the skill under
test closes its loop on command exit codes, the fixture's verification
commands must actually execute, exit non-zero while the work is absent
or wrong, and exit 0 when it is done correctly (small script stubs that
assert on the produced files are fine). An eval whose verification
cannot fail proves nothing about the skill's feedback loop.

**Batch the first commit to a green validator.** During a new skill's
development, the pre-commit hook assumes complete state — `evals/`
exists, `SKILL.md` exists, the skill dir is registered. Commit the
skill dir only when `python3 skills/authoring-skills/scripts/
validate_skill.py --all` is green. A mid-development commit that lands
earlier (e.g. evals present but the body unregistered) requires
`--no-verify`; that bypass is honest only with a dated record in the
scratch log (the cardinal rule applies — never green a check by
weakening the net). The `facilitating-product-vision` D1/D2 sequence
(2026-10-09, `.scratch/fpv-impl/progress/d1-evals.log`) is the worked
example of the bypass-with-record path.

### Step 4: Draft the skill

Create `skills/<name>/SKILL.md` plus bundled resources as needed. Follow the
library conventions below and the detailed guidance in
[references/anthropic-best-practices.md](references/anthropic-best-practices.md).
Validate frontmatter against
[references/opencode-spec.md](references/opencode-spec.md).

Structure decisions:

- **Body <500 lines** (aim far lower). Approaching the limit means content
  belongs in `references/`.
- **`references/`** for detail loaded only when needed; link each file
  directly from SKILL.md with guidance on *when* to read it; one level deep
  only; TOC at the top of any reference >100 lines.
- **`scripts/`** for repetitive tasks with deterministic output. Signal to
  bundle a script: test runs show the model repeatedly writing the same
  helper. State clearly whether a script is to *execute* or to *read as
  reference*.

From hierovision/ai-framework, MIT.

More from hierovision/ai-framework