skillopt-sleep

Files (1)

• SKILL.md

SKILL.md (first lines)

# SkillOpt-Sleep: usage-driven self-evolution for the dsh agent

SkillOpt-Sleep is Microsoft's [SkillOpt](https://github.com/microsoft/SkillOpt)
deployment-time companion engine: it reviews your past sessions (harvest), mines
recurring tasks (mine), replays them through a selected backend (replay), and
consolidates what it learns into skill documents behind a **held-out validation
gate** (consolidate).

This skill drives the engine through the 7 `skillopt_*` tools exposed by the
dsh-skillopt plugin. The default `mock` backend makes no model calls, which is
useful for verifying the plumbing; a real backend consumes your API budget.

## When to use

- "make my agent better the more I use it" / "learn my preferences across sessions"
- a one-off **offline self-evolution / sleep / dream** run (immediate or scheduled)
- review past sessions/trajectories and distill recurring tasks
- consolidate feedback into `AGENTS.md` / `SKILL.md` / managed skills
- schedule (cron) the cycle, or adopt a staged proposal

## The cycle (six stages)

1. **Harvest** — read-only scan of supported local session records → digests
2. **Mine** — digests → recurring task records (intent + outcome labels + checkable refs)
3. **Replay** — re-run tasks under the current skill+memory with the selected backend → (hard, soft) scores
4. **Consolidate** — reflect on failures → propose bounded edits → **validation gate** on a held-out slice (default: accept only on strict improvement)
5. **Stage** — write accepted proposals to `<project>/.skillopt-sleep/staging/<timestamp>/`. **Live files are unchanged.** A rejected run still has a report but no proposal files.
6. **Adopt** — explicit (or operator-configured `--auto-adopt`) copies staged files over live ones, backing up first.

## Driving it

Prefer the tools over hand-editing files:

| Tool | Behavior |
|---|---|
| `skillopt_status` | state, engine availability, latest staged proposal & report |
| `skillopt_dry_run` | full preview (harvest+mine+replay), stages nothing |
| `skillopt_run` | full cycle, stages a proposal (live files unchanged by default) |
| `skillopt_adopt` | apply latest staged proposal (with backup) — the live-change boundary |
| `skillopt_harvest` | read-only show/export of mined tasks |
| `skillopt_schedule` / `skillopt_unschedule` | install/remove the nightly cron entry for this project |

Typical flow:

```text
# 1. check state (default mock backend, zero cost)
skillopt_status

# 2. preview the cycle
skillopt_dry_run project=<dir> source=<claude|codex|…>

# 3. real run (consumes the selected backend's API budget)
skillopt_run project=<dir> backend=<codex|claude|…> preferences="Prefer pytest; keep commits imperative."

# 4. review the report, then adopt
skillopt_adopt project=<dir>

# 5. schedule nightly at 03:17
skillopt_schedule project=<dir> hour=3 minute=17 backend=<codex>
```

## Parameters

| Parameter | Default | Meaning |
|---|---|---|
| `project` | config or cwd | project directory to evolve |
| `backend` | `mock` | `mock\|claude\|codex\|copilot\|cursor\|pi\|opencode\|handoff\|azure_openai` (mock = no model calls) |
| `source` | config | transcript source: `claude\|codex\|copilot\|cursor\|pi\|opencode\|auto` |
| `model` | backend default | replay model override |
| `maxTasks` | 40 | mined-task cap |
| `preferences` | empty | house rules for the reflection prior (e.g. "always use async/await") |

## Configuration (cordis.yml / bundle patch)

```yaml
- insert:
    - id: skillopt
      name: './src/index.js'
      config:
        backend: codex
        project: /path/to/project
        preferences: 'Always use async/await'
        # auto-adopt is OPERATOR-ONLY — the model cannot set it
        autoAdopt: false
```

Advanced engine keys go in `~/.skillopt-sleep/config.json`:
`gate_mode` (on/off), `gate_metric` (hard/soft/mixed), `gate_no_regression`,
`dream_rollouts`, `recall_k`, `evolve_memory` / `evolve_skill`.

## Hard rules

- **Never** hand-edit `AGENTS.md` / `SKILL.md` around `skillopt_adopt`; let the
  engine's explicit adopt (or operator-configured `--auto-adopt`) apply the
  staging manifest, backing up live files first.
- Harvest is read-only; `mock` replay has no side effects.
- Real backends send truncated transcript excerpts and derived tasks to the
  selected provider for mining/replay/judging/reflection. For sensitive
  sessions, export tasks first (`skillopt_harvest output=<file>`), redact, set
  the top-level `"reviewed"` to `true`, then replay with `--tasks-file`; real
  backends refuse unreviewed task files.
- Show the user the **held-out baseline → candidate** score and the exact
  proposed edits before suggesting adoption. Evidence before adoption.

## Validate / demo (no API spend)

```bash
pip install skillopt
python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves
```

Deterministic synthetic demo: the score rises and the gate blocks a regression.
It validates the mechanism, not effectiveness on your own tasks.

See the [SkillOpt-Sleep docs](https://github.com/microsoft/SkillOpt/tree/main/docs/sleep)
for recorded results and limitations.

From microsoft/SkillOpt, MIT.

More from microsoft/SkillOpt