heartbeat

Files (1)

• SKILL.md

SKILL.md (first lines)

> **${var}** — selector. **Empty (default)** = the ambient fleet check — the live path a cron runs once a day; leave it empty for the scheduled run. **`brief`** = the priority brief. See the grammar below.

## Selector / `${var}` grammar

- **`` (empty)** — **Ambient check** across all skills / PRs / issues, and regenerate the public status page. This is the live, scheduled path (08:00 UTC daily); its behaviour is unchanged. Leave `${var}` empty for the cron.
- **`<area>`** (any non-empty value that is not `brief`, e.g. `crypto`, `prs`) — **Ambient check**, with the checks focused on that area (original heartbeat focus-area behaviour).
- **`brief`** — **Priority brief**: rank the 3 things to focus on today, why now, and what moved since yesterday; send via `./notify` + email.
- **`brief:<area>`** (e.g. `brief:crypto`) — **Priority brief** biased toward `<area>`.

## Shared setup (every run)

Read `memory/MEMORY.md` and the last 2 days of `memory/logs/` for context.

Parse `${var}` to pick the branch:
- **starts with `brief`** (i.e. `brief` or `brief:<area>`) → run the **Priority brief** branch. Any text after `brief:` is the emphasis area.
- **otherwise** (empty, or any other value) → run the **Ambient check** branch. A non-empty value is the focus area; empty runs all checks.

The two branches are mutually exclusive — run exactly one per invocation.

---

## Ambient check  (default — empty `${var}`; the LIVE scheduled path)

If `${var}` is set to a focus area, focus checks on that specific area.

### Checks (in priority order)

#### P0 — Failed & stuck skills (check first)

Read `memory/cron-state.json`. **If the file is missing or empty** (e.g. a fresh fork whose scheduler hasn't written it yet), treat state as empty: report `no cron-state yet` for the P0 tier, skip the failure/degradation checks below, and still render the status page (every enabled skill shows `not yet run`). This file tracks every scheduled skill's state and quality metrics:
```json
{
  "skill-name": {
    "last_dispatch": "2026-04-06T12:00:00Z",
    "last_status": "dispatched|success|failed",
    "last_success": "2026-04-06T12:05:00Z",
    "last_failed": "2026-04-05T12:03:00Z",
    "total_runs": 10,
    "total_successes": 8,
    "total_failures": 2,
    "consecutive_failures": 0,
    "success_rate": 0.80,
    "last_quality_score": 4,
    "last_error": "error signature text"
  }
}
```

**Bootstrap grace (fresh / warming-up fleets).** Before flagging anything, decide whether the fleet has *completed* any run yet. A skill has **completed a run** if its entry has `total_runs ≥ 1` **or** a non-null `last_success`/`last_failed`. A skill that only ever shows `last_status: "dispatched"` (or has no entry) has **not** completed a run — it is *warming up*, the normal state right after a fork or after a skill is enabled, **not** a failure. The scheduler stamps `dispatched` at run *start* and the outcome only lands *after* the run finishes, so a just-dispatched skill legitimately has no outcome yet.
- **If no skill has completed any run yet, the whole fleet is bootstrapping** — expected on a fresh fork. Do **not** flag warming-up skills as failed or stuck, do **not** fire a notification, and set the overall status to `🟢 OK` with a warming-up note (see [Overall status](#overall-status)). Still render the status page (skills show `⏳ warming up` / `not yet run`), then end.
- **Otherwise** (some skills have completed runs) run the checks below, but keep the distinction: a skill that has *never completed a run* is never a 🔴 "stuck" — it belongs on the softer warming-up line.

**Self-reference.** Heartbeat is, by definition, running *right now*, so its own entry is never evidence of a problem:
- **Exclude heartbeat's own entry from the Stuck check.** Its `dispatched` watermark is just the current (or a prior in-flight) heartbeat run.
- The Self-check below fires **only once heartbeat has succeeded at least once**. A never-succeeded heartbeat is bootstrap, not degradation — its first success only lands *after* this run finishes, so "no success yet" must never turn the page red.

Flag these conditions:
- **Failed skills**: any entry with `last_status: "failed"`. Report the skill name and when it failed. (A skill whose only recorded outcome is a failure is still a completed run — report it; the severity rules decide whether it reddens the page.)
- **Stuck skills**: any entry (**excluding heartbeat itself**) with `last_status: "dispatched"`, `last_dispatch` **>45 minutes ago**, that has **completed ≥1 run before** (`total_runs ≥ 1`) and whose `last_dispatch` is newer than `last_success`. The skill was working, then a later dispatch never reported back — a hang, or a lost outcome-write. If `last_success` is recent (within ~2h of the stale dispatch), lean toward a lost outcome-write (a 🟡 blip), not a hard hang.
  - A skill dispatched >45min ago that has **never** completed a run is *warming up*, not stuck — put it on the warming-up line, not P0. Only if that first dispatch is **>24h** old, surface it as a 🟡 WATCH (`dispatched Nh ago, never completed — scheduler may not be wired up`); still not 🔴.
- **API degradation**: any skill with `consecutive_failures >= 3`. This likely indicates an external API is down or rate-limiting. Report the skill, failure count, and `last_error`. If multiple skills share similar error signatures, flag the shared dependency.
- **Chronic failures**: any skill with `success_rate < 0.5` (and `total_runs >= 5`). The skill is failing more than it succeeds.
- **Self-check**: only if heartbeat's own entry has **≥1 success** (`total_successes ≥ 1`) **and** its `last_success` is **>36 hours ago**, note that heartbeat itself may be unreliable. If heartbeat has never succeeded, say nothing here — that's warming-up, covered by Bootstrap grace above.

#### P1 — Stalled PRs & urgent issues

- [ ] Any open PRs stalled > 24h? (use `gh pr list`)
- [ ] Any GitHub issues labeled urgent? (use `gh issue list`)

#### P2 — Flagged memory items

- [ ] Anything flagged in memory/MEMORY.md that needs follow-up?

#### P3 — Missing scheduled skills

Read `aeon.yml` for enabled skills with schedules. Cross-reference with `memory/cron-state.json`:
- If an enabled skill has **no entry at all** in the state file, it has never been dispatched by the scheduler.
- If a skill's `last_success` is **>2x its schedule interval** old (e.g., a daily skill hasn't succeeded in >48h), flag it.

**Skip P3 entirely on a bootstrapping fleet** (per [Bootstrap grace](#p0--failed--stuck-skills-check-first) — no skill has completed a run yet). On a fresh fork *every* skill is un-dispatched or warming up; that is expected, not a fleet of missing skills, and must not generate findings or a notification. Only run P3 once the fleet has warmed (at least one completed run), and even then a skill still in its very first dispatch window is warming up, not missing.

Do NOT use `gh run list` for this — the state file is authoritative.

### Dedup & notification

Before sending any notification, grep memory/logs/ for the same item. If it appears in the last 48h of logs, skip it. Never notify about the same item twice.

Batch all findings into a **single notification**, grouped by priority tier:
```
🔴 FAILED: skill-a (failed 2h ago), skill-b (stuck 1h ago)
🟡 STALLED: PR #42 open 3 days
🔵 MEMORY: follow-up on X flagged 2 days ago
```

### Public status page

After the priority checks (even when everything is green — this step **always** runs), regenerate `docs/status.md` so it reflects current fleet health.

#### Data sources
- `memory/cron-state.json` — per-skill run state (authoritative)
- `memory/issues/INDEX.md` — open issue table
- `aeon.yml` — enabled skill list with schedules
- Latest `output/articles/token-report-*.md` (most recent by filename date) — optional; powers the Token Pulse section. Skipped silently when no file exists.

#### Overall status
Compute one of three overall states from the same signals used above. This verdict drives the **public, fork-facing** status page, so reserve 🔴 for skills that are *currently broken* — a single transient failure a skill has already recovered from must not flip the whole page red, and a fresh fork whose skills simply haven't finished their first cycle must never read 🔴:

**Bootstrap first.** If the fleet is *warming up* — no skill has completed a run yet (per [Bootstrap grace](#p0--failed--stuck-skills-check-first)) — the status is `🟢 OK`, annotated `🌱 warming up — N skill(s) dispatched, awaiting first completed run`. Skip the rest of this ladder. Warming-up skills (dispatched, never completed) never count toward 🔴 or 🟡 (except the >24h "may not be wired up" watch-item), and heartbeat's own entry never counts toward its own verdict.

Otherwise:
- `🔴 DEGRADED` — a skill is **currently and persistently broken**: a stuck skill (per the refined Stuck rule — completed ≥1 run before and a later dispatch has hung; **not** a warming-up first dispatch, **not** heartbeat's own entry); `consecutive_failures ≥ 3`; chronic failures (`success_rate < 0.5` with `total_runs ≥ 5`); heartbeat self-check >36h stale (only once heartbeat has ≥1 success); or a `last_status: "failed"` skill that has **not recovered since** (`last_failed` ≥ `last_success`) **and** `consecutive_failures ≥ 2`.
- `🟡 WATCH` — a transient blip or watch-item: a `last_status: "failed"` skill that already recovered (`last_success` > `last_failed`); **or** any other `last_status: "failed"` skill that does not meet the 🔴 bar above (e.g. a first or isolated failure, `consecutive_failures ≤ 1`, including a skill whose only run so far failed) — a non-recovered failure must never read 🟢 OK; a stuck skill whose `last_success` is recent (likely a lost outcome-write, not a hang); a warming-up skill whose first dispatch is >24h old (possibly not wired up); or any P1/P2/P3 flag (stalled PRs, urgent issues, flagged memory items, skills >2x their schedule interval old); or any open issue with severity `critical` or `high`.
- `🟢 OK` — no flags at all (a fully warmed, healthy fleet, or a bootstrapping fleet per the Bootstrap-first clause).

This refines **only** the public status-page colour. It does **not** change the P0 notification rules above — a fresh `last_status: "failed"` still fires its notification (deduped per the rules above) so the operator is always told; the page just won't read 🔴 for a blip the fleet has already shrugged off.

#### Format

Write `docs/status.md` with frontmatter so it renders as a status page:

From aeonfun/aeon, MIT.

More from aeonfun/aeon