aide-background-tasks

Files (1)

• SKILL.md

SKILL.md (first lines)

# Background Tasks — Durable Outbox, Drain Worker, Survive Crashes

Born 2026-08-31 from the wiring audit. AIDE has no way to "kick off a long task and come back later". A `git commit` that takes 30 minutes, a `claude-style "review every PR over the weekend"` loop, a `download 50GB of model weights` — all of these need to survive daemon restarts, machine sleeps, and operator distraction. Cursor, Claude Code, Copilot all ship this. AIDE doesn't. This skill IS the wire-in.

## Why this matters

- **Real workflows are long**: training, downloading, code review, scheduled cleanup. None of these should block the user.
- **Crashes happen**: machine sleep, power loss, daemon restart. Anything in-memory is lost. Anything in the outbox survives.
- **No-Brick-Wall**: the user can close the laptop, come back tomorrow, the task is still running or has a result.
- **Cross-device**: paired with the Telegram bridge, the user starts a task on the desktop, gets a Telegram ping when it's done.

## The AIDE background-task contract (4 hard rules)

1. **A task is an idempotent JSONL entry** in `<workspace>/.aide/outbox.jsonl`. Each entry has `{idempotency_key, kind, payload, state, created_at, attempts, last_error, scheduled_for}`. Append-only.
2. **The drain worker owns execution**. It reads pending entries, executes them, writes the result back to the same entry. The worker survives crashes via `kill -9` recovery: on restart, it reads the outbox, finds `state === 'pending'` or `state === 'in_flight'`, replays them (idempotency key prevents double-execution).
3. **All outbound calls carry the idempotency key**. The same key + same payload = no double work. The same key + different payload = hash compare, the newer wins, the older is dropped.
4. **Jittered backoff on failure**. 1s, 2s, 4s, 8s, ..., capped at 5min. After 10 attempts, the entry moves to `state: 'failed'` and surfaces in the UI.

## Files to touch (when wiring)

| File | Change |
|---|---|
| `node/src/services/outbox.mjs` | NEW: `Outbox` class with `enqueue`, `claim`, `complete`, `fail`, `replay`, `list`, `purge`. |
| `node/src/routes/tasks.ts` | EXTEND: `POST /api/tasks/enqueue`, `GET /api/tasks/outbox`, `GET /api/tasks/outbox/:id`, `POST /api/tasks/outbox/:id/cancel`, `GET /api/tasks/outbox/stats`. |
| `node/src/services/drain-worker.mjs` | NEW: the worker loop. Reads the outbox every 5s, claims pending entries, dispatches by `kind`, writes the result. |
| `common/contracts/tasks.ts` | EXTEND: the 5 request/response zod schemas. |
| `node/src/server.ts` (or the routes aggregator) | Wire the new routes. Start the drain worker on boot. |
| `tests/arch/background-tasks.test.ts` | NEW: 5 tests (enqueue/claim/complete, replay after kill, backoff, dedup). |
| `scripts/aide-bundle.cjs` | (optional) `bundle task list` CLI. |
| `browser/src/...` | (optional) outbox badge in the status bar. |

## The contract (zod-strict)

```ts
// in common/contracts/tasks.ts

export const OutboxEntryKind = z.enum([
  'cloud_turn',         // long model call (e.g. opus-4.6 review)
  'download',           // HF model / file / repo download
  'training_run',       // full training job
  'agent_session',      // a long agent loop (e.g. multi-hour migration)
  'hook',               // a deferred hook call
  'desktop_action',     // a desktop control action (with grant)
  'cleanup',            // garbage collection, index rebuild
  'custom'              // user-defined
]);


## The outbox service

```js
// node/src/services/outbox.mjs
import { promises as fs } from 'node:fs';
import { randomUUID } from 'node:crypto';
import path from 'node:path';

export class Outbox {
  constructor({ workspace, file = '.aide/outbox.jsonl' }) {
    this.workspace = workspace;
    this.file = path.join(workspace, file);
  }

  // Append a new entry. Idempotency: if the same key exists in the file
  // with a hash-equal payload, return the existing entry (dedup).
  async enqueue({ kind, payload, scheduled_for, idempotency_key }) {
    const key = idempotency_key || randomUUID();
    const existing = await this.findByKey(key);
    if (existing && JSON.stringify(existing.payload) === JSON.stringify(payload)) {
      return existing;  // dedup
    }
    const entry = {
      idempotency_key: key,
      kind,
      payload,
      state: 'pending',
      created_at: Date.now(),
      scheduled_for: scheduled_for || null,
      attempts: 0,
      last_error: null,
      result: null,
      started_at: null,
      ended_at: null
    };
    await fs.appendFile(this.file, JSON.stringify(entry) + '\n', 'utf8');
    return entry;
  }

  // Claim: atomically read + mark a pending entry as in_flight.
  // Returns the claimed entry or null if none pending.
  async claim() {
    const entries = await this.readAll();
    const pending = entries.find(e => e.state === 'pending'
      && (e.scheduled_for === null || e.scheduled_for <= Date.now()));
    if (!pending) return null;
    pending.state = 'in_flight';
    pending.started_at = Date.now();
    pending.attempts += 1;
    await this.writeAll(entries);
    return pending;
  }

  async complete(idempotency_key, result) {
    const entries = await this.readAll();
    const e = entries.find(x => x.idempotency_key === idempotency_key);
    if (!e) return null;
    e.state = 'completed';
    e.result = result;
    e.ended_at = Date.now();
    await this.writeAll(entries);
    return e;
  }

  async fail(idempotency_key, error) {
    const entries = await this.readAll();
    const e = entries.find(x => x.idempotency_key === idempotency_key);
    if (!e) return null;
    e.last_error = String(error?.message ?? error);
    if (e.attempts >= 10) e.state = 'failed';
    else e.state = 'pending';  // retry

From AnonymousNomad/covert-coder, Apache-2.0.

More from AnonymousNomad/covert-coder