capture

Files (14)

• SKILL.md
• beats.mjs · examples/ferry-draft/
• draft.config.json · examples/ferry-draft/
• package-lock.json
• package.json
• capture.mjs · scripts/
• encode.sh · scripts/
• inventory.mjs · scripts/
• locate.mjs · scripts/
• montage.sh · scripts/
• motion.mjs · scripts/
• narrative.mjs · scripts/
• run.mjs · scripts/
• selfcheck.mjs · scripts/

SKILL.md (first lines)

# demo:capture

Capture real product footage from a live app. Compose it elsewhere.

Pairs with **demo:compose** (the sibling skill in this plugin): it takes the take directory this
skill writes, plus one config carrying the brand and copy, and produces the finished cut.

```
beats.mjs ──▶ run.mjs ──▶ capture.mjs ──▶ <out>/<chapter>/ cfr/ + events.json
                              │      ▲                  │
                 mintState (login)   motion.mjs     encode.sh ──▶ <chapter>.mp4
                                     (springs)
                                                        │
                                          montage.sh (grade the take)  ──▶ demo:compose
```

## Use / do not use

- USE for: authenticated SPA flows, public multi-page apps, feature walkthroughs, re-shooting one chapter after a UI change.
- DO NOT use for annotation, spotlight, labels, chapter cards, transitions, music, final cut. That is `demo:compose`, which renders through HyperFrames (`hyperframes`, `hyperframes-core`, `hyperframes-keyframes`, `hyperframes-animation`). HyperFrames renders video FROM HTML and cannot drive an authenticated SPA, which is why capture is a separate step.
- Keep footage free of annotations. Annotations then stay re-timeable without re-recording. The only things burned in are the synthetic cursor and click ripple.

## Run

Skill dir: this skill's own directory, referred to below as `$SKILL_DIR`. It pins `@playwright/test@1.62.0`.
First run only, from that directory: `npm ci`, then
`PLAYWRIGHT_BROWSERS_PATH="$PW" npx playwright install chromium`, where `$PW` is a writable directory you keep
(for example `~/.local/pw-browsers`). Neither `node_modules` nor the browsers ship with the plugin.

```bash
cd "$SKILL_DIR"
npm run check                                   # self-check, ~9 min, no app needed
node scripts/run.mjs /path/to/beats.mjs --dry-run       # every target resolved, no frames: seconds
node scripts/run.mjs /path/to/beats.mjs [chapter ...]   # dry run, then capture + encode each chapter
scripts/montage.sh <out>/<chapter>.mp4          # 4x3 contact sheet; LOOK at it
node scripts/inventory.mjs <url>                # what a beat can target on a page (see Drafting beats)
```

- Browsers: Playwright reads `PLAYWRIGHT_BROWSERS_PATH` when it is set and otherwise uses its own default (`~/.cache/ms-playwright` on Linux). Export `PLAYWRIGHT_BROWSERS_PATH="$PW"` in the shell that runs the rig and keep that directory stable; the scripts never set it for you. The installed revision must match `node_modules/playwright-core/browsers.json` (1.62.0 wants chromium 1234). On mismatch: `PLAYWRIGHT_BROWSERS_PATH="$PW" npx playwright install chromium`. Prefer a directory you control over `~/.cache/ms-playwright`, which other tooling clears: measured wiped twice on one machine with 34G free, so not disk pressure.
- Write beats files and `out` under a scratch dir, never in a project repo.
- Narration (composing a demo whose config has `narration`) runs `hyperframes tts`, which reads
  `HYPERFRAMES_PYTHON`: point it at a venv with `kokoro-onnx` and `soundfile` installed.
- After every take: open the montage. Check head is not blank/black, tail is not a splash, zoom frames are sharp.
- `run.mjs` prints each chapter's mode, fps, density and wall time, e.g.
  `fares: deterministic 60fps x2, 659 frames, 11.0s, 2 marks, filmed in 210s`. A chapter that fell back
  says `screencast 30fps x1` and the line above it gives the cause.

## Capture modes

```
capture.mode
  deterministic (default) ──▶ beginFrame + virtual time ──▶ PNG 2560x1440 @ exactly 60fps
        │  a request holds page time: network 'auto' forces it on (see below)
        │  no beginFrame / page time stuck even when forced / hard timeout / wrong frame size
        ▼  (one warning naming the cause, chapter re-filmed from scratch)
  screencast ──────────────▶ CDP screencast ──▶ JPEG 1280x720, resampled to 30fps
```

- **deterministic**: chrome-headless-shell runs under `HeadlessExperimental.beginFrame` with
  virtual time (`pauseIfNetworkFetchesPending`). The rig renders every frame on demand at exactly
  `n / fps` of page time, so CSS animations, `requestAnimationFrame` and timers all step on the
  footage clock. Frames come back at `--force-device-scale-factor`: 2560x1440 lossless PNG for a
  1280x720 viewport. Network latency is edited out (page time stands still while a fetch is in
  flight). Camera and cursor are posed every frame by the spring solver (`motion.mjs`). Costs
  several times real time: see the measured facts.
- **Requests that hold page time.** `pauseIfNetworkFetchesPending` waits for every request in
  flight, and two kinds never let it go: a stream that stays open (SSE), and, measured, any
  dynamic `import()` of a module, even an instant one, so every code-split route. With
  `network: 'auto'` (the default) a frame that waits `unstickMs` (1.5s of wall time) is forced
  through with policy `advance`; a lazy route then costs 1.5s once and the take stays fully
  deterministic. Chrome does not say which request holds page time, so a stream is recognised by
  behaviour: a request still in flight at two stalls, or stalls on three frames running (which
  also catches a holder no request event shows). Then the rest of the chapter runs under `advance` (page time moves on regardless of the network, so a
  load shows for its wall time, roughly a tenth of its real length), the rig says so once, and
  `capture.network` / `capture.networkNote` in events.json record it. `network: 'pause'` never
  forces and falls back to screencast after `stallMs` instead; `'advance'` forces from the start.
- **Why `auto` is the default.** Almost every modern SPA lazy-loads its routes, and under strict
  `pause` each of those loads freezes page time for good: a real app's first deterministic take
  fell back to screencast on its first route change. `auto` keeps those apps deterministic, edits
  network latency out everywhere else, and only gives that up (saying so) when a stream really
  holds page time. Use `pause` when the footage must never show a load at anything but its edited
  length, and accept a screencast fallback on apps that need it.
- **screencast**: the real-time path. It is used when deterministic capture cannot proceed: the
  browser has no beginFrame, page time stops even when forced, the chapter passes `timeoutMs` of
  wall time, frames come back at the wrong size, or `network: 'pause'` meets a request that never
  finishes. The rig prints one line naming the cause, re-films the
  chapter in screencast mode and records both in `events.json` (`capture.mode`,
  `capture.requested`, `capture.fallback`). With a `login`, it checks the session again first
  (and re-mints it): the failed take ran its beats, and one that signs out revoked it. It never
  hands over a broken take. Set
  `capture: { mode: 'screencast' }` to choose it outright.
- **Websockets do not stall it** (measured: a page receiving a frame every 200ms of wall time
  filmed a 2s hold deterministically, and a real SPA holding a Supabase realtime websocket and a
  Vite HMR socket filmed a home, click, lazy route, zoom chapter deterministically: 7.5s in 164s,
  network still `pause`). Their messages arrive on the wall clock, which runs several times slower
  than page time here, so live pushes land sooner in the footage than they would for a real viewer.

```js
capture: {
  mode: 'deterministic',  // or 'screencast'
  dpr: 2, fps: 60,        // pixel density and frame rate (deterministic only)
  blur: { samples: 4, max: 64, spacing: 1.5, shutter: 0.5, threshold: 2 },  // on by default; false for drafts, spacing 4 in between
  network: 'auto',        // or 'pause' / 'advance': requests that hold page time, see above
  unstickMs: 1500,        // wall ms a frame waits on the network before 'auto' forces it
  stallMs: 10000,         // wall ms before falling back: page time stuck, or any wait under 'pause'
  timeoutMs: 600000,      // wall ms a chapter may take before falling back; by default 120x its estimated footage when that is longer
}
```

**Motion blur** (deterministic only, ON by default). Pick the setting by how final the take is.
Measured on the ferry example (8 cores, no GPU, one take at a time):

| setting | capture time | a 5-minute demo films in | fast moves look |
|---|---|---|---|
| `capture: { blur: false }` (drafts) | about 6.5x real time | about 33 minutes | sharp, stepping frame to frame |
| `capture: { blur: { spacing: 4 } }` (middle ground) | about 17x | about 1 hour 25 minutes | smeared, faint steps up close |
| default (final takes) | about 35x | about 2 hours 55 minutes | smooth smear |

Screencast mode films at 1x. Set the option file-wide or per chapter: the footage is the same 2x,

From stevengonsalvez/ainb-toolkit, MIT.

More from stevengonsalvez/ainb-toolkit