browser-capture

Files (8)

• SKILL.md
• mcp-recorder.js · assets/
• gotchas.md · references/
• long-takes.md · references/
• other-routes.md · references/
• sanitized-screenshots.md · references/
• raise-chrome.sh · scripts/
• record-flow.js · scripts/

SKILL.md (first lines)

# browser-capture

Turn a browser flow into image or video files you can embed. Three routes, in strict order
of preference. Each obvious shortcut here fails *silently*, which is how this costs hours
instead of minutes — so pick the route first.

Dependencies: **ffmpeg** on PATH. Route 2 also needs **puppeteer-core** (the viz skill ships
one at `~/.claude/skills/viz/node_modules/puppeteer-core`; `record-flow.js` finds it).

## Pick the route

| | Route | Real login/SSO? | Human? | Use when |
|---|---|---|---|---|
| **1** | **MCP capture — the user's own Chrome** | ✅ **yes** | ~1s, no clicks | **Default. Prefer always.** |
| **2** | Spawned browser (`record-flow.js`) | ❌ no session | none | Route 1 impossible: no GUI, screen must not be touched, CI/background, or deterministic scripted choreography |
| **3** | GIF recorder (`gif_creator`) | ✅ yes | none | You specifically want a click-annotated how-to GIF and don't need smooth motion |

**Prefer Route 1 by default.** It records the user's *actual* browser — their real session,
their SSO, their data, at their real device pixel ratio (Retina). Nothing else can do that.
Route 2 is a capable fallback that never touches the screen. Route 3 is only for annotated
click-by-click GIFs — and note you can always convert a Route 1/2 video to GIF with ffmpeg,
which usually looks better than the GIF recorder anyway.

**The GIF trap:** `gif_creator` yields a **slideshow**, always — one frame per *action*, and
extra `screenshot` calls add **zero** frames. If a GIF looks "janky" or "choppy", that isn't
a quality bug to iterate on; it's Route 3 working as designed. Switch routes, don't try harder.

**Sizing a GIF for where it lands.** Converting a video to GIF is unbounded by default,
and the destination is what actually constrains it. Budgets worth hitting:

| Destination | Size | Frame rate | Colors | Length |
|---|---|---|---|---|
| Slack custom emoji | 128×128 | 10–20 fps | 48 | under 3s, must loop |
| Slack message / PR / Confluence | up to 480×480 | 15–30 fps | 64–128 | under 10s |

Colors are the cheapest lever — an `ffmpeg` palettegen pass at 64 colors is usually
indistinguishable and a fraction of the size. Dropping frames beats dropping dimensions
for a UI recording, because unreadable text is worse than slightly choppy motion.

**Routes 2 and 3 are written up in [`references/other-routes.md`](references/other-routes.md).**
Everything below is Route 1.

---

# Route 1 — record the user's real Chrome (default)

Records the tab the MCP is already driving, via `getDisplayMedia({preferCurrentTab})` +
`MediaRecorder`, downloading through a real `<a download>` click. Full frame rate, real
device pixel ratio, real session. **No picker appears and the user clicks nothing.**

### The one constraint

All three must hold **at the instant capture starts** — and only then:

1. the target tab is the **selected** tab in its window
2. its window is **frontmost / not occluded by another app**
3. a **trusted click** lands within ~5s (the MCP `computer` tool's clicks *are* trusted)

Miss any one → `InvalidStateError: Invalid state`. That error means **"tab not visible"** —
it does *not* mean capture is impossible. macOS marks a Chrome page `hidden` when its window
is **occluded by another app**, so an editor sitting on top is enough to break it.

**Once capture is live, Chrome pins the tab visible and none of it matters any more.**
Verified: with a recording running, burying Chrome behind another app kept
`visibilityState: "visible"` with `hasFocus: false` and the counter advancing exactly in real
time (50 ticks / 5.0s). The user can switch apps, cover the window and keep working.

> **Unverified:** switching to a *different Chrome tab* mid-capture. Occlusion is proven safe;
> tab-switching is probably fine (it's how Meet tab-sharing behaves) but has not been tested.
> Until it is, tell the user to avoid switching tabs while recording.

### The procedure

```
0. TELL THE USER what they are about to see (one line in chat, before step 2)
1. javascript_tool : inject assets/mcp-recorder.js         -> window.__cap
   (optional)      : inject the `ui-narration` skill       -> window.__narrate (cursor/highlight)
2. javascript_tool : window.__cap.armButton()              -> full-viewport click target
3. bash            : scripts/raise-chrome.sh <url-substring>
4. computer        : SCREENSHOT, then click the target's centre in THAT screenshot's coords
5. javascript_tool : window.__cap.state === 'RECORDING'    (else window.__cap.diagnose())
6. javascript_tool : window.__cap.park() -> coords; computer: click them (parks the real
                     OS pointer out of frame — getDisplayMedia captures the system cursor)
7. drive the flow  : javascript_tool / computer
8. javascript_tool : await window.__cap.stop()             -> ~/Downloads/<filename>
9. bash            : verify + convert (below)
```

**Anything longer than a few seconds: read
[`references/long-takes.md`](references/long-takes.md) first.** Steps 5–7 collapse into one
gated loop there, because checking `state` and then driving in separate calls loses every beat
that runs before the user has approved capture.

### Step 0 — say something first (do not skip)

`armButton()` throws a **red overlay across the user's entire screen**, and raise-chrome.sh
then yanks Chrome in front of whatever they were doing. **You** click the overlay — the MCP
`computer` tool's clicks are trusted, so the user never has to touch it. But they don't know
that, and from their side a giant red panel just ate their screen. Reported verbatim by a
user: *"I had no idea I, as a human, was supposed to click that."*

One line in chat before you arm it is enough:

> Recording the browser now — you'll see a red overlay flash across Chrome for a second.
> Ignore it, I click it myself; nothing is needed from you.

Same reason the default label is worded at the human ("Claude is starting a screen
recording / No action needed") rather than as the imperative "CLICK TO START RECORDING".
If you override `label`, keep that property.

**Step 4 is where this actually goes wrong.** Screenshot dimensions change between calls
(observed 1456×840 then 1502×818 for the same 1728px viewport), so a scale factor computed
earlier silently misses. **Always screenshot immediately before clicking and use that image's
coordinates.** `armButton()` covers the whole viewport for exactly this reason. If a click
seems to do nothing, verify events are landing at all:

```js
window.__clicks = [];
document.addEventListener('click', e => window.__clicks.push([e.clientX, e.clientY, e.isTrusted]), true);

From RascalTwo/ai-setup, MIT.

More from RascalTwo/ai-setup