chromatic-diff

Files (7)

• SKILL.md
• api.md · references/
• token.md · references/
• api.ts · scripts/
• preflight.ts · scripts/
• proxy.ts · scripts/
• resolve.ts · scripts/

SKILL.md (first lines)

# Chromatic Diff Investigation

Chromatic tells you *that* a snapshot changed. It does not tell you *why*. This skill closes that gap: it resolves a Chromatic URL into the two builds being compared, puts both published Storybooks on localhost, and renders the same story from each so the difference can be read as structure, computed style, and token values rather than pixels.

Working from the rendered page rather than the snapshot image matters, because the useful answers are named things — "the dark-theme surface token changed", "this element gained a wrapper", "both builds render identically, so the snapshot is flaky". A pixel diff cannot say any of those.

## How far you are allowed to go

Read this before the workflow. The steps below assume three things are present, and **which of them this session actually has decides how much of the workflow exists for you**:

| What is missing | How far this skill goes |
| --- | --- |
| `CHROMATIC_TOKEN` | Nowhere. Nothing else authenticates this API. Say it is missing, point at `references/token.md`, stop. |
| Bun | Nowhere. Say Bun is not installed, stop. |
| a chrome-devtools driver, meaning the MCP tools or the CLI | Steps 1 and 2. Report those, name the gap, stop. |
| nothing | Steps 1 through 5. |

Finishing the investigation is not the goal; a trustworthy answer is.

One row admits an exception. Bun installed but unreachable from `PATH` is an environment quirk rather than a missing capability, so finding the binary and calling it by absolute path is fine, **provided the answer says you did it**. Step 0 detects the situation and prints `OFF PATH` into the `env:` line for exactly that reason. Bun genuinely absent from the machine is the row above.

### Do not route around a missing piece

Two workarounds sit within reach, and neither is the fallback:

- **Playwright, Puppeteer, or headless Chrome launched from a shell** — step 4's checks are written against the chrome-devtools tool surface, and an improvised driver drops them without saying so.
- **Chromatic's own comparison images** — the one substitution that changes what the answer rests on, so it is refused whatever else is present. A run that stopped reports what it has instead of filling the gap with pixels.

The reason is not procedural. A substitute usually does produce an answer, and that is the problem: it reads exactly like one that took the checked path, so the reader has no way to weigh it. A sentence naming the gap is worth more, because they can close it.

### Stopping early is a deliverable, not a blank

A run that ends after step 2 still hands over most of what someone wants from a Chromatic link. Report, in this order:

1. Step 0's `env:` line.
2. The build — number, branch, commit, status.
3. Every story with unreviewed changes, with its viewport width and mode.
4. Both Storybook URLs, so the user can open them.
5. One sentence naming what you could not do and what would unblock it.

Then remove the `--out` directory step 1 wrote into, the same as a finished run would, and stop. Do not open a browser some other way to fill in the rest.

## Step 0 — Preflight

```sh
bun scripts/preflight.ts --devtools-mcp <yes|no>
```

The flag is your own answer to the one question the script cannot check for itself: does *this session* have tools named `mcp__chrome-devtools__*`? Read it off your tool list, not off the machine, and answer about those tools alone. Holding the `chrome-devtools` CLI instead is not a `yes`, and it does not need to be: the script looks for the CLI on `PATH` itself, and since the CLI drives the same server and exposes the same tools, either one carries steps 3 to 5.

Only one of the two is detectable, and the asymmetry is the point. The CLI is invoked through a shell, so finding it on `PATH` *is* the capability. The `chrome-devtools-mcp` binary sitting on the same `PATH` says nothing about whether this session received the tools, which is why the flag exists at all.

It prints one line naming the runtime, the token's remaining life, and which drivers are available:

```
env: bun 1.4.0 · token ok (29d left) · driver: chrome-devtools MCP tools
```

Keep that line. When the script exits non-zero it prints a `STOP` line naming where the workflow ends for this run, and that line governs the rest of the session.

If the command does not run at all, `bun` is not on `PATH`; see the exception under the table above before doing anything else.

## Step 1 — Resolve the URL

```sh
bun scripts/resolve.ts <chromatic-url> --out <workdir>
```

Accepts either URL form the Chromatic UI produces — `/build?appId=…&number=…` or `/test?appId=…&id=…`. It reports the build, its branch and commit, every test with unreviewed changes, and the published Storybook host for both sides. `context.json` in `--out` carries the same data with the story ids, viewport widths, and modes you need later.

If the build is out of reach, the script says why and what to do about it; see `references/api.md` for the `lastBuild` limitation behind it.

## Step 2 — Decide what you are comparing

By default the other side is `Test.baseline` — the exact build Chromatic compared against. **Check its branch before drawing conclusions.** Chromatic rebaselines within a branch, so the baseline is frequently an earlier build on the same branch, and "what changed versus this branch's previous commit" is a different question from "what changed versus `dev`".

When the user means the latter, name what to compare against. `--against` takes a branch name, a build number, a build id, or another Chromatic URL — the branch form covers the usual case, since nobody has the other build's number to hand:

```sh
bun scripts/resolve.ts <chromatic-url> --against dev --out <workdir>
```

## Step 3 — Put both Storybooks on localhost

> Steps 3 to 5 need a chrome-devtools driver. If step 0 found neither the MCP tools nor the CLI, this run ended at step 2 and none of what follows applies to it.

```sh
bun scripts/proxy.ts <base-storybook-host> <head-storybook-host>
```

Published Storybooks reject anonymous navigation. The proxy adds the token upstream and prints one `http://127.0.0.1:<port>` per host. Run it in the background; it serves until killed.

## Step 4 — Render both and compare

Open each side at the story from step 1, through whichever driver step 0 recorded. The tool names below are the same either way; the CLI takes them as subcommands with an explicit `pageId`:

```
http://127.0.0.1:<port>/iframe.html?id=<storyId>&viewMode=story
```

Three things decide whether the comparison means anything:

**Match the capture conditions.** Resize to `parameters.viewport.width` from `context.json`, and when `mode.globals` is not empty, pass it through the iframe's `globals` query parameter. Comparing under conditions Chromatic never captured produces differences that do not exist and hides ones that do.

**Confirm the viewport actually applied, in both tabs.** `resize_page` silently does nothing on a background tab, so select the page with `bringToFront: true` first, then read `window.innerWidth` and `window.innerHeight` inside the page to check. This is worth the extra call: a component sized against viewport units renders at a different height in a tab that never got resized, and the resulting "difference" is entirely your own doing. Height matters as much as width.

**Wait for the story, not the page.** Storybook serves a loading shell and renders client-side, so poll until `#storybook-root` has children before reading anything. Two blank pages compare equal.

Then pull the same data out of both pages with `evaluate_script`. Extract identical fields in an identical order from each side — two dumps of different shapes cannot be diffed. What earns its place depends on the question, but this set covers most of them:

- the story root's `outerHTML`, for structural changes
- per-element computed styles keyed by a stable structural path, plus `getBoundingClientRect`, for style and layout changes
- every custom property resolved on `:root`, which is where design token changes surface
- `documentElement` attributes (theme flags) and loaded stylesheet URLs, to confirm both sides really are in the state you think

`evaluate_script` takes a `filePath`, so write each dump straight to disk and diff the files rather than carrying them through the conversation. Under the MCP tools that path has to sit inside a workspace root, since a scratch directory elsewhere on the machine is refused, so write into the project under `.chromatic-diff/` and delete what you wrote when you are done. The CLI accepts any path it can write to, but use the same directory anyway so both drivers leave the same trail.

Give each run its own subdirectory — `.chromatic-diff/<something unique to this run>/` — and stamp each dump with which side it came from. Two investigations running at once otherwise write over each other's files, and the damage is quiet: the dumps still parse, so you end up diffing one session's base against another's head and reporting differences that belong to neither comparison. Remove your own subdirectory when you finish, and leave anyone else's alone — these dumps run to tens of megabytes.

From daangn/seed-design, Apache-2.0.

More from daangn/seed-design