e2e-maestro

Files (91)

• SKILL.md
• shared.json · assets/
• template-placeholders.json · assets/
• canvas-tap-flow.yaml · examples/
• evidence-report.md · examples/
• checks.md · references/
• flows.md · references/
• maestro-setup.md · references/
• network-proof.md · references/
• runner-and-devices.md · references/
• screenshot-matrix.md · references/
• check-e2e-report.mjs · scripts/
• check-e2e-setup.mjs · scripts/
• check-flows.mjs · scripts/
• check-lib.mjs · scripts/
• e2e-repo.mjs · scripts/lib/
• flow-rules.mjs · scripts/lib/
• maestro-spawns.mjs · scripts/lib/
• maestro-yaml.mjs · scripts/lib/
• selftest.mjs · scripts/
• 02-complete-task-survives-kill.yaml · templates/.maestro/flows/journeys/
• 01-first-launch.yaml · templates/.maestro/flows/smoke/
• matrix.yaml · templates/.maestro/screenshots/
• airplane-mode.yaml · templates/.maestro/subflows/
• debug-setup.yaml · templates/.maestro/subflows/
• open-link.yaml · templates/.maestro/subflows/
• set-language.yaml · templates/.maestro/subflows/
• shoot-screen.yaml · templates/.maestro/subflows/
• e2e.json · templates/contracts/
• e2e.schema.json · templates/contracts/
• android-device.test.ts · templates/scripts/e2e/
• android-device.ts · templates/scripts/e2e/
• build-guard.test.ts · templates/scripts/e2e/
• build-guard.ts · templates/scripts/e2e/
• capture-screenshots.ts · templates/scripts/e2e/
• e2e-cli.test.ts · templates/scripts/e2e/
• e2e-cli.ts · templates/scripts/e2e/
• e2e-config.test.ts · templates/scripts/e2e/
• e2e-config.ts · templates/scripts/e2e/
• e2e-device.ts · templates/scripts/e2e/
• install-maestro.sh · templates/scripts/e2e/
• ios-device.test.ts · templates/scripts/e2e/
• ios-device.ts · templates/scripts/e2e/
• maestro-args.test.ts · templates/scripts/e2e/
• maestro-args.ts · templates/scripts/e2e/
• maestro-run.test.ts · templates/scripts/e2e/
• maestro-run.ts · templates/scripts/e2e/
• network-sampler.test.ts · templates/scripts/e2e/
• network-sampler.ts · templates/scripts/e2e/
• open-device.ts · templates/scripts/e2e/
• png-diff.test.ts · templates/scripts/e2e/
• png-diff.ts · templates/scripts/e2e/
• run-e2e.ts · templates/scripts/e2e/
• sample-sockets.ts · templates/scripts/e2e/
• sampler-evidence.test.ts · templates/scripts/e2e/
• sampler-evidence.ts · templates/scripts/e2e/
• screenshot-compare.ts · templates/scripts/e2e/
• screenshots-cli.test.ts · templates/scripts/e2e/
• screenshots-cli.ts · templates/scripts/e2e/
• write-gallery.test.ts · templates/scripts/e2e/
• …and 31 more

SKILL.md (first lines)

# End-to-end flows with Maestro

The app's wiring is proven on real Release builds: Maestro flows drive the journeys on a simulator
and an emulator the repo owns, a socket sampler proves the app's network policy while they run, the
accessibility flows pass at the largest text size, and every screen is captured per platform, locale
and theme against reviewed baselines. This skill sets up that tooling, writes the flows, runs them,
and its scripts prove the setup, every flow (statically) and each run's reports. Verified on
2026-10-14 with Maestro 2.10.0, Expo SDK 57 (expo 57.0.26), React Native 0.86.3 and Xcode 26.6 with
the iOS 26.5 runtime; the Maestro facts were read in the 2.10.0 jar (re-verify:
references/maestro-setup.md, "Changing the pin"). The templates pass tsc, ESLint and Jest in the
reference app, and on 2026-10-14 the runner, the capture and the report checker ran end to end on a
Release preview build of the reference app: an iOS 26.5 simulator and an Android API 35 emulator,
phone and tablet each. The Android facts were observed on API 35; the default API 36 image is not yet
run end to end. The sampler liveness proof, the per-platform capture folders and the two-link matrix
setup came after that run and are proven by the self-test and Jest only.

## Rules that must hold

1. **Maestro comes only from the pinned installer and runs quietly.** `scripts/e2e/install-maestro.sh`
   downloads 2.10.0, checks its SHA-256, and unzips into the git-ignored `tools/maestro/`; Maestro runs
   with Java 17 and `MAESTRO_CLI_NO_ANALYTICS`, `MAESTRO_CLI_ANALYSIS_NOTIFICATION_DISABLED` and
   `MAESTRO_DISABLE_UPDATE_CHECK` set to `true`. Why: Homebrew and `curl | bash` are unpinned and
   unverified, and by default Maestro sends analytics and checks for updates.
   (`maestro-pin`, `maestro-install-source`, `maestro-env`)
2. **Every Maestro run names its device and a driver port of its own, before the command.**
   `maestro --device <udid or serial> --driver-host-port <free port> test ...`, built by
   `maestroGlobalArgs`; never the per-command `--udid` alone, never port 7001. `simctl` names a UDID,
   `adb` gets `-s <serial>`, and the devices are the repo's own `<prefix>-<purpose>`. Why: on a shared
   machine the default driver port answered from another session's simulator.
   (`maestro-device`, `shared-device`, `run-command-device`)
3. **Flows run only through the runner, on a Release build of a test variant with ads off.**
   `npm run e2e:ios` and `npm run e2e:android` refuse a Debug build (no embedded bundle: Metro's
   WebSocket hides the app's traffic), the production variant (no debug link) and a build whose
   `expo.extra` misses `contracts/e2e.json` `buildExtra` (`adsMode: off` for an app with ads: an
   ads-on build shows the consent form over every cleared launch and its ad SDK opens sockets).
   Evidence is the full run: no tag filter, the large-text step included. (`runner-build-guard`,
   `e2e-ads-off`, `runner-steps`, `run-record`, `run-filtered`)
4. **Select elements by `id:` only, with ids from the contracts.** `contracts/screens.json`
   (screen-contracts) plus `contracts/e2e.json` `extraTestIDs`; text only for OS-owned UI under a
   `# system-ui: <why>` comment; never a crop-only part (inside an accessible element, or hidden from
   screen readers). Why: the flow then runs in every locale on both platforms, and an unknown id fails
   in a second instead of after a 15 s timeout. (`text-selector`, `testid-format`, `unknown-testid`,
   `unreachable-testid`, `no-testid-step`)
5. **State comes from a cleared launch and one debug link per setup.** Each flow starts with
   `launchApp: { clearState: true }`; state is set only through `subflows/debug-setup.yaml`
   (debug-menu-and-test-hooks' `<scheme>://debug/setup?...`), whose query ends on the screen it opens
   (`open=`) and waits for that screen's testID. A language that can flip the layout direction goes
   alone, first, through `subflows/set-language.yaml` (the launch link; it waits out the restart). No
   `openLink` in flows, no tapping through menus; the one tap is the control under test. Why: setting
   up through the UI is slow and flaky, waiting for the start screen passes with a lost link, and a
   direction restart drops the rest of a link. (`first-step-launch`, `raw-open-link`, `debug-query`,
   `debug-open`, `debug-lang-alone`)
6. **iOS's "Open in …?" alert is accepted by its title, and tools never use `simctl openurl`.** Guard
   on `'Open in .*\?'` and tap `Open` below it with `optional: true`. Why: Maestro's text match ignores
   case, so a bare `'Open'` also matches any in-app OPEN button, and `openurl` leaves the alert up so
   the app never gets the link. Android-only steps (`back`, `pressKey: back`, airplane mode) run
   under `runFlow: { when: { platform: Android } }`: iOS skips them silently. (`open-alert-guard`,
   `openurl-prompt`, `platform-guard`)
7. **A flow never sets `APP_ID`, `APP_SCHEME`, `LANG` or `THEME` in `env:`.** Why: in Maestro 2.10
   a flow's `env:` beats `-e`, so every run would silently use the flow's value. (`env-shadows-cli`)
8. **Flows live in `.maestro/flows/<area>/<nn>-<name>.yaml`.** Shared steps in `subflows/`, the matrix
   in `screenshots/`, hand-run flows in declared manual folders. Why: Maestro runs only the top level
   of the folder it is given, so a nested flow never runs and a sub-flow under `flows/` runs alone and
   fails. (`flow-location`, `flow-name`, `subflow-in-flows`, `runflow-missing`, `flow-header`,
   `flow-tags`, `flow-parse`)
9. **No AI commands and nothing that uploads.** `assertWithAI`, `assertNoDefectsWithAI`,
   `extractTextWithAI`, `maestro cloud`, `maestro login`, `test --analyze`, `record` without
   `--local`. Why: they send screenshots or the app to a third party. (`ai-command`, `maestro-upload`)
10. **A flaky flow is a bug.** Wait on ids (`extendedWaitUntil`), never sleep, no `retry:` around app
    steps; a non-smoke flow may be quarantined for 7 days with `# quarantine YYYY-MM-DD: <reason>`, a
    smoke flow never. Why: retries and sleeps hide the races users hit. (`retry-app-assertion`,
    `quarantine-note`, `smoke-quarantine`)
11. **The run proves the network policy at runtime, on its own devices, with live samplers.** One
    sampler per device of the run lists the app's sockets (`lsof` on iOS, `/proc/net` by uid on
    Android); every socket `contracts/e2e.json` `network` does not allow (`offline-only`: any other
    host; `allowlist`: hosts outside `allowHosts`) fails the run, and so does a sampler that did not
    sample until the end or never found the app (`network-samplers.json`). Why: iOS simulators have no
    airplane mode, a JS-only check misses native SDKs, and a dead sampler leaves the same empty report
    as a quiet app. (`e2e-contract`, `runner-network`, `sampler-own-devices`, `sampler-liveness`,
    `network-sampler`, `network-sockets`)
12. **The accessibility flows pass at the largest text size.** `flows/a11y/` runs on the phone and
    the tablet in every `largeText.locales` entry (a left-to-right and, when the app has one, a
    right-to-left locale). Why: clipping and overlap at the largest system size (iOS AX5, about 3.6x in
    React Native; Android font scale 2.0) only show on a device, and each a11y flow sets the locale it
    is given through `set-language.yaml`. (`runner-steps`, `large-text-lang`, `large-text-missing`,
    `large-text-failed`)
13. **Screenshots come only from a still, deterministic screen, with a fixed tolerance.** Status bar
    9:41, animations held (`motion=still`), a fixed day, one device model per baseline set, baselines
    per platform; 0.2 % of pixels at a 0.1 colour distance, never raised. Each capture empties
    `reports/screenshots/<platform>/` first and records a combination that failed instead of stopping.
    Why: an animating screen differed by about 1 % between two captures a second apart, and a shared or
    stale summary passed for another platform or an earlier run. (`deterministic-capture`,
    `screenshot-tolerance`, `capture-evidence`, `matrix-incomplete`, `screenshot-capture-failed`,
    `screenshot-platform`)
14. **Baselines change only on purpose.** `--update` narrowed to the sets a change touched, every new
    or changed PNG and its diff opened, one commit with a `Gate-Change:` trailer. Why: an unread
    baseline freezes a bug for every later run. (`baseline-matrix`, `screenshot-changed`)
15. **Hand-run flows live in declared manual folders the runner never lists.** Flows that need
    production SDK behaviour or a real system prompt sit in `contracts/e2e.json` `manualFolders`; when
    `launchApp` would pre-grant the permission under test they start outside Maestro and never call it.
    Why: `launchApp` grants every permission by default, so the prompt would never show.
    (`manual-folder`, `runner-steps`)

## Workflow

1. **Prerequisites.** Screen testIDs in `contracts/screens.json` (screen-contracts), the debug link and
   `contracts/debug-links.json` (debug-menu-and-test-hooks), Release builds of the preview variant
   (ios-simulator-build, android-emulator-build, or eas-build-and-submit with `--local`), and Java 17.
   If neither `/usr/libexec/java_home -v 17` nor Android Studio's JDK exists, stop and ask the developer.
2. **Install the templates** (line 1 names each repo path): `templates/scripts/e2e/` (with every test),
   `templates/.maestro/` and `templates/contracts/e2e.json` with its schema; fill `productionAppIds`,
   `network`, `largeText` and `screenshots`, and the flows' testIDs (`assets/template-placeholders.json`
   lists what each placeholder stands for). Add the four npm scripts and `/tools/maestro/` to
   `.gitignore`; declare `screenshots` and each manual folder in `contracts/naming.json`
   `extraE2eFolders`. Read [references/maestro-setup.md](references/maestro-setup.md) and
   [references/runner-and-devices.md](references/runner-and-devices.md) the first time. Run
   `npx jest scripts/e2e --ci --maxWorkers=2`, `npm run typecheck` and `npx eslint scripts/e2e --max-warnings 0`.
3. **Check the setup:** `node ${CLAUDE_SKILL_DIR}/scripts/check-e2e-setup.mjs .`; fix every `FAIL`
   (file, rule, fix) and rerun.

From zakariaf/ReactNative-Skills, MIT.

More from zakariaf/ReactNative-Skills