browser-relay

Files (3)

• SKILL.md
• legacy-api.md · references/
• runtime.md · references/

SKILL.md (first lines)

# Browser Relay

Use the user's existing browser and login state. Select an actual tab ID by URL
and title; a foreground change must not redirect your work.

```bash
browser-relay tabs
browser-relay read --tab <id> --session <task-name>
```

Use a distinct session name for each independent task. Managed operations claim
the tab for that session. Another owner must release or hand it off before you
operate it; never stop someone else's session merely to gain access. These
leases coordinate trusted clients, not access control against local software.

## Choose the observation that answers the question

- **Reading:** `read` focuses on main content and preserves semantic groups,
  complete text and full URLs. `read --ref <ref>` reads an observed subtree.
- **Operating:** `observe` includes the whole page, controls, states and refs.
  Use it to find navigation, dialogs or controls outside the main content.
- **Seeing:** use screenshots for canvas, layout, missing accessible text or a
  semantic action with an unexpected outcome. In the runtime,
  `tab.observe({mode:'both'})` returns a snapshot and screenshot together. With
  CLI use `observe --mode both --output /tmp/page.png --tab <id> --session <name>`.

Follow `nextCursor` (`read --cursor <cursor>`) when an observation is truncated.
A cursor continues the same captured text; it is not a new live observation.
After navigation it expires. If the page exceeds the cache limit, read a smaller
subtree. Do not report a partial result as the complete page or article.
Runtime output has a separate budget: if runtimeOutput.nextCursor is returned,
run its readWith expression in the same session to recover the clipped output.
Then follow any browser nextCursor it contains; a new diff cannot recover it.
A site's “Show more” is separate from output pagination: expand the site control
when the task needs the full article. Group parents, quoted material and replies
remain distinct; do not count recommendations as comments or duplicate cards.

Read the returned title, URL, readiness and warnings. A loading shell, login page,
verification page or unchanged feed is a state to handle, not completed reading.
When scrolling returns `contentChanged:false`, do not count it as new content.
That comparison detects changed text, not a guarantee of new records; use links
or another content identity to deduplicate a feed.

## Observe, act, verify

Use fresh refs, URLs or unique role/name targets from observed content. Refs
expire on navigation or removal. Use `within`/parent context and scoped locators
for repeated controls; use frameId for an iframe. Never guess refs or tab IDs.

Group actions whose targets and sequence are already known. For example, fill a
known field, select an option, submit, and wait for an expected result. Stop the
group before an unknown page, unexpected dialog or decision needing new evidence.
Set per-action timeoutMs when a target is expected to appear, become enabled,
stop moving or become uncovered. Readiness checks never replay dispatched input.

```bash
browser-relay actions --tab <id> --session <task-name> --stdin <<'JSON'
[
  {"type":"fill","target":{"role":"textbox","name":"Search"},"text":"invoice"},
  {"type":"click","target":{"role":"button","name":"Search records"}},
  {"type":"wait","target":{"role":"heading","name":"Results"},"timeoutMs":10000}
]
JSON
```

Actions return an updated observation; use it rather than immediately reading
again. `--diff --session <task-name>` reduces unchanged state. A truncated initial
observation does not advance the diff baseline past unread content; completing
its continuation pages advances that baseline. If nothing
changed, identify what you are waiting for before requesting the same state again.
Navigation waits for document readiness by default, not for all site data. Add a
wait for the observed result/control when the site loads content asynchronously.

Confirm success with the relevant page content, URL or business result. A control
value or successful tool return alone does not prove a click/change handler ran.
Failures retain completed results and identify an interrupted action when it may
have partly executed. Inspect them before recovery; never replay a group blindly.

## Foreground, ownership and interruptions

Keep the user's current tab and window in front. Read, navigate, fill, click
semantic targets and scroll in the background by default. Do not call `focus`
or set `allowFocus:true` just to inspect a page, scroll, or recover from an error.
Use them only when the user explicitly requests foreground operation or a visual
demonstration. A browser task by itself does not authorize stealing focus.
Background scrolling uses DOM scrolling and reports strategy:'dom'. A site may
defer rendering new content while hidden; inspect the returned state and report
that limitation instead of automatically bringing it forward.
Background semantic clicks may use DOM activation, reported as strategy:'dom';
that does not imply a trusted mouse gesture.

Use `new-tab <url> --session <task-name>` for a separate task tab. Close only
unneeded task-created tabs or tabs the user authorized you to close. Release
preserves the page; handoff transfers its ownership and created-tab provenance.

```bash
browser-relay release --tab <id> --session <task-name>
browser-relay handoff --tab <id> --session <owner> --to <receiver>
browser-relay session stop --session <task-name>
```

Long-lived SDK/MCP runtimes send heartbeats. Standalone CLI commands renew the
lease on each operation; `session heartbeat --session <name>` keeps it alive
between operations. Idle leases expire after two minutes. A stopped/expired
session cannot restart implicitly: use a new session ID for explicitly resumed
work. Stopping cancels pending work and releases claims, not completed actions
or user tabs. The extension popup shows owners and lets the user stop sessions.

Use `actions --async` for slow jobs and retain the returned task ID. `task <id>
--session <name> --cancel` requests cancellation. Do not mix legacy/CDP commands
with a claimed tab; release first when deliberately changing clients. User
cancellation is a stop instruction, not a reason to retry automatically.
Request cancellation/timeouts retain task IDs and available partial progress.
Inspect that task before recovery; cancellationError means delivery was not confirmed.

## Runtime and visual work

For repetition, conditional work or multiple tabs, use MCP `browser_exec` or the
CLI `exec --file workflow.js`. `repl` retains JS bindings across NDJSON input lines;

From reliefeai/browser-relay, MIT.