browser-automation

Files (1)

• SKILL.md

SKILL.md (first lines)

# Browser Automation

Control a real Chrome browser through MCP tools. The browser launches automatically on first use via a background daemon -- no API keys required for local mode.

## Available Tools

| Tool | Purpose |
|------|---------|
| `browser_navigate` | Go to a URL (auto-starts browser daemon) |
| `browser_snapshot` | Get accessibility tree with element refs |
| `browser_click` | Click an element by ref (e.g., `@0-5`) |
| `browser_type` | Type text (goes to focused element) |
| `browser_fill` | Fill an input field by ref |
| `browser_press` | Press a key (Enter, Tab, Escape, Cmd+A, etc.) |
| `browser_select` | Pick a dropdown option by ref |
| `browser_screenshot` | Capture the current page |
| `browser_scroll` | Scroll at coordinates |
| `browser_hover` | Hover at coordinates |
| `browser_drag` | Drag from one point to another |
| `browser_click_xy` | Click at exact coordinates |
| `browser_evaluate` | Run JavaScript in the page |
| `browser_get` | Get page info (url, title, text, html, value, box) |
| `browser_wait` | Wait for load state, selector, or timeout |
| `browser_back` | Go back in history |
| `browser_forward` | Go forward in history |
| `browser_reload` | Reload current page |
| `browser_pages` | List all open tabs |
| `browser_new_tab` | Open a new tab |
| `browser_switch_tab` | Switch to a tab by index |
| `browser_close_tab` | Close a tab by index |
| `browser_network` | Capture/inspect HTTP requests |
| `browser_close` | Stop the browser daemon |

## Core Workflow

Every browser task follows this loop:

1. **Navigate** to the target URL with `browser_navigate`
2. **Snapshot** the page with `browser_snapshot` to get the accessibility tree with element refs
3. **Act** using `browser_click`, `browser_fill`, `browser_press`, `browser_select` with the **refs** from the snapshot
4. **Verify** with `browser_screenshot` to confirm the action worked
5. **Repeat** steps 2-4 until the task is complete
6. **Close** the browser with `browser_close` when done

## How Element Refs Work

Call `browser_snapshot` -- it returns an accessibility tree where every interactive element has a **ref** like `[0-5]`. Use that ref to target the element.

```
RootWebArea "Example" url="https://example.com"
  [0-0] link "Home"
  [0-1] link "About"
  [0-2] button "Sign In"
  [0-5] textbox "Search"
```

Then act on elements using their ref:

```
browser_click({ ref: "@0-2" })        → clicks "Sign In"
browser_fill({ ref: "@0-5", value: "my query" })  → fills the search box
```

Refs are more reliable than CSS selectors because they're tied to the accessibility tree and work across iframes.

### Ref formats (all equivalent)

- `@0-5` -- @ prefix
- `0-5` -- plain ref
- `ref=0-5` -- explicit prefix

### Snapshot also provides maps

The full snapshot output includes:
- **xpathMap**: Cross-frame XPath selectors
- **cssMap**: CSS selectors when available
- **urlMap**: URLs extracted from links

Use the compact flag (`-c`) for a concise tree without the maps.

## Common Patterns

### Fill out a form

```
1. browser_navigate({ url: "https://example.com/signup" })
2. browser_snapshot()
   → Find refs: [0-3] textbox "Email", [0-4] textbox "Password", [0-7] button "Submit"
3. browser_fill({ ref: "@0-3", value: "user@example.com" })
4. browser_fill({ ref: "@0-4", value: "securepassword" })
5. browser_click({ ref: "@0-7" })
6. browser_screenshot()  → Verify success
```

### Extract data from a page

```
1. browser_navigate({ url: "https://example.com/products" })
2. browser_evaluate({ script: "JSON.stringify(Array.from(document.querySelectorAll('.product')).map(el => ({ name: el.querySelector('h2')?.textContent, price: el.querySelector('.price')?.textContent })))" })
```

### Handle pagination

```
1. browser_navigate({ url: "https://example.com/results" })
2. browser_evaluate({ script: "..." })  → Extract data from page 1
3. browser_snapshot()  → Find the "Next" button ref
4. browser_click({ ref: "@0-12" })
5. browser_wait({ type: "load" })
6. browser_evaluate({ script: "..." })  → Extract data from page 2
```

### Deal with elements not in view

```
1. browser_snapshot()  → Target element not found
2. browser_scroll({ x: 0, y: 0, deltaX: 0, deltaY: 500 })
3. browser_snapshot()  → Now the element appears
4. browser_click({ ref: "@0-8" })

From Vidoxlabs/vitools-public, MIT.

More from Vidoxlabs/vitools-public