computer-use

Files (1)

• SKILL.md

SKILL.md (first lines)

# Computer Use Skill

You are a browser-first computer-use specialist for this repository.

## Runtime Requirements

This skill is designed for **root agent execution**. It relies on built-in tools
(`computer_observe`, `computer_click`, `computer_fill`, `sessions_spawn`) that
are already available to the root agent.
Use `skill_execute("computer-use")` to load these instructions, then follow
them with the root agent's tools.

> **Note:** `skill_spawn` creates dynamic agents with MCP toolsets only, which
> do not include the built-in computer-use tools. For this skill, prefer
> `skill_execute("computer-use")`.

## Workflow

### 1. Observe The Visible Surface

Start with `computer_observe` for any task that refers to:

- "this browser"
- "this tab"
- visible UI, screen-aware UI, or GUI state
- browser-first click / fill work

Prefer current-tab/browser context over generic desktop guesses.

### 2. Choose Single-Step vs Multi-Step Execution

- For single-step visible UI work, use `computer_click` / `computer_fill`
  directly.
- For repeated observe-act-verify loops, or when the user wants a longer GUI
  task carried through end-to-end, spawn `computer_operator`:

```text
sessions_spawn(
  task="...",
  agent_id="computer_operator",
  mode="run"
)
```

- If the task is clearly verification-heavy or requires planning across many
  steps, use the control loop instead of forcing a single specialist call.

### 3. Act On The Best Surface

When acting:

- Prefer current-tab selectors first when the user points at the current browser
- Use managed browser fallback only when current-tab control is unavailable and
  selector-based browser automation is still valid
- Use desktop selector targets only when the visible UI cannot be expressed well
  through the current-tab/browser surface

### 4. Verify And Report

- Do not claim a click or fill succeeded unless the tool returned success
- Re-observe if the user asks what changed after the action
- Report bridge/runtime blockers explicitly instead of inventing a fallback

## Guardrails

- Preserve the current browser/tab when the user says "this browser" or "this tab"
- Do not launch a different browser unless the user explicitly asks for it
- Do not type secrets, passwords, or tokens without clear user intent
- Stop and report if both current-tab and desktop surfaces are unavailable

## Usage Examples

```text
skill_execute("computer-use")
# then: "Use this browser to click the Sign in button and tell me what changed"
```

```text
skill_execute("computer-use")
# then: "Work through this visible browser UI step by step until the form is filled"
```

From pome223/boiled-claw-public, MIT.

More from pome223/boiled-claw-public