Clawdcursor

Navigate the agent-owned browser to a URL (waits for load). Requires browser_connect first.

Read the current page as structured DOM: interactive elements (links/buttons/inputs with selectors), or text for a CSS selector. Use instead of read_text on web pages. Requires browser_connect first.

Type text into a page input by CSS selector or associated label (DOM input — no coordinates). Requires browser_connect first.

Build a properly-encoded URI from a scheme + path + query JSON. Returns the URI text; pair with open_uri to dispatch. Examples: scheme="mailto" path="bob@example.com" query={"subject":"hi","body":"hello"} → "mailto:bob@example.com?subject=hi&body=hello".

Escalate from blind mode to vision — the a11y snapshot doesn't contain what you need. Only available in blind mode.

Click a DOM element by CSS selector or by visible text content.

Description is built at runtime.

Execute JavaScript in the browser page context. Returns the result.

List all open browser tabs with their URLs and titles.

Get a structured list of interactive elements on the current browser page (inputs, buttons, links with selectors and positions).

Read text content from a DOM element. Useful for extracting information from web pages.

Scroll the browser page via DOM (window.scrollBy). Works regardless of mouse position — use for reliable page scrolling.

Select an option in a <select> dropdown by value or visible text.

Switch CDP connection to a different browser tab by URL or title substring.

Type text into a DOM input field by CSS selector or by associated label text.

Wait for a DOM element matching a CSS selector to appear and become visible.

Click at (x,y). The default coordinate space follows context (image-space while a screenshot is in your context, else screen-space) — pass `space` explicitly when mixing sources: space:"screen" for a11y/@x,y map coords, space:"image" for coords read off the screenshot. Prefer invoke_element when the target has an a11y name.

Polite close request (WM_CLOSE / AXCloseAction / _NET_CLOSE_WINDOW). App may prompt.

Compile the current screen into one fused UI map (a11y + OCR + lazy vision) of elements with stable ids, roles, confidence and sources. Returns a ranked element list with a snapshot id; act on a specific element via invoke_element/set_field_value with {element_id, snapshot_id}. a11y-first; pulls OCR only when a11y is sparse or target_text is missing; pass max_cost:'cheap' to forbid OCR, or 'vision_ok' to allow screenshots.

Description is built at runtime.

Read ALL text of the focused window / web page EXACTLY. Reads the document through accessibility first (no keystrokes, nothing selected); where that is unavailable, select-all + copy, then the clipboard is restored. Exact characters (IDs, amounts, codes) — prefer it over OCR for reading pages and documents. Refuses in terminals (Ctrl+C would interrupt them). Never presses select-all while focus is in an editable field, an editor document or a list — it returns the accessibility text then. After a select-all copy the content stays selected (the result says so); focus the right window first.

Read the current mouse cursor position in image-space coordinates (the same space the mouse_* tools accept — a cursor_position read round-trips with mouse_hover). Computer-use parity: pointer-state read, no side effects.

Hand a whole task to clawdcursor's built-in thin agent loop, which drives the toolbox with the model configured via `clawdcursor doctor` (perceive → act → iterate until done). For an expensive frontier model this is the delegation lever: hand grunt work to a cheaper configured model that takes the wheel. BOUNDED-SYNC: waits up to `timeout` seconds (default 45) for completion. A finished task returns its result directly; a longer task returns a {status:"running"} receipt with progress while the loop CONTINUES in the background — re-call with the SAME task text to keep waiting (re-attaches, neve…

LAST RESORT — take a screenshot only when the accessibility tree and OCR are both insufficient (custom canvas, icon-only UI, pixel-level verification). Prefer read_screen first, then ocr_read_screen; escalate to screenshot only when those fail. Returns the image downscaled to fit 1280px on its long edge (the reply states its size and scale).

Take a zoomed screenshot of a specific screen region for detailed inspection. Coordinates are in image-space (from desktop_screenshot).

Enumerate running Electron / WebView2 / Chromium-shell apps (e.g. New Outlook, Teams, Discord, Slack, VS Code) whose accessibility trees are typically sparse because the UI is rendered inside a web view. Returns each candidate with a CDP-discovery hint — if a remote-debugging port is already live, you can attach to it via `browser({"action":"connect"})`; otherwise the hint shows the relaunch command the user should run to enable CDP. Use this WHEN `read_screen` returns an unexpectedly empty tree on a "native" app.

Declare the task complete. Provide SPECIFIC screen evidence — a window title, a value visible in the document, a status bar message. Do NOT use hedging words ("should", "might", "probably", "I think", "I believe") — that means you are guessing. If the task CHANGED anything you MUST pass `assertions` (same types as the verify tool, plus `file_changed_since_start` for a file you wrote) that prove the RESULT — and the proof must reflect your change, not state that was already there (an ambient clock, an already-open window). The harness re-checks them against the live screen and rejects done if a…

Drag the mouse from (startX,startY) to (endX,endY) — select text, draw, resize. To TRACE A CURVE/PATH (gesture, curved track, drawing), pass `path` = an array of 12–20 {x,y} points instead: press at the first point, move through each, release at the last. The default coordinate space follows context; if you read coords off the SCREENSHOT, pass space:"image" so the tool scales them.

Add a task string to the favorites list. No-op if already present. Returns the updated list.

Return the list of starred ("favorite") task strings. Stored locally at ~/.clawdcursor/.clawdcursor-favorites.json — never sent over the network.

Remove a task string from the favorites list. Returns the updated list.

Semantically locate the best clickable element for an intent (e.g. "submit", "cancel", "search") over the compiled UI. Returns JSON {status:"ok"|"ambiguous"|"none", snapshot_id, best?, candidates}. On "ok", act with invoke_element({element_id: best.element_id, snapshot_id}). Deterministic synonym + text + confidence match.

Search for UI elements by name, control type, or automation ID within a process. Returns matching elements with bounds. For browser windows with CDP attached (the focused one, or `processId` of the CDP browser), falls back to a DOM query when UIA returns empty so canvas / SPA content is reachable too (results are flagged with a "via CDP DOM" header and use viewport-relative coords).

Semantically locate the best editable field for a purpose (e.g. "recipient", "subject", "body", "search") over the compiled UI, including label-less fields via their adjacent label. Returns JSON {status, snapshot_id, best?, candidates}. On "ok", fill with set_field_value({element_id: best.element_id, snapshot_id, value}). Deterministic.

Keyboard-focus an element by a11y name. Does NOT raise window — use focus_window first if needed.

Bring a window to the foreground. Match by processName, pid, or title substring.

Get the currently focused/foreground window.

Get state flags of a named element (focused/enabled/disabled/selected/busy/offscreen/expandable/expanded, and checked for checkboxes/toggles).

Get the currently focused UI element (keyboard focus). Returns name, control type, value, bounds, and process ID.

Get the screen dimensions and scale factor.

Return the canonical agent system prompt. Use this to mirror clawdcursor's behavioral rules in your own agent loop — coordinate syntax, key combo syntax, web-service policy (never type "browser" into search bars), protocol escape hatches (mailto:/https:/file:/...), stagnation recovery, termination rules (done/give_up/cannot_read), untrusted-screen-content discipline. Reflects the prompt at the current installed clawdcursor version — pull this rather than hardcoding rules so they stay in sync with upstream changes.

Return current system time (ISO, epoch, timezone). Zero I/O.

List all visible windows with their title, process name, PID, and bounds.

Abandon the task when it's impossible from here (stuck after retries). When a human is at the machine this pauses for them first; for sign-in, 2FA or CAPTCHA prefer ask_user.

Click/activate a UI element by its accessibility name. MORE RELIABLE than coord clicks — use this when the snapshot shows a named target.

Press a key or key combo. Use "mod" for Ctrl/Cmd. Use "+" for a chord (e.g. "mod+s", "shift+Tab"). Space-separate for a sequence ("Down Down End"). Examples: "Return", "Tab", "Escape", "F5", "ctrl+a".

Press a key without releasing. Pair with key_up. Use to hold modifiers (shift, ctrl) during clicks.

Press a keyboard key or key combination. Use "+" for a chord (e.g. "ctrl+s", "shift+enter", "alt+tab"). Separate multiple presses with SPACES to send them in sequence (e.g. "Down Down End", "ctrl+a Delete"). Single keys: "Return", "Tab", "Escape", "Backspace", "Delete", "F1"-"F12", "Left/Right/Up/Down".

Release a key previously pressed with key_down.

Enumerate connected displays with logical bounds + DPI ratio. Use before display-specific screenshots.

List visible top-level windows with title, process, and bounds. Useful when the active window is wrong or missing.

Return the last 200 captured console log entries from the daemon (level + timestamp + message). Empty array when no log buffer is attached (e.g. stdio MCP without a running daemon).

Maximize the foreground window (or a matched window). Polite request; WM may interpret.

Minimize the foreground or matched window to the taskbar / Dock.

Minimize a window (hide to taskbar/Dock). Counterpart to the existing `minimize_window` in a11y.ts but routed through the OS-agnostic PlatformAdapter and accepts explicit target queries.

Click the left mouse button at the given image-space coordinates.

Double-click the left mouse button at the given image-space coordinates.

Press a mouse button without releasing. Pair with mouse_up. Enables hold-and-drag + modifier clicks.

Drag from one image-space coordinate to another (click-hold-move-release). Useful for selecting text, moving objects, or resizing.

Drag the mouse along a multi-point path in image-space. Path is an array of {x,y} points. Useful for Paint-style drawing or gesture input. Press occurs at the first point and release at the last.

Move the mouse to the given image-space coordinates without clicking. Useful for revealing tooltips or hover menus.

Middle-click (wheel-click) at image-space (x, y). Opens links in new tab in most browsers; pans in some apps.

Move cursor by a relative offset (dx, dy). Wayland-safe via cursor cache.

Right-click at the given image-space coordinates (opens context menu).

Scroll the mouse wheel at the given image-space coordinates.

Scroll horizontally at image-space (x, y). On Windows uses Shift+wheel synthesis; on Linux uses xdotool wheel buttons when available; on macOS uses Shift+wheel.

Triple-click the left mouse button at image-space (x, y) — selects a paragraph in most text editors. In single-line edit fields (dialog filename boxes etc.) a select-all follows automatically when the triple-click left nothing selected, so subsequent typing reliably REPLACES the existing text.

Release a mouse button previously pressed with mouse_down.

Move the cursor to (x,y) WITHOUT clicking — hover/dwell over a target (pair with wait(ms) for a required dwell time). The default coordinate space follows context; pass space:"image" for coords read off the screenshot.

Open a URL in the agent's CDP browser for DOM interaction — attaches to the browser already on the debug port or launches a dedicated agent-owned instance, then navigates. cdp_page_context / cdp_click / cdp_type work on that page directly. Tier 2 (mutation): triggers network egress to an arbitrary destination + spawns/attaches to a browser process.

Step 2 of cheap-first perception: use when the a11y tree (read_screen) is empty or too sparse to identify your target. OS-level OCR returns text elements with pixel coordinates — no image bytes, no vision model. Much cheaper than a screenshot. Coordinates are screen coordinates — click them with space:"screen". scope:"window" reads only the focused window — faster, and it does not read the text of other apps.

Open an application by name (e.g. "Notepad", "TextEdit", "Safari").

Open a file or folder in the OS default app (explorer / open / xdg-open).

Open ANY registered URI scheme via the OS protocol-handler registry. ONE tool replaces dozens of app-specific shortcuts. Examples: mailto:bob@example.com?subject=hi&body=hello (mail), tel:+15551234 (phone), slack://channel?team=T123&id=C456 (Slack), vscode://file/path (VS Code), webcal://server/cal.ics (calendar), spotify:track:ID (Spotify), https://example.com (browser). Must be properly URL-encoded — pair with build_uri when you have semantic fields.

Open a URL in the default browser. Use instead of navigate_browser when you don't care which browser.

Hand the desktop to the user for a step only they can do (sign in, a password or 2FA code, a CAPTCHA, a choice only they can make, a plan limit or upgrade wall — never click Upgrade / Subscribe yourself) instead of stopping or asking them to come back to this chat. Ask for that ONE step only — e.g. "Sign in to Glama" — and do everything before and after it yourself (navigating, filling forms, clicking through): keep the user on screen as briefly as possible. For a sign-in, a code or a CAPTCHA, pass `done_when` (what shows once their step is done, e.g. [{"type":"element_exists","name":"Sign out…

Read the OS clipboard.

START HERE — cheapest perception. Read the accessibility tree of the focused window: buttons, inputs, text elements with coordinates. The snapshot is auto-attached each turn; call this again only when you expect the screen changed since the last turn. If the tree is empty, escalate to read_text (OCR) next, then screenshot only as a last resort.

OCR the screen and return visible text + positions. Use when the a11y snapshot is empty/sparse (webview, canvas, PDF, game) to READ on-screen content. Cheaper than a screenshot (no image bytes). May take 1–3s.

Relaunch a WebView2 / Electron app with a remote-debugging port enabled so `browser` tools can attach. The relaunch closes the existing instance (asks the OS to close politely via window_close first), then starts a new one with `--remote-debugging-port=<port>`. The user MUST consent — this WILL lose any unsaved work in the app. Returns instructions + the new port. Prefer the hint from `detect_webview_apps` when possible.

Set the foreground (or matched) window bounds. Units are SCREEN px — the same as list_windows and accessibility coordinates; pass space:"image" for screenshot coordinates. Omitted fields preserved.

Restore a minimized or maximized window to its previous bounds.

Resume after a pause for the user when the user tells YOU they finished (normally they click Resume on screen).

Create a recurring task that runs on a cron schedule. The task submits to the autonomous agent each time the cron tick fires; if the agent is busy at that moment, the tick is skipped (no queue). Cron is standard 5-field syntax (`* * * * *` = every minute) or 6-field with leading seconds. Returns the created task with id + next-run timestamp. Persisted across daemon restarts.

Delete one scheduled task by id. No-op if id is unknown.

Return every scheduled task: id, cron expression, task text, enabled state, run/skip counters, last-run / last-error, next-run timestamp.

Pause/resume a scheduled task without deleting it. When disabled the cron job is unregistered (no firing); state stays persisted.

LAST RESORT — expensive: sends image bytes into LLM context. Escalation order: read_screen (a11y tree, free) → read_text (OCR, cheap) → screenshot (this, expensive). Only call this when both a11y and OCR failed to provide what you need (canvas-only app, icon-only UI, pixel-level verification).

Capture the full primary display. Returns base64 image bytes plus a `format` field (`jpeg` by default, `png` if configured); the MCP `image.mimeType` is set to match. Equivalent to desktop_screenshot but returns the full frame unconditionally (no region cropping). Useful for the dashboard's /screenshot REST replacement.

Scroll at (x,y) in a direction. Omit x,y to scroll at the screen center. If you read x,y off the SCREENSHOT, pass space:"image".

Prepare a report of THIS session for the clawdcursor team when something went wrong: which tools ran, blind vs screenshot use, errors — no typed text, clipboard, screenshots, names or paths. Sends nothing: returns a prefilled GitHub issue link to give the USER, who reviews and submits it. Pass `note` with what went wrong.

Set an editable field's value directly via accessibility (more reliable than click+type for forms).

Execute a keyboard shortcut by describing what you want to do (e.g. "scroll down", "close tab", "upvote"). Uses fuzzy matching against the shortcuts database. Provide context (active app name) for app-specific shortcuts like Reddit or Outlook.

List available keyboard shortcuts. Filter by category (navigation, browser, editing, social, window, file, view, quick) and/or context (e.g. "reddit", "outlook"). Returns shortcut names, descriptions, and key combos for the current platform. Use this BEFORE reaching for mouse_scroll or mouse_click — there is often a faster keyboard shortcut.

OCR-locate visible text on screen and click its center. Use when the a11y tree is empty and invoke_element fails (webview/canvas). Pass the exact visible text (e.g. "Search", a video title, "Sign in").

Read text from the screen with automatic fallback. OCR-first pipeline: runs OCR (primary) and a11y tree (supplement) in parallel. Returns structured text without needing a screenshot. Use this as your primary perception tool.

Type text into a UI element. With a target, finds and focuses the element, then REPLACES its whole content (select-all, then paste) — pass mode:"append" to keep the content and type at the cursor. Without a target, types at the cursor of the focused element. Uses clipboard paste for reliability (no dropped characters).

Submit a redacted task-log report to clawdcursor's telemetry endpoint. Opt-in only — never runs unless explicitly invoked. Returns the server-issued report ID and a preview of the redacted payload, or an error reason when submission fails.

Submit a natural-language task to the autonomous agent. The agent classifies, decomposes, and executes the task through the unified pipeline (a11y → text-agent → vision fallback). Returns immediately with `accepted: true`; poll agent_status to track progress. Runs locally on a cheap model — prefer this over driving the mouse_*/screenshot tools yourself to save tokens.

Cycle next/previous browser tab (Ctrl+Tab / Ctrl+Shift+Tab on every OS) or jump to tab N (mod+1..9).

Everything that is not a window or a UI element: exact screen text, the clipboard, shortcuts, handing a step to the user, launching by URI. For elements use `accessibility`; for raw clicks and keys `computer`; for windows `window`. Read: `copy_all_text` (the exact text of the focused page or document — IDs, amounts, codes; read through accessibility with no keystrokes where possible; prefer it to OCR), `ocr` (scope "screen" | "window": text lines with screen positions, for when the accessibility tree is empty). Clipboard: `clipboard_read`; `clipboard_write` (text — overwrites the user's clipbo…

**Requires the `clawdcursor agent` daemon to be running** (binds 127.0.0.1:3847 with an LLM configured). Hand clawdcursor a WHOLE natural-language task and let its internal agent loop execute it (accessibility first, screenshots when the screen has no readable structure). Use this when you don't want to micromanage every primitive — clawdcursor decomposes the task, picks the cheapest execution path, and returns a trace. BOUNDED-SYNC: waits up to `timeout` seconds (default 45) — a longer task returns {status:"running"} with progress while it CONTINUES in the background; re-call with the SAME in…

Return the structured log entries for the currently-executing task (or the most recent if no task is running). Each entry is a JSONL event from the pipeline trace.

List the last 50 task summaries from the daemon's structured log. Each summary contains task text, correlation ID, start/end timestamps, and final status. Returns [] when no logger is attached.

Type text into the currently focused input. Prefer set_field_value when a field has an a11y name.

Type text into the currently focused element. Internally uses clipboard paste for reliability (no dropped chars). The user clipboard is saved before and restored after, so calling type_text never clobbers any text the caller had previously placed on the clipboard.

Send the OS Undo keystroke (mod+Z).

Deterministically check CURRENT state against machine-checkable assertions — the harness executes them, no guessing. Types: window_title_contains{value}, app_running{name}, element_exists{name}, element_value_contains{name,value}, clipboard_contains{value}, file_exists{path}, file_contains{path,value}, ocr_contains{value}, file_changed_since_start{path} (proves a file was written during THIS task). Cheaper and more reliable than a screenshot — use after a critical step or before done().

Pause for N milliseconds (max 5000). Use after actions that trigger animations or page loads.

Poll the a11y tree until an element matching name/controlType appears. Useful after an action spawns a dialog.

Windows, apps and displays at the OS level: find, open, focus and arrange windows. For what is INSIDE a window use `accessibility`; for a web page's DOM use `browser`. Read-only: `list` (every visible window: title, process, pid, screen bounds), `active` (the focused window), `list_displays` (monitors with bounds and DPI), `screen_size`. Arrange — target with processName, processId or title (substring); none given = the foreground window: `focus` (bring to front, keeps it maximized), `maximize`, `minimize`, `restore`, `resize` (x, y, width, height in screen px — the units `list` and `accessibi…

Write text to the OS clipboard.