android-remote-control

Files (1)

• SKILL.md

SKILL.md (first lines)

# Android Remote Control

Control a physical Android device via the Android Remote Control MCP server.
The device runs an accessibility-service-based MCP app — it does NOT require
ADB or root. All interaction happens through MCP tool calls that map to
accessibility actions, gestures, intents, and file/storage APIs.

## Orientation

Run the following before proceeding to confirm the MCP server is connected and
discover available storage and apps:

```
android_list_storage_locations
android_list_apps filter=user
android_get_screen_state
```

## How to read a page (screen state)

Call `android_get_screen_state`. This returns:

- `screen: <w>x<h> density:<n> orientation:<landscape|portrait>`
- One block per window, each with `pkg`, `title`, `activity`, `focused`
- A flat node table: `node_id | class | text | desc | res_id | bounds | flags`
- A `hierarchy:` section showing node nesting via indentation

Flags: `on`=onscreen, `off`=offscreen, `clk`=clickable, `lclk`=longClickable,
`foc`=focusable, `scr`=scrollable, `edt`=editable, `ena`=enabled.

### Pagination

Large screens split into pages of 200 nodes. The response includes a
`page:N/total` line and a `cursor`. Call again with that `cursor` to fetch the
next page. Stop as soon as the needed node is found — do not fetch every page.

### Screenshots

`include_screenshot=true` can ONLY be requested on page 1 (no cursor). The
screenshot is a low-resolution image attachment. Use it when the node tree is
insufficient (custom views, Canvas, game UI, Samsung One UI preferences).

### Find a specific element

```
android_find_nodes by="text" value="Settings"
android_find_nodes by="resource_id" value="com.android.chrome:id/url_bar"
android_find_nodes by="content_desc" value="Chrome"
android_find_nodes by="class_name" value="android.widget.EditText"
```

`exact_match=false` (default) matches contains, case-insensitive.
`exact_match=true` matches exactly.

### Retrieve full text

`android_get_node_details` with one or more `node_ids` returns full untruncated
text and description — use when the node table shows `...truncated`.

### Wait for an element

```
android_wait_for_node by="text" value="Example Domain" timeout=10000
```

### Wait for UI to settle

```
android_wait_for_idle timeout=5000
```

Returns when the screen stops changing (similarity-based). Some pages never go
fully idle (animations, feeds); use a lower `match_percentage` (e.g. 95) or rely
on `wait_for_node` instead. Action tools already wait for their gesture to
complete before returning — only call `wait_for_idle` when confirming a
transition.

## Go to home screen

```
android_press_home
```

Then call `android_get_screen_state` to see launcher icons. Tap an icon with
`android_tap_node` (uses node_id from the tree).

## Go to Chrome

Two approaches:

**Via `open_app` (preferred — launches the app directly):**
```
android_open_app package_id="com.android.chrome"
```

**Via home screen tap (if the app icon is visible on the launcher):**
```
android_press_home
android_get_screen_state
# Find the Chrome icon node, then:
android_tap_node node_id="<chrome_icon_node_id>"
```

After launching, call `android_get_screen_state` to see Chrome's UI. On a new
tab the URL bar has `res_id=com.android.chrome:id/url_bar` (toolbar) or
`com.android.chrome:id/search_box_text` (large centered search box on the
new tab page).

### Chrome toolbar resource IDs

| Element | resource_id | content_desc |
|---------|-------------|--------------|
| URL bar | `com.android.chrome:id/url_bar` | - |
| Home button | `com.android.chrome:id/home_button` | "Open the home page" |
| New tab button | `com.android.chrome:id/optional_toolbar_button` | "New tab" |
| Tab switcher | `com.android.chrome:id/tab_switcher_button` | "See N tabs" |
| 3-dot menu | `com.android.chrome:id/menu_button` | "Customize and control Google Chrome" |
| Voice search | `com.android.chrome:id/mic_button` | "Start voice search" |
| Google Lens | `com.android.chrome:id/lens_camera_button` | "Search with your camera using Google Lens" |

From jason-riddle/agent-skills, MIT.

More from jason-riddle/agent-skills