messages

Files (2)

• SKILL.md
• get-last-messages.mjs · scripts/

SKILL.md (first lines)

# Messages.app Automation

## Connect

```
connect_app({ name: "Messages" })
```

## Layout

Messages has a two-pane layout: conversation list (left) and transcript (right).

- **Window title** reflects the currently open conversation's contact name.
- The window identifier is always `SceneWindow`.

## Key Elements

### Toolbar

| Query | Label | Action |
|---|---|---|
| `[{role:"window"}, {role:"toolbar"}, {role:"button"}]` | Compose | Start new conversation |
| `[{role:"window"}, {role:"toolbar"}, {role:"menu_button", index:0}]` | filter | Filter: Messages / Spam / Recently Deleted |
| `[{role:"window"}, {role:"toolbar"}, {role:"menu_button", index:1}]` | Start FaceTime | Call menu |

### Search

```json
[{"role":"window"}, {"role":"group", "identifier":"CKConversationListCollectionView"}, {"role":"text_field"}]
```
Label: "Search". Use `fill` to type a search query.

### Conversation List

Container:
```json
[{"role":"window"}, {"role":"group", "identifier":"ConversationList"}]
```

Each conversation is a `text` element. Iterate with `index`:

```json
[{"role":"window"}, {"role":"group", "identifier":"ConversationList"}, {"role":"text", "index": N}]
```

- **Pinned contacts** come first (indices 0, 1, 2, ...). Label format: `"Name, Pinned"`
- **Regular conversations** follow. Label format: `"Name, Preview text, Date"`

The `label` field contains all info — contact name, last message preview, and timestamp are comma-separated.

To open a conversation, the user must click it manually — see Gotchas below.

### Conversation Title

```json
[{"role":"window"}, {"role":"button", "identifier":"ConversationTitle"}]
```

Shows the currently open contact's name in `label`. Click to view contact details.

### Message Transcript

Container:
```json
[{"role":"window"}, {"role":"group", "identifier":"TranscriptCollectionView"}]
```

Two types of children:

**Date/time separators** — `text` elements. Iterate with `index`:
```json
[..., {"role":"text", "index": N}]
```
Label examples: `"Nov 6, 2022 at 16:30"`, `"Sun, Mar 1 at 00:50"`

**Message bubbles** — `group` elements with `identifier: "Sticker"`. Iterate with `index`:
```json
[..., {"role":"group", "identifier":"Sticker", "index": N}]
```

- The group's `label` contains a summary: `"Your iMessage, message text, time"` or `"Contact name, message text, time"`. Use this to determine sender and direction.
- For the full message text, go one level deeper:
  ```json
  [..., {"role":"group", "identifier":"Sticker", "index": N}, {"role":"text_area", "identifier":"CKBalloonTextView"}]
  ```
  The `value` field has the complete message content.
- **SMS messages** may not have a `CKBalloonTextView` child. In that case, extract the text from the Sticker group's `label` — the format is `"Sender, message text, time"`, so strip the first and last comma-separated segments.

### Message Input

Container:
```json
[{"role":"window"}, {"role":"group", "identifier":"MessageEntryView"}]
```

| Query (from MessageEntryView) | Label | Purpose |
|---|---|---|
| `{"role":"text_field", "identifier":"messageBodyField"}` | Message / iMessage | Text input |
| `{"role":"button", "index":0}` | add | Attachment picker (+) |
| `{"role":"button", "index":1}` | Record audio | Voice message |
| `{"role":"button", "index":2}` | Emoji picker | Emoji keyboard |

### Useful Menu Items

| Path | Action |
|---|---|
| `File > New Message` (id: `new_message`) | Cmd+N — new conversation |
| `Edit > Send Message` (id: `keyCommandSend:`) | Send the composed message |
| `Conversation > Delete Conversation…` (id: `keyCommandDeleteConversation:`) | Delete current conversation |
| `Conversation > Mark as Unread` (id: `keyCommandToggleUnreadState:`) | Toggle unread state |
| `View > Show Times` (id: `keyCommandToggleTimeStamp:`) | Show/hide timestamps |

## Gotchas

### Clicking conversation items causes multi-select

Messages uses a SwiftUI list for the conversation sidebar. Synthesized CGEvent clicks (coordinate-based) are always interpreted as Cmd+click, causing multi-select instead of switching conversations. This is a macOS-level behavior — physical hardware clicks work, synthesized ones don't. **No workaround exists at the CGEvent level.**

**Consequence:** You cannot programmatically switch between conversations. You can only read from the **currently open** conversation.

From wende/macbeth, MIT.

More from wende/macbeth