ios-simulator-skill

Files (46)

• SKILL.md
• openai.yaml · agents/
• accessibility_audit.py · scripts/
• app_launcher.py · scripts/
• app_state_capture.py · scripts/
• appearance.py · scripts/
• build_and_test.py · scripts/
• clipboard.py · scripts/
• __init__.py · scripts/common/
• cache_utils.py · scripts/common/
• device_utils.py · scripts/common/
• env_config.py · scripts/common/
• hang_pipeline.py · scripts/common/
• hang_sessions.py · scripts/common/
• idb_utils.py · scripts/common/
• screenshot_utils.py · scripts/common/
• container.py · scripts/
• gesture.py · scripts/
• hang_watcher.py · scripts/
• keyboard.py · scripts/
• localization_audit.py · scripts/
• location.py · scripts/
• log_monitor.py · scripts/
• media.py · scripts/
• model_inspector.py · scripts/
• navigator.py · scripts/
• privacy_manager.py · scripts/
• push_notification.py · scripts/
• screen_mapper.py · scripts/
• sim_health_check.sh · scripts/
• sim_list.py · scripts/
• simctl_boot.py · scripts/
• simctl_create.py · scripts/
• simctl_delete.py · scripts/
• simctl_erase.py · scripts/
• simctl_shutdown.py · scripts/
• simulator_selector.py · scripts/
• status_bar.py · scripts/
• test_recorder.py · scripts/
• visual_diff.py · scripts/
• __init__.py · scripts/xcode/
• builder.py · scripts/xcode/
• cache.py · scripts/xcode/
• config.py · scripts/xcode/
• reporter.py · scripts/xcode/
• xcresult.py · scripts/xcode/

SKILL.md (first lines)

# iOS Simulator Skill

Build, test, and automate iOS applications using accessibility-driven navigation and structured data instead of pixel coordinates.

## Quick Start

```bash
# 1. Check environment
bash scripts/sim_health_check.sh

# 2. Launch app
python3 scripts/app_launcher.py --launch com.example.app

# 3. Map screen to see elements
python3 scripts/screen_mapper.py

# 4. Tap button
python3 scripts/navigator.py --find-text "Login" --tap

# 5. Enter text
python3 scripts/navigator.py --find-type TextField --enter-text "user@example.com"
```

**Paths:** `scripts/` is relative to the directory containing this SKILL.md, not the user's project.
Examples write `scripts/...` for brevity; run them by absolute path from the user's project
(e.g. `python3 <skill-dir>/scripts/navigator.py ...`) so relative Xcode project paths still resolve.

All scripts support `--help` for detailed options; most support `--json` for machine-readable output.

## Navigation Strategy

**Always prefer the accessibility tree over screenshots for navigation.** The accessibility tree gives you element types, labels, frames, and tap targets — structured data that's cheaper and more reliable than image analysis.

Use this priority:
1. `screen_mapper.py` → structured element list (5-7 lines, ~10 tokens)
2. `navigator.py --find-text/--find-type/--find-id` → semantic interaction
3. Screenshots → only for visual verification, bug reports, or visual diff

Screenshots cost 1,600–6,300 tokens depending on size. The accessibility tree costs 10–50 tokens in default mode.

## 30 Production Scripts

### Build & Development (2 scripts)

1. **build_and_test.py** - Build Xcode projects, run tests, parse results with progressive disclosure
   - Build with live result streaming
   - Parse errors and warnings from xcresult bundles
   - Retrieve detailed build logs on demand
   - Options: `--project`, `--scheme`, `--clean`, `--test`, `--verbose`, `--json`

2. **log_monitor.py** - Real-time log monitoring with intelligent filtering
   - Stream logs or capture by duration
   - Filter by severity (error/warning/info/debug)
   - Deduplicate repeated messages
   - Options: `--app`, `--severity`, `--follow`, `--duration`, `--output`, `--json`

### Device State (3 scripts)

3. **appearance.py** - Control simulator appearance: dark mode, Dynamic Type size, contrast, and locale/region
   - Toggle light/dark theme via `xcrun simctl ui`
   - Set Dynamic Type size with friendly aliases (XS through AX5)
   - `--contrast high|normal` toggles Increase Contrast and reads it back to confirm; `--reset` also restores normal contrast
   - Write locale and region defaults; optional app restart via `--bundle-id`
   - RTL flagged automatically for ar/he/fa/ur/yi locales
   - Options: `--theme`, `--text-size`, `--contrast`, `--locale`, `--region`, `--reset`, `--bundle-id`, `--udid`, `--json`, `--verbose`

4. **location.py** - Simulate GPS coordinates, named city presets, and GPX scenario playback
   - Fix a coordinate with `--lat`/`--lng` or pick a city with `--city`
   - Play a built-in scenario (City Run, Freeway Drive, etc.) via `--gpx <scenario>`
   - Animate multi-waypoint paths with configurable speed via `--waypoints` and `--speed`
   - Clear simulated location with `--clear`; list available scenarios with `--list-scenarios`
   - Options: `--lat`, `--lng`, `--city`, `--gpx`, `--waypoints`, `--speed`, `--clear`, `--list-scenarios`, `--udid`, `--json`, `--verbose`

5. **media.py** - Seed or empty the photo library for photo-picker tests
   - `--add a.jpg b.png` via `xcrun simctl addmedia`
   - `--clear --yes` empties the library via `idb photos clear` (idb 1.7+); `--clear --yes --add` leaves exactly the added files
   - Without `--yes`, `--clear` refuses if the library has photos (they can't be recovered): ask the user first
   - Fails fast on a missing file
   - Options: `--add`, `--clear`, `--yes`, `--udid`, `--json`

### Navigation & Interaction (5 scripts)

6. **screen_mapper.py** - Analyze current screen and list interactive elements
   - Element type breakdown
   - Interactive button list
   - Text field status
   - If labelled elements cover under 10% of the screen, adds a `Coverage:` line suggesting `--deep`
   - `--deep` reads through idb's axbridge backend (idb 1.7+): also sees Safari/WKWebView content
     and in-app alerts the default read misses, plus many unlabelled container views. It turns on
     the device's accessibility automation mode (not restored afterwards). Web-content frames are
     relative to the web view, so use `--deep` to find elements, not to take tap coordinates
   - Options: `--verbose`, `--hints`, `--deep`, `--json`

7. **navigator.py** - Find and interact with elements semantically
   - Find by text (fuzzy, or exact with `--find-exact`); interactive leaf elements rank above containers sharing the label
   - Find by element type
   - Find by accessibility ID
   - Enter text or tap elements
   - `--tap` on a Button/Link/Switch/CheckBox/Tab with a unique accessibility ID or label uses an accessibility press
     (idb 1.7+), so overlays like a floating search bar can't intercept it; output says
     `(accessibility press)`. Otherwise it taps the frame center
   - `--set-value TEXT` replaces a field's value without the keyboard (any characters, idb 1.7+).
     Focus doesn't move, so use `--enter-text` when a Return key must submit afterwards
     The value is read back before success is claimed (not for secure fields); a mismatch exits 1
   - `--tap --expect VALUE` taps only if the element's value is VALUE (toggles: `1` on, `0` off) and
     exits 1 without tapping otherwise, so "turn X off" is safe to repeat
   - iOS 27 reports toggles as `CheckBox`. Only a press flips them (no HID touch does, even on the
     switch itself); if neither ID nor label is unique, the output warns to check the value
   - A frozen app stalls every read and tap until its main thread responds
   - Failures include idb's stderr (e.g. a dead companion)
   - Options: `--find-text`, `--find-exact`, `--find-type`, `--find-id`, `--index`, `--tap`, `--expect`, `--tap-at`, `--enter-text`, `--set-value`, `--list`

8. **gesture.py** - Perform swipes, scrolls, pinches, and complex gestures
   - Directional swipes (up/down/left/right) - `--swipe` is the direction the finger moves
   - Multi-swipe scrolling - `--scroll` is the direction the content moves (`down` reveals content below)
   - Pinch zoom via `idb ui pinch` (`--pinch out` zooms in; `--scale` sets an explicit factor)
   - Long press via `idb ui tap --duration`
   - Pull to refresh
   - `--double-tap X,Y [--tap-count N]` - multi-tap at a point (idb 1.7+)

From conorluddy/ios-simulator-skill, MIT.

More from conorluddy/ios-simulator-skill