Skip to content
Agentmart
Submit a tool

macOS Vision MCP

Local OCR & screen understanding for agents: read any window, find what to click.

Connector by woladi · GitHub account verified by the MCP Registry

IndexedMITLocal (stdio)★ 7
Write a review

Install

Works with Claude Code, Codex, VS Code, Cursor, Windsurf, Claude Desktop. Read the repository before you give any tool access to your data.

npx agentmarket install connector/woladi-macos-vision-mcp

Asks which of your tools to add it to (Claude Code, Codex, Cursor, VS Code, Windsurf, Gemini CLI, Claude Desktop) and shows the change first. See https://agentmart.dev/cli.

Get the weekly diff

The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.

We use your email only for this newsletter. Privacy

About

Local OCR & screen understanding for agents: read any window, find what to click. No uploads.

Reviews

Write a review

No reviews yet. Installed it? Reviews come from verified installers only. How reviews work

Permissions

What this server can do and what it asks for, from its tool list, install config and code. Checked Oct 26, 2026.

Tools
10 read · 1 write · 0 delete · 2 not yet classified
Secrets
None asked for
Install scripts
None found
Runs as
Node.js package

Tools by what they do 13

Read from the source code; the running server may list different tools.

Read 10 · Look things up without changing anything.

  • capture_screen
  • classify_image
  • detect_barcodes
  • detect_document
  • detect_faces
  • find_element
  • list_windows
  • ocr_image
  • read_screen_text
  • ui_snapshot

Write 1 · Create, change, send or run things.

  • assert_text

Not yet classified 2 · We could not tell from the name and description.

  • analyze_document
  • vision_capabilities

Network hosts it talks to 4

  • github.com · found in code
  • glama.ai · found in code
  • raw.githubusercontent.com · found in code
  • unpkg.com · found in code

Files it touches

  • Any path the agent passes to its tools

Tools (13)

Read from the source code; the running server may differ. Descriptions are the server’s own words; hidden characters are shown as [U+…].

  • analyze_document

    Run a full analysis pipeline on a local image or PDF and return structured JSON for document reconstruction: OCR (with line/paragraph grouping in reading order), face detection, barcode/QR detection, and rectangle detection — all in parallel, fully offline, no API key needed. USE WHEN: The user wants the model to reconstruct a document into Markdown, HTML, DOCX, or any other format — invoices, scanned reports, contracts, IDs, receipts, mixed-content scans. Returns enough structure (paragraphs + raw text blocks with bounding boxes) that the model can render the ou…

  • assert_text

    Assert that text is present on (or absent from) the screen, a window, or a pre-captured image — a local pass/fail verdict computed on this machine. The deterministic check runs here; no screenshot and no OCR dump needs to reach a cloud model for a UI test to pass. USE WHEN: Verifying UI state after an action: "the dialog says Saved", "the error banner is gone", "all three menu items are visible". Parameters: targeting like capture_screen (app / window_id / rect / display_id) or a pre-captured image via path (+ optional frame to map matches to screen points), plus: expect — string …

  • capture_screen

    Take a screenshot locally — of the main display, a specific window, an app's frontmost window, or a screen region. The image is saved to disk and NEVER returned to the model: you get back the file path plus geometry, so follow-up OCR/assertion tools can map results to screen coordinates. USE WHEN: Starting any UI check. Then chain: read_screen_text / find_element / assert_text on the returned path — or skip this tool entirely, since those tools can capture on their own. Requires: Screen Recording permission for the app hosting this MCP server. Returns: JSON { path, pixelWidth, pixelHeight, …

  • classify_image

    Classify the content of a local image into categories using Apple Vision (offline, no API key needed). USE WHEN: The user wants to know what is depicted in an image — objects, scenes, activities, animals, food, etc. Works with 1000+ categories and returns confidence scores. DO NOT USE for: text extraction (use ocr_image), face/barcode detection (dedicated tools), images that need detailed visual description (use the model's built-in vision). Returns: JSON array of classification labels sorted by confidence (highest first), each with a label name and confide…

  • detect_barcodes

    Detect and decode barcodes or QR codes in a local image file using Apple Vision (offline, no API key needed). USE WHEN: The user wants to read a QR code, barcode, EAN, UPC, Code128, PDF417, Aztec, DataMatrix or other 1D/2D code from a local file. DO NOT USE for: text extraction (use ocr_image), face detection (use detect_faces). Supported symbologies: QR, EAN-8, EAN-13, UPC-E, Code39, Code93, Code128, ITF, PDF417, Aztec, DataMatrix, GS1DataBar and more. Returns: JSON array of detected codes, each with its decoded value and symbology type.

  • detect_document

    Detect the boundary of a document in a local image using Apple Vision (offline, no API key needed). USE WHEN: The user has a photo of a piece of paper, a receipt, a card, an ID, or any rectangular document and wants the four corner points — typically as a hint for cropping, deskewing, or straightening the image before further OCR. DO NOT USE for: reading the document text (use ocr_image), classifying the image (use classify_image), or analyzing a PDF (PDFs are already rectangular pages). Returns: JSON with the four corner points of the detected document — …

  • detect_faces

    Detect human faces in a local image file using Apple Vision (offline, no API key needed). USE WHEN: The user wants to know how many faces are in a local image, or needs their positions. DO NOT USE for: text extraction (use ocr_image), barcode reading (use detect_barcodes). Returns: JSON with the total face count and an array of face positions expressed as percentage of image dimensions (top, left, width, height).

  • find_element

    Find a UI element by its visible text and return WHERE TO CLICK — center coordinates in global screen points, ready to hand to any input driver (macos-mcp Click, cliclick, CGEvent). Capture, OCR, and matching all run locally; no screenshot is ever sent to the model or the cloud. USE WHEN: An agent needs to click/interact with something it can name ("click Save", "focus the search field") — this replaces sending a screenshot to a vision model to locate the element. Matching: text is normalized (unicode, whitespace, dashes/quotes, case) and matched exact → substring → fuzzy (Levenshtein). Subs…

  • list_windows

    List on-screen application windows with their global screen-point bounds — front-to-back order (first window of an app = its frontmost one). USE WHEN: You need a windowId for capture_screen / find_element, or want to know what apps are visible before testing their UI. Returns: JSON { titlesAvailable, note?, windows: [{ windowId, app, pid, title, x, y, w, h, layer, isOnScreen }] }. Coordinates are global screen POINTS with top-left origin — the same space click drivers use. Window TITLES require Screen Recording permission for the host process. Without it macOS returns no names and every tit…

  • ocr_image

    Extract text from a local image or PDF file using Apple Vision OCR (offline, no API key needed). USE WHEN: The user provides a local file path to an image, screenshot, scanned document, or PDF and wants to extract the text from it. DO NOT USE for: images hosted on URLs (download first), non-macOS systems, or when the user wants face/barcode detection (use the dedicated tools). Supported formats: jpg, jpeg, png, heic, heif, tiff, bmp, pdf Parameters: path — absolute or relative path to the image/PDF file format — "text" returns a single plain-text string (default) …

  • read_screen_text

    Capture the screen (or a window/app/region) and OCR it in one step — fully local, the screenshot never leaves this machine. Returns the visible text so you can check UI state without sending any image to the cloud. USE WHEN: "What does the screen/app show right now?", verifying a dialog appeared, reading an error message, checking state after an action. Parameters: same targeting as capture_screen (app / window_id / rect / display_id), plus: format — "text" (default) plain text in reading order, or "blocks" with bboxes + confidence. Returns: JSON { capture: { path, frame, ... }, text } or…

  • ui_snapshot

    Return the layout of an app's window as structured JSON: every element's exact box, role, label and state from the macOS accessibility tree, optionally with colours and fonts — plus the visible text the tree does NOT account for. All computed on this Mac; the screenshot it takes never reaches the model. USE WHEN: You need to understand or reconstruct a layout rather than just find one element — "what is on this screen", "review this dialog's layout", "which controls are disabled", "is anything unlabelled for screen readers". For locating a single element to click, find_element is far cheaper.…

  • vision_capabilities

    Report what this machine can do: macOS version, Screen Recording / Accessibility permission state, available displays (with point sizes and Retina scale), and capture engine status. USE WHEN: Before starting a UI-testing session, or when capture/OCR tools fail unexpectedly — this tells you whether the problem is a missing permission and what to tell the user. Returns: JSON { macosVersion, helperVersion, permissions: { screenRecording, accessibility, screenLocked }, displays: [{ displayId, isMain, x, y, w, h, scale }], capture: { engine, windowCapture, regionCapture, windowTitles }, ready, bl…

Compare and alternatives

    Featured in

    Signals

    Stars
    7
    Forks
    0
    Open issues
    0
    Last commit
    Aug 25, 2026 (2 months ago)
    Last release
    Aug 25, 2026 (2 months ago)

    From GitHub (fetched Oct 24, 2026).

    Details

    Registry name
    io.github.woladi/macos-vision-mcp
    Version
    0.7.0
    Auth
    No auth
    Packages
    npm macos-vision-mcp
    License
    MIT
    Repository
    github.com/woladi/macos-vision-mcp
    First seen
    Oct 24, 2026
    Last checked
    Oct 24, 2026
    Source
    Official MCP Registrygithub-graphql

    More from woladi

    Cite this page

    macOS Vision MCP. Agent Mart. https://agentmart.dev/connectors/woladi-macos-vision-mcp (last updated 2026-10-24)