AI Process Manager (AIPM)

Ping the AI Process Manager backend. Call this FIRST in a new session. Returns version, API URL, whether the local service is reachable, and two things that decide your first action: executor (which tools act through UI Automation and need NO foreground, versus which emit real input and REQUIRE it) and forgepilot (whether the real-input arm is installed and running), plus warnings[] with a next_action when the arm is installed but stopped. The route catalogue is deliberately omitted — ask aipm_ja_faz(intent) for that.

List running processes sorted by CPU. Use "top" to limit (default 25, max 200). Prefer check_process when searching for one process. Command lines truncated unless full_command_line=true.

CATALOGUE of the desktop windows: one entry per window with title, process, minimized/maximized state, position, Z-order and which has focus. Answers "what windows are open", "which is in the foreground", "is X minimized". It does NOT give the text inside a window (use read_window) and does NOT give the UI element structure (use get_ui_tree). If the user asks to READ something or CLICK something, this is not the tool.

Leave a note on the local noticeboard for the next agent (or for the human reading later). Notes stay on this machine: never telemetry, never network. Worth writing when you measured something that contradicts a document, hit a trap that cost you time, made a decision the next agent would otherwise re-litigate, or finished something they might redo. Say what you MEASURED and with which command — a note asserting state without the command that produced it is a memory, not a finding. Max 500 chars, so write the conclusion, not the story. Mention @<name> to address a specific agent (they find it …

THE NOTICEBOARD other agents left for you on this machine — findings, decisions, blockers and done reports. Call this FIRST in a session, before measuring anything: what looks like a fresh discovery is usually already answered here, and a note may say the thing you are about to investigate was fixed hours ago. Notes never leave this machine (never telemetry, never network). Start with no filter to get the `subjects` map (rooms + note counts) and pick the room where the work actually happens — the biggest room is rarely the one you assume. Then re-call with subject=. Use para=<your-name> as the…

Read the raw TEXT that is INSIDE a window via UI Automation — no screenshot. Answers "what does the window say": console output, render progress ("frame 4812/5000"), editor content, chat messages. Works with consoles (cmd, PowerShell, Windows Terminal), editors and most native apps. You MUST identify the window first: pass title_contains (or hwnd from list_windows). It does NOT list windows (use list_windows) and does NOT return the UI element tree (use get_ui_tree).

Report one UI action outcome (invoke/set_value/focus). Feeds per-app recipes in get_app_knowledge. Metadata only — never screen content or typed text. Report FAILURES too (ok=false plus reason): they become common_failures and stop the next agent from repeating the same dead end.

Report task outcome to local telemetry (metadata only — never screen content). Call at END of a computer-use task. Feeds get_app_knowledge and get_economy_stats.

ACT BY INTENT - the first thing to try for click / type / pick. One call replaces choosing between ui_invoke / ui_set_value / ui_select_option AND doing the fallback dance by hand: it tries UI Automation first, drops to real input BY ITSELF only when UIA reports the pattern is missing, and tells you which one ran (mecanismo: uia | fisico). Two measured reasons to prefer it over ui_invoke. (1) AMBIGUITY. name_contains is a SUBSTRING match and every other action tool silently takes the FIRST hit - in a chat window that regularly means a word out of the conversation history, not the control you m…

Drag from one element to another (or between two absolute screen pixels). Use this for anything that only moves by dragging: video-editor timelines, kanban cards, calendar events, reordering lists, drag-to-upload. There is no UIA pattern for dragging, so this emits REAL mouse input with intermediate movement steps — measured: a teleport fires dragstart but never drop; the steps are what makes the drop land. Prefer from_index/to_index (the ledger then records the element ROLE, so the action is auditable); the pixel form exists for canvas/GDI surfaces with no UIA tree. REQUIRES the target window…

List interactive UI elements in a window (buttons, fields, menus first; grid cells last). Each item has an element_index plus can_invoke/can_set_value flags. Read-only, no action tier required, and it does NOT need the window focused or in the foreground. Pass element_index to ui_act (preferred: it is the aim that is never refused for ambiguity) or to ui_invoke/ui_set_value/ui_select_option. TWO TRAPS, both measured: (1) element_index is valid for THIS SNAPSHOT ONLY — it is an identity within one walk of the tree, not a durable handle, and it ages on every re-render (a re-render has been seen …

Find the UI element under an ABSOLUTE screen pixel (x, y) in a window. Returns the SAME global element_index as ui_find (so you can probe coordinates then act), plus element_role/element_name when available. For UIA-empty windows (canvas games, Java, GDI) it returns element_index -1 with a hint — no accessibility tree; use real clicks at those coordinates instead. NEVER clicks anything, but like the action endpoints it is opt-in (action tier + allowlist).

Semantically CLICK a UI element via UI Automation (button, menu item, checkbox). Target by name_contains or element_index from ui_find. USE ui_act INSTEAD WHEN THE NAME MAY NOT BE UNIQUE: name_contains is a SUBSTRING and this tool takes the FIRST hit SILENTLY (in a chat window it routinely matches the conversation history, not the control); ui_act refuses that with 409 plus the candidates. Prefer ui_act too when this returns elemento_nao_suporta_invoke - it falls to real input by itself instead of dead-ending. This tool stays right when you already hold a fresh element_index and want UIA only.…

Pick an option in a dropdown / <select> / combo box by its visible text. Use this instead of ui_invoke or ui_set_value for lists: a Chromium <select> supports neither invoke nor toggle, and set_value silently does nothing. WORKS WITH THE WINDOW IN THE BACKGROUND: it goes through UI Automation, not synthetic input, so the window does NOT need focus or foreground — do not call focus_window first "just in case" (only ui_send_keys and ui_drag require it). Expands the list, re-walks the tree (options do not exist while it is closed), selects by text, verifies the displayed value changed, then colla…

LAST RESORT for typing, and there is now a shorter road: ui_act (intent=escrever) makes the ui_set_value attempt AND this fallback in a single call, deciding between them itself and reporting mecanismo: uia | fisico. Reach for ui_send_keys directly when you need a KEY PRESS (Enter, Tab, ctrl+enter - ui_act does not press keys), or when you want to drive each step by hand. Check the key parameter before planning: the vocabulary is closed and has no letters, no F-keys and no arrows, so ctrl+a / ctrl+s / ctrl+v are NOT available. Otherwise: use ui_set_value FIRST, and only reach for this when it …

PREFERRED WAY TO TYPE. Set text in an editable field via UI Automation (no key simulation). WORKS WITH THE WINDOW IN THE BACKGROUND: it writes through the accessibility provider, not the keyboard queue, so the window does NOT need to be focused or in the foreground — unlike ui_send_keys, it can fill a form while the user types in another app. Do NOT call focus_window first "just in case". The write is VERIFIED: the value is read back and compared, so ok:true means it actually landed; a control that silently ignores the write returns valor_nao_aplicado — and the shortest way out of that is ui_a…

Long-poll until a condition is met (or timeout). Replaces polling loops. Conditions: process_ended, process_started, file_stable (render/download done), file_exists, window_title_contains, note_after_seq (a new note on the agent mural). Max 50s per call; if satisfied=false, call again to keep waiting.