What is Computer use?

In Anthropic's computer use tool, the model controls a desktop through screenshots, mouse actions and keyboard input. It is a client tool: you declare it in the tools array, the model asks for an action, and your application performs that action in an environment you control. The model never connects to the machine directly.

The documentation describes an agent loop. The model replies with one or more tool_use blocks, such as taking a screenshot, clicking at pixel coordinates, typing text or pressing a key. Your code runs them in order and returns a tool_result for each, with images for screenshots. The model reviews the results and either continues or gives a final answer. The docs recommend an iteration cap to limit cost. The reference setup is a Docker container with a virtual display, a lightweight desktop and a few preinstalled applications.

The page lists several limits. Screenshots must be scaled to fit model image limits and coordinates mapped back to the real display. Each screenshot adds roughly 1,000 to 1,800 tokens of context. Clicks can miss because of scaling errors or small targets.

The security guidance is explicit. Run it in a dedicated virtual machine or container with minimal privileges, avoid giving the model sensitive data such as login details, and ask a human to confirm decisions with real-world consequences, because instructions on webpages or in images might override your instructions.