AI5 min readArs Technica AI

Claude Code can now take over your computer to complete tasks

P
Redakcja Pixelift0 views
Share
Claude Code can now take over your computer to complete tasks

Foto: Getty Images

Artificial intelligence has just gained physical control over our computer cursors. Anthropic has announced that its Claude Code and Claude Cowork tools can now independently navigate the desktop, open files, use browsers, and run developer tools to complete assigned tasks. While the system's priority remains using direct integrations like Connectors, in their absence, the AI will ask the user for permission to "scroll, click, and explore" screen content in real time. For the global community of creators and developers, this signifies a shift from simple code generation to full workflow automation. Crucially, thanks to the Claude Dispatch feature, these processes can be initiated and supervised remotely, provided the target machine remains powered on. The practical implications are vast: from autonomous debugging of complex projects to automatic data population between applications that lack an official API. Rather than serving as a passive advisor, Claude is becoming a digital agent capable of performing operations directly within the user's workspace, drastically reducing the time required for repetitive technical tasks. This mechanism pushes the boundary between a support tool and an autonomous collaborator operating at the graphical interface level.

The line between an AI assistant and an autonomous system operator has just blurred. Anthropic, one of the leading players in the artificial intelligence market, has announced the introduction of a feature that allows the Claude Code and Claude Cowork models to take direct control of a computer's desktop. This is no longer just about generating code or suggesting text edits; it is an evolution toward full-fledged AI agents capable of "seeing" a graphical interface, clicking icons, and navigating the operating system exactly as a human would.

The decision to release tools capable of independently operating on files and browsers places Anthropic right at the center of the race for dominance in the "AI Agents" category. However, the company makes no secret of the fact that this technology is currently in a research preview phase, which comes with a clear warning: safety measures are not absolute. In a world where AI can independently run developer tools and explore disk resources, the issue of trust and control becomes a key point of technological debate.

Autonomy within the operating system

The operating mechanism of Claude Code is based on the ability to visually interpret what is happening on the user's screen. The model is no longer limited to working within a closed chat environment; it can now "point, click, and navigate", allowing it to open files, use a browser, and automatically run developer tools. This is a response to the growing demand for process automation that previously required tedious, manual switching between application windows.

It is worth noting the hierarchy of actions adopted by Anthropic. The system first attempts to use so-called Connectors, which are dedicated connections allowing direct communication with external applications or data sources. Only when such a connection is unavailable does Claude ask for permission to take direct control of the machine. The user must grant consent for the AI to "scroll, click to open, and explore" the system in search of the information necessary to complete the task.

Advanced computer systems operated by AI
New Claude Code features allow for direct interaction with the operating system's graphical interface.

Remote management and the Dispatch tool

One of the most intriguing aspects of the new update is the integration with the Claude Dispatch tool. It allows for initiating and managing agent actions remotely. This means that Claude Code can perform complex operations on a target computer even if the user is not physically at the keyboard – the only condition is that the machine remains turned on. This opens the door to scenarios where developers commission long-term compilation, testing, or debugging processes, and the AI agent executes them autonomously in the local environment.

However, such a system architecture necessitates a new approach to session security. Since Claude operates directly on the UI (User Interface) layer, any mistake by the model could have real consequences for system stability or data integrity. Anthropic emphasizes that while the system is powerful, users should remain vigilant, especially regarding operations on sensitive system files. The ability to remotely trigger actions via Dispatch only intensifies the need for precise logs and mechanisms for immediate agent termination.

  • Direct navigation: The ability to visually interpret the desktop and click on interface elements.
  • Dev-tools automation: Independent launching of terminals, compilers, and debuggers.
  • Remote access: Task management via the Dispatch tool without the need for physical presence.
  • Permissions system: User consent required for system exploration in direct mode.
Modern programmer workspace with AI
Claude Cowork becomes a partner in daily tasks, operating on the same tools as a human.

Security challenges in research preview

The introduction of agents capable of computer control is a step the industry is watching with a mix of admiration and concern. Anthropic openly admits that the security features in the research preview version do not guarantee 100% protection against unforeseen actions. The problem lies in the fact that language models, despite their advanced logic, can still misinterpret screen content or instructions, which – when granted "click and scroll" permissions – could lead to the deletion of important data or software misconfiguration.

A key challenge for engineers will be refining the Claude Cowork mechanisms so that the agent can distinguish between safe and critical operations. Currently, the system relies on interaction with the user, who acts as a supervisor. The AI industry faces the question: how far can we push autonomy before the risk outweighs the productivity benefits? Anthropic is betting on transparency by informing users of limitations, but simultaneously pushing the technology forward, forcing competitors (such as OpenAI or Google) to take similar steps toward integration with operating systems.

"Claude Code and Cowork will prioritize the use of Connectors, but will request full screen navigation access if necessary to complete a task."

The introduction of these features is a clear signal that the era of AI as a simple chatbot is coming to an end. We are entering the "Action-Oriented AI" phase, where a model's value is measured not only by the quality of generated text but by its effectiveness in navigating complex digital ecosystems. For developers, Claude Code is becoming more than just an editor extension – it is becoming a virtual collaborator that has its own "hands" in the form of a mouse cursor and keyboard. The success of this solution will depend on how quickly Anthropic can transform the current research preview into a stable and secure work environment that does not require constant oversight of the agent's every move.

Comments

Loading...