“AI software company Anthropic has announced a new tool that can take control of the user’s mouse cursor and perform basic tasks on their computer.

Announced alongside other improvements to Anthropic’s Claude and Haiku models, the tool is straightforwardly called ‘Computer Use.’ It’s available exclusively with the company’s mid-range 3.5 Sonnet model right now, via the API. Users can give multi-step instructions (Anthropic claims it can go for tens or even hundreds of steps) to accomplish tasks on the user’s computer by ‘looking at a screen, moving a cursor, clicking buttons, and typing text.’

Competing companies like OpenAI have been working on equivalent tools but have not made them publicly available yet. It’s something of an arms race, as these tools are projected to generate a lot of revenue in a few years if they progress as expected.

There’s a belief that these tools could eventually automate many menial tasks in office jobs. It could also be a useful tool for developers in that it could ‘automate repetitive tasks’ and streamline laborious QA and optimization work.”

From Ars Technica.