Anthropic’s new AI can use computers like a human, redefining automation for enterprises

Anthropicthe AI ​​research and security company, has announced a new set of capabilities, including an enhanced version of its flagship AI model, Claude 3.5 Sonnetand a new model, Claude 3.5 Haiku– that could change the way companies automate complex workflows. But the most notable development in this release is a new feature: Claude can now use a computer like a human, navigating screens, clicking buttons and typing text.

This new feature, called “Computer use”, could have far-reaching consequences for industries that rely on repetitive tasks involving multiple applications and tabs. From data entry to research to customer service, the potential applications are broad and potentially industry-defining.

AI moves from text to screen interaction

Since its founding, Anthropic has focused on creating AI models that are safe, reliable and capable of complex reasoning. With Claude 3.5 Sonnet and Haiku, the company is expanding the model’s capabilities even further. The new Compute Usage feature allows AI to perform tasks previously only performed by human workers, such as opening applications, interacting with interfaces, and filling out forms.

“Computer usability capabilities have the potential to change the way tasks that require navigation through multiple applications are performed,” said Mike Krieger, Chief Product Officer at Anthropic, in an exclusive interview with VentureBeat. “This could lead to more innovative product experiences and streamlined back-office processes.” Krieger emphasized that the new capability is still in beta, but as the technology evolves it could improve data analysis, visualization and user interface interactions, making many tasks more efficient.

“We expect it will be especially useful for tasks such as conducting online research, performing repetitive processes such as testing new software, and automating complex, multi-step tasks,” he said. “As the technology matures, it can improve data analysis, visualization, and user interface interactions, potentially improving accessibility… We’re excited to see how developers will leverage this opportunity to create new tools and workflows that improve the improve productivity and user experiences across industries.”

Early adopters see potential

Anthropic’s early partners including GitLab, KanvaAnd Repetitionalready benefit from the new features of Claude 3.5 Sonnet. GitLab, which specializes in software development and security, tested the model for automating tasks in their development pipeline. According to the company, Claude has improved reasoning ability by up to 10% without slowing down performance, making it well suited for complex, multi-step processes such as software testing and deployment.

Replit, a coding platform, has gone one step further. Michele Catasta, president of Replit, said the model “opens the door to creating a powerful autonomous verifier that can evaluate apps as they are built.” This could eliminate the bottlenecks in software development, where testing often delays project timelines.

Meanwhile, Canva, the graphic design platform, is exploring how Claude’s computing skills can speed up design creation and editing. Danny Wu, head of AI Products at Canva, said in a statement: “We’re discovering time savings within our team that could be game-changing for users.”

What does ‘computing’ actually mean?

What sets this new capability apart from traditional automation tools is that Claude is not limited to specific workflows or software programs. Instead, it can “see” a screen using screenshots, interact with different applications, and adapt to different tasks as they arise. This flexibility makes it more versatile than current robotic process automation (RPA) technologies.

For example, in a demo shared by Anthropic, Claude helps fill out a supplier request form for Ant Equipment Co. In the video, Claude starts by taking a screenshot of the computer screen, determines that necessary information is missing from a spreadsheet, and then navigates to a CRM system, locates the necessary data, and fills out the form, all without human intervention.

This level of automation could have a major impact on industries such as finance, legal services and customer support, where tasks often involve switching between multiple systems and applications. “Claude could open spreadsheets, perform analyzes and create visualizations. For customer service, it could be navigating CRM systems to quickly find and update customer information,” Krieger told VentureBeat.

Security and privacy issues

However, AI’s ability to control a computer poses serious security and privacy concerns. Anthropic has built several security measures into the system to address these risks. The company made it clear that Claude cannot access a computer without a developer providing the necessary tools.

‘Claude can’t ‘just use your computer.’ The computing feature requires developers to provide tools such as a screenshot tool and an action execution layer, which allow Claude to perform mouse movements and keystrokes,” Krieger explains.

Anthropic is also taking a cautious approach by releasing the feature in a limited public beta, available only through an API. This allows developers to test it in controlled environments before it becomes more widely available. The company has also developed classifications to detect misuse and prevent the AI ​​from interacting with sensitive websites, such as government portals. “Our methods for scanning for prohibited activity are designed to ensure the privacy and confidentiality of customer data,” Krieger said.

A new era for office automation?

In the short term, companies could see immediate productivity gains in areas such as data entry, customer service and IT support. But as the technology matures, its potential applications could extend far beyond these initial use cases.

Imagine a world where AI handles complex legal processes, from reviewing contracts to filling out compliance forms. Or imagine AI helping doctors navigate electronic health records and diagnose patients by comparing medical databases.

Claude’s new ‘Computing’ feature brings us closer to a future where AI can perform a wide range of tasks spanning different software applications and systems. This gives it a level of flexibility previously unthinkable for AI technologies, which were often limited to specific, narrow tasks.

Proceed with caution

Still, it’s important to remember that this opportunity is still in its early stages. Claude’s ability to use computers is not yet perfect, and Anthropic acknowledges that it struggles with tasks that humans find trivial, such as scrolling or zooming. “Because it is still in beta and may occasionally miss short-lived actions, we recommend human supervision for high-stakes tasks,” Krieger said.

That said, Anthropic is committed to refining the technology. “We have developed new classifications and rapid analysis tools to identify potential misuse of computer usage features,” Krieger added, signaling that the company is serious about addressing the risks associated with this powerful technology.

What’s next?

As AI continues to evolve, the way we work could change dramatically. For business decision makers, the benefits of automating multi-step workflows can be significant. But this also raises questions about the future of jobs that depend on these tasks.

For now, Anthropic is focusing on the immediate benefits of Claude 3.5 Sonnet and Haiku while ensuring the technology is deployed responsibly. As Krieger put it, “We’re excited to see how developers will leverage this opportunity to create new tools and workflows that improve productivity and user experiences across industries.”

With companies like GitLab, Canva and Replit already exploring its potential, it’s clear that AI is poised to play an even bigger role in the future of work – perhaps sooner than we think.