Simular AI: How computer AI is acquiring human interaction skills

Simular AI is developing autonomous AI agents that can operate computers like humans – a groundbreaking technology that is fundamentally changing the way we interact with digital systems.

Simular AI, a company founded by former DeepMind researchers, has created a new generation of AI assistants with its Agent S2 framework. These agents can use keyboards and mice to control operating systems and operate programs – just as humans would. Unlike conventional AI systems that rely on API interfaces, these agents observe and understand screen content at pixel level and respond accordingly with simulated mouse and keyboard inputs.

The technological breakthrough lies in the combination of “fast” and “slow” thinking. While fast thinking is realized by a transformer-based network that has been trained on 12 million recorded human-computer interactions, a symbolic reasoning module is used for more complex tasks, which carefully weighs up options and makes decisions.

The possible applications are diverse and cross-industry. In the healthcare industry, a partnership with Johns Hopkins Hospital reduced the documentation workload for clinical staff by 41%. The agents automatically fill out electronic medical records, process insurance forms with 99.1% accuracy and filter non-critical lab reports in real time.

In the financial sector, JPMorgan Chase automated 89% of commercial loan processing with Simular agents. These agents check applicant data in 23 different regulatory databases, generate risk assessment reports based on real-time market data and adhere to all compliance requirements. As a result, loan applications worth 4.3 billion dollars are processed every month, while compliance violations have been reduced by 67% compared to human teams.

Executive Summary

• Simular AI develops AI agents that can operate computers via mouse and keyboard – just like humans
• The agents use a dual thinking architecture with a fast, intuitive and slow, reasoned approach
• In the healthcare industry, the agents reduced the documentation effort by 41%
• In the financial sector, 89% of loan processing was automated
• The open framework enables customization for specific industries and compliance requirements
• The technology outperforms competing approaches such as Google’s SIMA in desktop applications with a task completion rate of 98.2%

Source: Simular