Despite the loud promises of developers, modern AI agents mostly do not cope with typical office tasks. According to a study by Carnegie Mellon University (CMU) and Salesforce, artificial intelligence successfully completes only 30-35% of multi-step tasks such as navigating web pages, writing software code, or communicating with colleagues. informs The Register.
CMU created TheAgentCompany simulation environment, which simulates the work of a small IT company with typical office scenarios. Leading models participated in the tests, including the Gemini 2.5 Pro (30.3% success), Claude 3.7 Sonnet (26.3%), GPT-4o (8.6%) and Amazon Nova Pro (1.7%). Some AIs have even resorted to cheating by changing user names to simulate successful task completion.
Salesforce offered a separate CRMArena-Pro benchmark focused on customer service and sales tasks. There, the models achieved up to 58% accuracy in simple cases, but in multi-step situations the efficiency dropped to 35%. All systems tested also showed a weak understanding of privacy, raising doubts about their suitability for enterprise use.
Separately, Gartner warns about the phenomenon of agent washing — a situation where simple chatbots or RPA systems are positioned as full-fledged agents. Of the more than a thousand companies that offer similar solutions, only about 130 actually use the corresponding technologies.
Despite current limitations, Gartner analysts predict that by 2028, AI agents will make 15% of daily business decisions, and a third of enterprise software will have corresponding functions. At the same time, experts advise not to overestimate current capabilities – modern systems are still far from the level of a virtual assistant like JARVIS from “Iron Man”: most of them are not able to independently carry out complex instructions or work with a real-time interface.
