AI agents are all the rage, but theyâre not quite as capable as their purveyors would have you believe.
Efforts to benchmark agents tasked with complex work show that they arenât typically reliable enough to function without close supervision. LiveBench 2.0 shows Claude Fable 5.2 and ChatGPT-6 Astra succeeding on 82.9 and 75.6 out of 100 tasks, respectively, and an effort by the AI infrastructure company EON to test agents in a simulated corporate environment found the most successful model tested, Claude Opus 4.8, performed tasks at 76.8% accuracy.
Now, a startup known as Plurai wants to boost those numbers with the same techniques that put self-driving cars on the street: Simulations using synthetic data based on a companyâs own digital systems, meant to prepare models to tackle difficult edge cases.
Co-founders Ilan Kadar and Elad Levi previously worked for Nexar, which develops autonomous driving technology. Kadar worked with Waymo to simulate and evaluate its end-to-end deep learning driving software.
âThe moment that we saw the wave of AI agents, we saw the same pattern: That it is very easy to build a demo, but [hard] to get something that will be reliable and cost effective,â Kadar told TechCrunch. âSince we saw the same pattern, the same pitch was that we want to build this simulation layer.â
On Wednesday, the startup said it had raised $13 million in seed funding to make that happen. The round was led by Team8, with participation from U&I, Velocity and Mercer.
Like the end-to-end models trained to drive cars, AI agents are probabilistic systems operating in unpredictable environments. When pressed by customers, agents tasked with booking travel can ignore spending policies. Or, when tasked with querying one database, agents can reliably provide an answer, but combining information from multiple data sets tends to gum up the works. A long conversation with a customer service bot can overwhelm an agent with too much context and lead to wrong turns.
To fix that, Plurai creates a digital twin of the companyâs electronic systems, including the knowledge base and tools that an agent will have access to in production. It then maps all the possible flows of the agents, and uses that graph to prepare the model with evaluations and fixes for possible edge cases. The platform works alongside existing agentic tools like Claude Code, Codex, Copilot and Cursor.
However, Kadar says the companyâs real promise is using the data it gathers to post-train small, open-weight models tailored to individual customer needs. Tests performed in cooperation with Nvidia and Nebius showed the companyâs models outperforming mid-tier frontier models at lower costs.
âWe can take a large language model, we can take the task, decompose it into a smaller task using the simulation, train [an open-weight] model, and using the simulation, verify that the quality is higher or comparable to the frontier model, and an order or two orders of magnitude cost reduction,â he said.
Reah Miyara, a Google DeepMind researcher who backed Plurai as an angel investor, said the companyâs edge comes from understanding failure modes in production, and generating synthetic data to tune models. As businesses try to optimize their AI expense, Pluraiâs approach could prove a real advantage, since IT leaders are seeing that âfrontier-capable models are out of distribution with my specific data, and the cost of these things isnât sustainable in the long term,â Miyara said.
Plurai hopes analogies between AI agents in physical and digital environments can prove predictive: While powerful general AI models are imbuing robots with fascinating capabilities, they arenât yet reliable enough to be deployed widely. Meanwhile, robots powered by AI models built for specific tasks are in the field right now, working for customers and gathering valuable real-world data.
âMy view is that no matter how smart OpenAI and Anthropic model will be â and they will be much smarter â they will not know your business and your policy,â Kadar said.
When you purchase through links in our articles, we may earn a small commission. This doesnât affect our editorial independence.
Tim Fernholz is a journalist who writes about technology, finance and public policy. He has closely covered the rise of the private space industry and is the author of Rocket Billionaires: Elon Musk, Jeff Bezos and the New Space Race. Formerly, he was a senior reporter at Quartz, the global business news site, for more than a decade, and began his career as a political reporter in Washington, D.C. You can contact or verify outreach from Tim by emailing [email protected] or via an encrypted message to tim_fernholz.21 on Signal.
Source: TechCrunch · Tim Fernholz
At a glance
- Topic
- Technology
- Country
- đșđž US
- Publisher language
- EN
- Added to Allnira