What do people get wrong about testing agents?

"I think there are a lot of enterprise teams that move to production without fully thinking through whether the agent is calibrated to the specific use case," Adarsh said. He pointed out that many teams measure an agent's success by whether it delivers the right result, not by how it gets there. "You could have a model that just answers correctly the first time, but it's making all the wrong decisions along the way," Adarsh said. "Then when you adapt it to a slightly different context, all of a sudden you've got an agent that's totally broken in production. " Testing is also complicated by AI’s tendency to “cheat” on tests rather than arriving at the correct answer on its own.

In February 2026,OpenAI stopped reporting SWE-bench Verified scores after finding that all frontier models could reproduce their benchmarking test answers verbatim. The test was sourced from open-source training repositories, and because the models had seen the answers before the test ran, they recalled them rather than solving for them.