Insights

    Agentic AI

    XYBench shows agent evaluations must test the problem behind the request

    Devence Lab

    · 2 min read

    Share
    XYBench shows agent evaluations must test the problem behind the request
    Photograph · Jakub Zerdzicki via Pexels

    Ai2 found that five models often answered a user's proposed method while missing the underlying goal. Agent teams should test whether a system can redirect a mistaken request before it acts.

    Ai2 released XYBench on 13 November with 8,115 questions built around a common failure: a person asks for a method that will not solve the problem they actually have. On Stack Overflow questions, five tested models focused primarily on that underlying goal in only 35% to 41% of their responses, according to Ai2. That is a poor starting point for an agent allowed to act on a user's first instruction.

    The benchmark separates the stated request from the intended goal. A user might ask how to strengthen frosting that slides off a cupcake when the cupcake's temperature is the cause. The useful response identifies the misconception and changes course. An agent with tools could instead spend time or money executing the mistaken method.

    Mentioning the right fix is insufficient

    Ai2 measures whether a response includes a solution to each request, where it puts its emphasis, and whether it steers away from the misconception. On Stack Overflow, the models mentioned an underlying-goal solution in 59% to 71% of answers, yet focused mainly on that goal in only 35% to 41%. A passing sentence can conceal an answer dominated by the wrong plan.

    Human reference answers were more likely to redirect. Ai2 reports that top human answers steered users to the underlying goal in 79% of those questions; the tested models ranged from 47% to 60%. The figures describe a single-turn English-language evaluation, mostly of procedural problems. They do not measure what a deployed multi-step agent will do.

    An agent should earn permission to execute the plan by showing it understood the goal.

    Recognition and generation diverge

    Ai2 also offered models two candidate answers: one pursued the literal request, the other corrected the premise. The models selected the goal-focused answer in 71% to 84% of Stack Overflow cases. Their own written answers focused on the goal far less often. A multiple-choice preference test therefore misses the behaviour an agent needs to produce.

    Even supplying the intended goal did not reliably fix the problem. In one oracle test, Sonnet 4.6 included more underlying-goal solutions but identified misconceptions less often. This suggests that detecting intent and giving it priority are separate capabilities.

    Put a goal check before action

    For agent deployments, add cases where the user's proposed step is plausible but counterproductive. Score whether the system names the likely goal, tests the assumption, and seeks clarification when evidence is thin. Then test the full workflow: did it pause before a purchase, file change or external message that only served the mistaken method?

    That gate is an operational inference from Ai2's response study. XYBench supplies a useful set of questions and separate measures, but teams should add their own tasks, languages and multi-turn tool use. The release makes one omission harder to defend: measuring literal task completion while never checking whether the task was the right one.

    Sources

    1. XYBench: Helping with the problem behind the question · Allen Institute for AI

    Written by the Devence Lab research team.

    Share