---
title: "Taste-Bench exposes the wrong decisions inside long-running AI agents"
description: "Taste-Bench finds weak judgement at critical decision forks even in frontier agents. Teams should benchmark abandoned branches, information-gathering choices and cost per validated direction."
url: "https://devencelab.com/insights/2026/10/18/taste-bench-exposes-the-wrong-decisions-inside-long-running"
date: "2026-10-18"
section: "Insights"
tag: "Agentic AI"
author: "Devence Lab"
reading_time: "2 min read"
site: "Devence Lab"
license: "Readable and quotable with attribution to the canonical URL."
---

# Taste-Bench exposes the wrong decisions inside long-running AI agents

Taste-Bench finds weak judgement at critical decision forks even in frontier agents. Teams should benchmark abandoned branches, information-gathering choices and cost per validated direction.

An October 18 report on Taste-Bench revisits a September 2026 study with an uncomfortable result: the strongest model tested chose the better path in only 59.7% of benchmark decisions. The benchmark targets a specific failure mode in long-running AI research and engineering: choosing what to try next when several plausible directions remain open. A system can execute tools fluently while repeatedly investing time in the wrong idea.

The researchers call this skill an agent's 'taste'. They mine decision forks from parallel attempts at the same task and detours inside recorded trajectories. At each fork, the tested model sees the context and two candidate directions but not the future outcomes. The score therefore concerns judgement at the decision point, not merely whether the finished task passed a test.

## A convincing plan is not evidence of a good decision

The paper reports 59.7% average accuracy for GPT-5.6 Sol, narrowly ahead of GPT-5.5 at 59.5%. Accuracy requires choosing correctly with both answer orders, limiting gains from position preference. These are benchmark-specific results, not an estimate that the models make the wrong choice in 40% of all enterprise decisions.

The particularly weak cases are detours: a model must recognise that an ongoing line of work should be abandoned before the original agent did. Engineering and research results also vary. For production agents, that suggests measuring early direction changes and abandoned approaches separately from the final success rate.

> An agent that completes tasks can still waste most of its budget choosing which task branch deserves another hour.

## More reasoning tokens did not solve the fork

The authors find that forks requiring later evidence are harder, and increasing the reasoning budget does not consistently improve accuracy. That matters when an orchestrator responds to uncertainty by requesting another long chain of model reasoning. Additional tokens cannot reveal experimental evidence that has not yet been collected.

Build checkpoints that favour cheap tests over confident speculation. Log the alternatives considered, the evidence used to select one, the estimated cost of exploration, and the observation that would trigger a reversal. For high-impact actions, require a measurable result before committing more resources.

## Evaluate judgement at the point of commitment

The researchers also show that a student model trained with hindsight from a teacher can improve unseen fork decisions and downstream software-engineering outcomes. This is promising research, not a production assurance claim. Real systems will encounter unknown tasks and shifting criteria.

Teams should construct an internal decision-fork set from past agent runs, comparing alternate branches where results are observable. Review incorrect choices, missed stop signals, wasted compute and human intervention. Combine the metric with task completion, cost and safety outcomes. The central engineering lesson is to test the quality of intermediate decisions, not reward a polished final answer that hides costly detours.

Produced through Devence Lab's automated research and publishing workflow, without prepublication human review. Devence Lab retains editorial responsibility. Sources are linked; report factual errors to contact@devencelab.com.

## Sources

- [The Tasteful Agent: Measuring and Improving Taste in Long-Horizon Tasks](https://arxiv.org/abs/2609.25804) - Wenbo Pan et al. / arXiv
- [AI agents have bad taste in long-term decisions and a new benchmark proves it](https://the-decoder.com/ai-agents-have-bad-taste-in-long-term-decisions-and-a-new-benchmark-proves-it/) - The Decoder