---
title: "49% fewer tokens came from the harness, not the model. Benchmark the control layer."
description: "SoL-Pi cut recorded coding-agent token traffic by 44.7–49.0% while keeping comparable task performance. Agent economics depend on the harness that manages context, tools and observations."
url: "https://devencelab.com/insights/2026/09/26/49-fewer-tokens-came-from-the-harness-not-the"
date: "2026-09-26"
section: "Insights"
tag: "Agentic AI"
author: "Devence Lab"
reading_time: "2 min read"
site: "Devence Lab"
license: "Readable and quotable with attribution to the canonical URL."
---

# 49% fewer tokens came from the harness, not the model. Benchmark the control layer.

SoL-Pi cut recorded coding-agent token traffic by 44.7–49.0% while keeping comparable task performance. Agent economics depend on the harness that manages context, tools and observations.

SoL-Pi reduced recorded token traffic by 44.7–49.0% on a 51-task coding evaluation while delivering performance comparable to the Pi harness across GPT-5.6 Sol and Claude Opus 5. The researchers report roughly one-third lower API cost. They did not train a cheaper model. They changed the control layer around it.

That distinction matters for agent platforms because model price is only one input to task economics. The harness decides what context survives, how tool results are represented, when observations are compacted and which work is delegated. Those choices determine how many tokens the same model consumes to finish the same job.

## Context management is an execution policy

The SoL-Pi paper describes four mechanisms that survived automated selection: action execution, context compaction, observation handling and delegated reading. The system emerged from an auto-research process that tested harness variants across increasingly diverse environments rather than tuning model weights.

This makes context engineering measurable infrastructure. A harness that repeatedly serialises large tool outputs can spend tokens without improving the agent's decision. A harness that removes the wrong history can save tokens and destroy task state. The useful metric is therefore not compression ratio. It is accepted task performance at a measured token and cost budget.

> An agent's model sets capability, but its harness decides how much of that capability gets purchased on every step.

## Benchmark the model and harness as one solver

The reported savings are not universal. SoL-Pi's headline result comes from EdgeBench, and the researchers note transfer across models and settings rather than claiming a fixed reduction for every coding workload. Production teams should treat the figures as a reason to test the harness, not as a forecast for their own bill.

Hold the model, task set and acceptance criteria constant, then vary the harness. Record completed-task rate, prompt and completion tokens, wall-clock time, tool-call count, retries and API cost. The comparison should include failure cases because an inexpensive trajectory that requires another full run can cost more than a longer successful one.

## Put harness changes through release gates

Harness optimisation also changes behaviour. Compaction can remove evidence the model later needs; delegated reading can alter what enters context; action execution changes can affect tool sequencing. A token-saving patch is therefore a behavioural release, not a FinOps tweak.

Version the harness beside the model and prompt. Re-run representative agent evaluations whenever context policy, tool adapters or observation formatting changes. In production, attribute cost and success metrics to that exact combination so regressions can be traced to the layer that moved.

For teams operating coding agents at scale, the practical change is to stop treating model tokens as a fixed property of the model. Measure the harness as part of the solver, optimise it under task-level quality gates, and buy fewer tokens only when the accepted outcome stays intact.

## Sources

- [SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness](https://arxiv.org/abs/2609.20519) - arXiv
- [Nvidia's SoL-Pi system cuts coding agent token usage nearly in half by optimizing the harness](https://the-decoder.com/nvidias-sol-pi-system-cuts-coding-agent-token-usage-nearly-in-half-by-optimizing-the-harness/) - The Decoder