---
title: "BioBasicsBench makes failed controls the test for lab agents"
description: "OpenAI's 100-task wet-lab benchmark shows a higher average score alongside wrong colony selections and counting errors. Lab assistants need explicit control checks and human review before their output guides the next experiment."
url: "https://devencelab.com/insights/2026/11/08/biobasicsbench-makes-failed-controls-the-test-for-lab-agents"
date: "2026-11-08"
section: "Insights"
tag: "Model Releases"
author: "Devence Lab"
reading_time: "2 min read"
site: "Devence Lab"
license: "Readable and quotable with attribution to the canonical URL."
---

# BioBasicsBench makes failed controls the test for lab agents

OpenAI's 100-task wet-lab benchmark shows a higher average score alongside wrong colony selections and counting errors. Lab assistants need explicit control checks and human review before their output guides the next experiment.

OpenAI's November 6 BioBasicsBench report gives GPT-6 Astra 66.7% of available credit on 100 routine wet-lab tasks at its xhigh reasoning setting, up from 43.8% for GPT-5.6 Sol. That is a substantial gain in a controlled evaluation. It still leaves room for errors that change what a scientist does next.

The benchmark uses raw gels, colony plates, microscopy, instrument exports and lab notes. Its rubrics award partial credit, so the headline number is a mean task score, not the share of experiments an agent can safely interpret without review. That distinction matters whenever a recommendation selects a sample or rules out a failed control.

## A correct observation can lead to the wrong decision

OpenAI describes a colony PCR task where Astra separated the visible bands into two groups but assigned the groups the wrong experimental meaning. The reference selected colonies 5, 9 and 10; the model selected 4, 6, 7 and 8. The intermediate visual reading was useful, yet the proposed next step would have purified the wrong plasmid.

Another example shows Astra overcounting colonies by about 12% relative to the benchmark reference. It missed close neighbours, skipped visible colonies and marked empty agar. A polished final answer could conceal those specific mistakes unless the count is tied back to individual image regions.

> A lab agent should earn trust at the control and decision boundary, not from its average score.

## Make uncertainty an evaluated output

The stronger model did recognise some ambiguity better. In one image, overlapping dye detection channels meant brightness alone could not establish that cells were dead; Astra asked for better controls. It also spotted a negative control plate labelled with the wrong antibiotic in a cloning task. These are the behaviours an assistant should be rewarded for: noticing when the supplied evidence cannot support the requested conclusion.

But the report also documents cases where Astra let a familiar assumption override control comparisons or ignored lighter vacuoles after observing them. Better perception does not guarantee that the final recommendation carries the contradiction forward. Evaluation should score the final decision and its stated limits separately from object detection or arithmetic.

## Gate the next experiment, not just the answer

For teams piloting lab assistants, require a trace from every recommendation to the relevant image, instrument reading and control. Ask the agent to name missing or invalid controls and to withhold a selection when they determine the result. Have a scientist review any output that chooses colonies, changes a protocol or rejects a dataset.

BioBasicsBench is evidence of progress on routine analysis, not a field validation of autonomous lab work. The useful release gate is narrower: can the system preserve uncertainty, catch control failures and avoid acting on a plausible but unsupported conclusion?

## Sources

- [Measuring how AI agents handle everyday wet-lab science](https://openai.com/index/measuring-how-ai-agents-handle-wet-lab-science) - OpenAI
