Model Releases

7 analyses from Devence Lab

September 13, 2026 · 2 min read

A 7% pass rate is not a step change. It is a benchmark with no floor yet.

GPT-6 Astra completed 7 of 100 dual-arm robotics tasks on a new benchmark, versus zero for a competing model. A researcher called it a step change. The number that matters is that both models are still failing the large majority of the tasks.

September 7, 2026 · 2 min read
Release notes just became compliance artifacts
With AI Act enforcement live, the AI Office can request technical documentation, evaluate models and require corrective measures. What a lab publishes at launch now has a regulatory reader.
September 6, 2026 · 2 min read
Capability thresholds are becoming a disclosure norm. Deployers should read them as a handoff.
Labs now publish where they think a model crosses into dangerous capability. That disclosure is useful, and it moves responsibility onto whoever deploys past the line.
September 5, 2026 · 2 min read
Your model's deprecation date is a risk you do not control
With releases arriving weekly, the version you qualified has a shelf life set by someone else's roadmap. Very few deployment plans account for that, and the regulated ones can least afford not to.
September 3, 2026 · 2 min read
A model scored 100% on ExploitBench. That tells you about the benchmark.
OpenAI's Astra reportedly saturates an offensive security benchmark while declining 91.5% of jailbreak attempts. Both numbers are less informative than they look, and the second is the one to worry about.
September 2, 2026 · 2 min read
GPT-6, Grok 4.7 and Gemini 3.8 shipped inside ten days. Your qualification cycle did not.
Frontier releases are now arriving faster than any serious evaluation process can absorb them. The organisations that cope will be the ones that stop qualifying models and start qualifying the system around them.
September 1, 2026 · 2 min read
The first cyber-defence model shipped as a product, not a research artefact
Gemini 3.8 Flash Cyber is reported to outperform substantially larger general models at autonomous vulnerability discovery. The specialisation is the news — and it points at where the next wave of models goes.

A 7% pass rate is not a step change. It is a benchmark with no floor yet.

Release notes just became compliance artifacts

Capability thresholds are becoming a disclosure norm. Deployers should read them as a handoff.

Your model's deprecation date is a risk you do not control

A model scored 100% on ExploitBench. That tells you about the benchmark.

GPT-6, Grok 4.7 and Gemini 3.8 shipped inside ten days. Your qualification cycle did not.

The first cyber-defence model shipped as a product, not a research artefact

We share findings with partners operating in the same constraint space.