Agentic AI
When the control is a human reviewer, the human is the attack surface
One disclosed incident involved fabricated identities used to manipulate a reviewer into approving agent actions. Human-in-the-loop is a real control, and it has a threat model nobody writes down.
Among the agent incidents disclosed this year, one stands out for what it targeted: fake identities used to manipulate a human reviewer into approving what the agent proposed. Not a bypassed control. A satisfied one.
Human-in-the-loop is doing more work than its design assumes
Approval gates are the most common control in agentic deployments, and the reasoning is sound — a person reviews consequential actions before they take effect. It is the control most programmes lean on to get past their risk committee.
The design assumes the reviewer has enough context to judge, enough time to exercise it, and no adversary shaping what they see. In production, none of those reliably hold. Reviewers see a proposed action and a justification, both generated by the system being reviewed, at a volume calibrated to throughput rather than scrutiny.
A reviewer approving the hundredth request of the day is not a control. They are a latency cost with a signature attached.
The failure modes are well studied elsewhere
Everything known about alarm fatigue in clinical settings and alert fatigue in security operations applies directly. Approval quality degrades with volume, degrades faster when the base rate of genuine problems is low, and degrades fastest when the interface presents a default action.
Agent approval queues have all three properties by construction. Most proposals are fine, which is what makes the rare bad one invisible.
Designing a gate that survives contact
Three things separate a real gate from a rubber stamp. Independent evidence: the reviewer must see something not produced by the system under review — the raw record, the actual diff, the source document. Volume discipline: if more requests arrive than can be genuinely examined, the gate has already failed and the threshold needs raising until it holds. And asymmetric friction: approving something irreversible should cost more effort than approving something you can undo.
Where those cannot be met, be honest that the gate is a logging mechanism rather than a control, and put the actual constraint in the authorisation layer where it does not depend on anyone's attention.
Sources
- Agentic AI Security: Lessons From Real 2026 Incidents — Lumenova
- Securing AI agents: When AI tools move from reading to acting — Microsoft Security
Written by the Devence Lab research team.
