---
title: "AI vulnerability reports need evidence gates before they reach human triage"
description: "Google paused product submissions to its OSS bug bounty after a surge of invalid AI-generated reports. Security programmes need machine-speed evidence gates before scarce human review."
url: "https://devencelab.com/insights/2026/10/03/ai-vulnerability-reports-need-evidence-gates-before-they-reach"
date: "2026-10-03"
section: "Insights"
tag: "AI Security"
author: "Devence Lab"
reading_time: "2 min read"
site: "Devence Lab"
license: "Readable and quotable with attribution to the canonical URL."
---

# AI vulnerability reports need evidence gates before they reach human triage

Google paused product submissions to its OSS bug bounty after a surge of invalid AI-generated reports. Security programmes need machine-speed evidence gates before scarce human review.

Google paused product vulnerability submissions to its Open Source Software Vulnerability Rewards Program on 1 October after receiving a surge of invalid reports generated with AI. The failure is not that researchers used models. It is that submission throughput scaled faster than evidence quality.

Bug bounty and vulnerability-intake teams should expect this asymmetry to persist. A model can generate plausible findings, reproduction narratives and variant hypotheses cheaply. Human maintainers still pay the expensive part: reproducing the claim, tracing affected code and deciding whether the report describes a security boundary at all.

## Put proof before prose

An intake system should score evidence before it rewards a polished explanation. Require a minimal reproducible case, affected version, exact execution path and observable security impact. Where practical, run the supplied reproducer in an isolated harness and compare its claimed result with telemetry from the target.

That changes the role of AI-assisted submissions. Models can help researchers inspect code or explain a bug, but generated confidence does not substitute for an artefact that another system can execute. Reports that cannot cross that evidence threshold should never consume the same queue as reproducible vulnerabilities.

> When report generation becomes machine-speed, evidence validation has to become machine-speed too.

## Rate-limit uncertainty, not researchers

Traditional rate limits treat every submission as equivalent. AI changes the economics. Programmes should instead use progressive trust: reporters with reproducible findings earn more intake capacity, while accounts producing unverifiable claims face tighter queues and higher proof requirements.

This is also safer than banning AI-assisted research. A blanket restriction discards useful automation while doing little to distinguish a model-assisted real vulnerability from fabricated output. Evidence-based admission keeps the control attached to what matters: whether the claim survives independent verification.

## Protect maintainer attention as a security resource

Triage capacity is part of vulnerability-management availability. Flooding a queue with low-confidence reports can delay patches for real defects even when no submission is malicious. Security teams should monitor invalid-report ratios, median reproduction time and reviewer load alongside ordinary bounty metrics.

Google's pause is a warning for every programme accepting machine-generated security output. Before increasing automated discovery, build the validation path that can reject unsupported claims without human investigation. The bottleneck has moved from finding possible bugs to proving which findings deserve attention.

## Sources

- [OSS VRP pauses product vulnerability submissions](https://bughunters.google.com/blog/94085569467863680/oss-vrp-pauses-product-vulnerability-submissions) - Google Bug Hunters
- [Google freezes open-source bug bounty program amid flood of invalid AI slop submissions](https://www.tomshardware.com/tech-industry/artificial-intelligence/google-suspends-part-of-the-oss-vrp-bug-bounty-program-due-to-an-influx-of-invalid-ai-submissions-product-vulnerability-submissions-ended-october-1) - Tom's Hardware