Two-thirds of tested publisher homepages let every synthetic bot through. OpenRTB still treats device.ua as identity telemetry.

What the benchmark actually measured

The external test is not a crawl of ad tags. Researchers sent automated requests to homepages across fifteen industries, routing through residential proxies in the United States, Canada, and France so sites could not filter on datacenter IP alone. Each domain received ten bot types across four tiers: two basic bots with minimal headers, four spoofed AI agents and crawlers with forged headers for a major search crawler, a conversational agent, a training crawler, and another assistant crawler, one disguised bot with forged TLS and HTTP/2 behavior, and three real-browser automations with anti-detection tooling.

A site counted as fully protected only if it blocked or challenged all ten. Partial protection meant at least one tier was stopped. Unprotected meant none were. In 2026, 65.3 percent were unprotected, up from 61.2 percent in a 2025 edition with a smaller sample. Partial protection fell from 36 percent to 32.3 percent. Fully protected sites fell from 2.8 percent in 2025 to 2.4 percent in 2026, and from 8.4 percent in 2024. The vendor warns the year-on-year decline is directional because the 2026 suite added spoofed AI agents and more sophisticated bot types, and the sample grew by more than 4,500 domains.

Tier-level results matter for how you read user-agent strings elsewhere. Only 5.5 percent of sites detected every basic Tier 1 bot. Tier 2, the spoofed AI and crawler names, had the highest detection rate: 29.3 percent stopped at least one forged agent and 14.1 percent stopped all four. The report attributes that to blocklists: publishers who added rules for well-known training crawlers catch spoofed versions as a side effect, not because the site can detect impersonation. More than seven in ten sites still let a spoofed AI agent through without challenge, and a bot using an unknown agent string passes the same lists untouched.

The customer-traffic half of the report is a different population: more than 75,000 sites that already pay for downstream bot filtering, often behind a CDN with basic bot filtering enabled. Bad bot traffic on that base grew 124.0 percent over the year while human traffic grew 13.2 percent. AI agent and crawler traffic totaled 52.7 billion requests. AI requests to login pages rose from 11.9 million in January 2026 to 99.7 million in June, an increase the report frames as 735.8 percent. Login pages took 51.7 percent of high-risk AI endpoint traffic in the first half of 2026, up from 23 percent in the prior edition. Forms still grew in absolute terms but fell as a share. None of that appears on a seller fill report. It appears on pages where lead pixels and checkout events fire after the click.

Why the bidstream sees a different gate

OpenRTB carries the user agent on the device object. Pre-bid lists, datacenter filters, and many IVT classifiers treat that string, together with IP, app or site identifiers, and schain, as the telemetry available at auction time. Media Rating Council invalid traffic guidance lists unmodified user agent among the signals that determine whether an impression is classifiable. A request that clears the exchange with a plausible device.ua is not automatically fraud. It may be exactly what a classifier needs. It is also not proof that the publisher's public homepage would have challenged the same automation on a form post.

// Same string family, different enforcement surfaces
"device": {
  "ua": "Mozilla/5.0 ... GPTBot/1.0 ...",  // pre-bid list may GIVT-flag
  "ip": "203.0.113.10",
  "ifa": "..."
},
"site": {
  "page": "https://publisher.example/signup",
  "publisher": { "id": "..." }
}

The homepage benchmark tests the marketing domain's edge. The bid request may arrive from an in-app WebView, a reseller path, or SSAI middleware where the user agent is a server string or a frozen Chrome reduction value. Buyers who sample SSP logs see one row. Security teams who run bot tests see another. Comparing them without labeling the surface is how a low platform-side IVT rate coexists with lead forms that never saw a human typist.

Spoofing rates rose in the report's February to July 2026 window. Traffic claiming known agent identities without being those agents increased 45 percent in aggregate across tracked agents. Meta-ExternalAgent saw 16.4 million spoofed requests in February and 12.7 million in July. Individual agents such as PerplexityBot showed spoofing rates around 2.4 percent in February and conversational agents near 5.6 percent spoofed by July in one category breakdown. OpenRTB has no field that says this user agent was cryptographically issued by the crawler owner. It has a string. Blocklists on the buy side and blocklists on the publish side both key off that string, which is why Tier 2 detection peaks above Tier 1 even when overall protection collapses.

The mechanism from impression to polluted downstream pools

Scraping dominated bad bot volume on the vendor customer base at 70.9 percent and grew 185.2 percent over twelve months. Simple bots, the kind that expose WebDriver signals from basic automation frameworks, made up 49 percent of a classified bad-bot sample and evaded detection on 88.6 percent of websites tested. The report's explanation for why attackers still run them is blunt: they continue to work. When those sessions trigger display clicks, fill lead forms, or add items to carts, they enter retargeting and lookalike pools that DSPs price on a later bid request with no memory of the homepage test outcome.

Coverage of the same report ties form endpoints to marketer pain explicitly: 198.9 million AI requests reached forms in six months on the vendor network, with lead fraud and form abuse among the named risks. DoubleVerify reported on July 29, 2026 that AI bots generated up to ten times more clicks than humans in some unprotected campaigns. A Lunio survey cited in the same synthesis found only 5.3 percent of marketers running a dedicated invalid traffic platform while 75.6 percent estimated losing budget to bots. Those numbers sit in click and conversion denominators. The OpenRTB impression that preceded them can look ordinary.

AI-referred traffic adds a measurement split. The report counted 89.4 million AI-referred visits to customer sites in the first half of 2026, with one assistant product generating 83.4 percent of referrals. Google Analytics added an AI Assistant channel on May 13, 2026 to classify some of that traffic. Referrals and automated requests are different quantities: roughly one referred visit for every 330 AI requests in the same half-year window in the report's arithmetic. A buyer optimizing on referral growth is not automatically filtering the crawl and login traffic that shares the same brand names in user-agent strings.

None of this replaces MRC decision-rate discipline on classified impressions. It complicates the story that if the bid request is clean, the path to conversion is clean. Unknown impressions remain unknown under MRC rules. Homepage bot gates that fail open push uncertainty to the post-click surface where pixels, not OpenRTB, record outcomes.

What to do on the buy and sell side

• Separate impression IVT from landing-page bot exposure in QBRs. A protected campaign rate on classified impressions does not describe form fills on a site that blocked zero of ten synthetic homepage visitors in June 2026.
• Sample device.ua and device.sua by supply path, not only by domain. Compare SSAI and reseller hops where the user agent is middleware against direct app traffic on the same publisher brand.
• Treat spoofed crawler strings as a list problem, not proof of intent. Tier 2 detection in the benchmark worked when names matched published crawlers. Unknown agents and disguised TLS bots bypass the same logic on both homepage and bidstream.
• Validate structural bid requests before blaming fraud tools. Missing device fields, inconsistent video objects, and broken schain push impressions toward unknown in accredited measurement. Paste production traffic into the bid request tester or follow the CI validation guide on every exchange integration release.

RTBlint does not detect bots, invalid traffic, or homepage bot management. It flags malformed or inconsistent OpenRTB payloads. A well-formed request with a spoofed or automation-heavy user agent can pass validation and still fail every business test on the landing page.

Sources

Figures on the 21,491-domain benchmark, tier detection rates, traffic growth, AI login volume, spoofing counts, and referrals come from coverage of the September 22, 2026 State of Bot and Agent Security Report 2026. OpenRTB field names follow IAB Tech Lab publications. MRC invalid traffic terminology follows accredited guidance cited in prior RTBlint posts. The analysis is our own. RTBlint is independent and not affiliated with DataDome, IAB Tech Lab, or PPC Land.

• PPC Land: Full bot protection drops to 2.4% of popular websites (September 26, 2026), synthesis of the September 22, 2026 bot and agent security report
• DataDome: State of Bot and Agent Security Report 2026 (publisher page)
• RTBlint: MRC invalid traffic rules require a decision rate (August 31, 2026)