Bot Detection Regex Inconsistently Handles Google Inspection Bots

[vitaliemiron]

Link to the code that reproduces this issue

https://github.com/vitaliemiron/reproduction-next-bot-detection

To Reproduce

The current isBot() function in Next.js does not correctly identify certain Google bot variants, specifically:

Google-InspectionTool
Google-CloudVertexBot
Google-Other

These bots are not recognized by the existing bot detection regex, which can lead to unexpected rendering behaviors or potential performance issues.
Reproduction Steps

Create a Next.js application
Use a user agent string containing one of the undetected bot types:

Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Google-InspectionTool/1.0;)

Observe that the current isBot() function returns false

Current vs. Expected behavior

The isBot() function should return true for all Google-related bot user agents, including:

Google-InspectionTool
Google-CloudVertexBot
Google-Other

Current Implementation
Current regex in is-bot.js:

javascriptCopy/Googlebot|Mediapartners-Google|AdsBot-Google|googleweblight|Storebot-Google|Google-PageRenderer|Bingbot|BingPreview|Slurp|DuckDuckBot|baiduspider|yandex|sogou|LinkedInBot|bitlybot|tumblr|vkShare|quora link preview|facebookexternalhit|facebookcatalog|Twitterbot|applebot|redditbot|Slackbot|Discordbot|WhatsApp|SkypeUriPreview|ia_archiver/i

Proposed Solution
Update the bot detection regex to include the missing Google bot variants:

javascriptCopy/Googlebot|Mediapartners-Google|AdsBot-Google|googleweblight|Storebot-Google|Google-PageRenderer|Bingbot|BingPreview|Slurp|DuckDuckBot|baiduspider|yandex|sogou|LinkedInBot|bitlybot|tumblr|vkShare|quora link preview|facebookexternalhit|facebookcatalog|Twitterbot|applebot|redditbot|Slackbot|Discordbot|WhatsApp|SkypeUriPreview|ia_archiver|Google-InspectionTool|Google-CloudVertexBot|Google-Other/i

Impact

Ensures consistent bot detection across all Google bot variants
Prevents potential rendering or caching issues
Improves compatibility with various Google bot inspection tools

Provide environment information

Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 23.2.0: Wed Nov 15 21:53:34 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T8103
  Available memory (MB): 16384
  Available CPU cores: 8
Binaries:
  Node: 22.12.0
  npm: 10.9.0
  Yarn: 1.22.19
  pnpm: N/A
Relevant Packages:
  next: 15.2.0-canary.13
  eslint-config-next: 15.2.0-canary.13
  react: 19.0.0-rc-b01722d5-20241114
  react-dom: 19.0.0-rc-b01722d5-20241114
  typescript: 5.6.3
Next.js Config:
  output: N/A
✨  Done in 0.86s.

Which area(s) are affected? (Select all that apply)

Not sure

Which stage(s) are affected? (Select all that apply)

Vercel (Deployed), next start (local), Other (Deployed), next build (local)

Additional context

available for last canary