Daily SEO Note — September 10, 2026: Vercel Makes Production Sign-In Walls Free on Every Plan
1. SEO for Content Writers
Google's ranking, spam, and Discover systems logged nothing in the window. The Search Status Dashboard was clear as of 2026-09-10 06:03 UTC, and the newest entry in the Search documentation changelog is still 8 September. The one editorial change with a Google source behind it sits in Merchant Center, where AI Performance Insights now names the search intents, terms, and product attributes that conversational shopping queries actually use — which turns product copy from a guess into a measured surface. The bigger risk on this desk is not editorial at all: on Monday, a CDN setting can remove a site from Google without anyone touching a word of it.
Merchant Center Now Names the Terms and Attributes AI Shoppers Use
Google's AI Performance Insights help page now documents three insight categories alongside the existing share-of-voice, frequency, and products-showing metrics: top search intents (the context behind a conversational query, not the keyword), top terms (what shoppers in a product category are actively prioritising), and popular attributes (structured specs such as Size, Color, or Material that customers look for). The report splits all of it across three shopping stages — Discovery, Evaluation, and Ready to Buy. Search Engine Roundtable surfaced the addition on 9 September 2026; the help page carries the detail itself.
Who it affects: e-commerce content only, and only some of it. The insights are live for English-language queries on Merchant Center accounts in Australia, Canada, India, New Zealand, and the United States. The data covers organic AI traffic — free listings — with paid ads excluded, and refreshes daily with a few days of lag.
What to do differently in the next brief: write the product description against the top-terms list for that category instead of against a keyword export, and treat the popular-attributes list as a gap report. If the report says shoppers ask about Material and your description never says it, that is a sentence to add, not a spec table to invent. Brief the copy by shopping stage too: a Discovery description and a Ready to Buy description are answering different questions.
What to stop: stop building product copy from classic web-search keyword volumes alone. The intents in this report are conversational and stage-segmented, and Google is now telling you which ones you are and are not matching.
A One-Hour Search Console Blip Is Not a Reason to Rewrite Anything
On the evening of 8 September 2026, Search Console began reporting indexed pages as “Crawled — currently not indexed.” Google's John Mueller confirmed the fault on Bluesky the following morning: “I double-checked with the team, and it was a short blip that was resolved quickly in the night.” It affected the reporting layer only — the pages stayed indexed throughout — and it was resolved in roughly an hour.
Who it affects: all content, briefly. The practical consequence is not the blip but what teams do with it. A spike in “Crawled — currently not indexed” reads like a quality verdict, and it is the single most common trigger for an unplanned rewrite queue.
What to do: annotate 8 September in the content calendar and move on. Before any page enters a rewrite queue on the strength of a Console status, confirm it with a live URL Inspection test and wait for the status to hold across two crawls. What to stop: stop reading a single evening's Console snapshot as an editorial signal. Note too that this never appeared on the Search Status Dashboard — the dashboard tracks Search itself, not faults in the reports about it.
Your Search Eligibility Could Change on Monday Without an Edit
This is the item to raise in today's stand-up. Cloudflare's new AI-bot defaults take effect on 15 September 2026, announced on 1 July 2026. Under them, mixed-purpose crawlers — the ones that combine Search with Training — are blocked by every configuration that blocks AI training, including the legacy “Block AI bots” toggle. Cloudflare names the three affected crawlers explicitly: Googlebot, Applebot, and BingBot.
Who it affects: every piece of content on any site fronted by Cloudflare, regardless of vertical. The consequence is not a ranking adjustment; it is the crawler being turned away at the edge, and pages falling out of the index over the following days.
What to do before Monday: ask whoever owns the CDN one question, in writing, and get an answer today — is AI-training blocking or the legacy “Block AI bots” option enabled on our zone, and has the opt-out been marked. Section 2 names the exact setting and a way to verify it from outside the network. What to stop: stop treating “should we block AI crawlers” as an editorial policy question that content can settle on its own. As of Monday it is a search-visibility switch, and it belongs in a joint decision with engineering.
No Ranking, Spam, or Discover Update Is Running
The Search Status Dashboard shows no incident across Crawling, Indexing, Ranking, or Serving for 2–9 September. The documentation changelog has logged nothing since 8 September, when Google added guidance on regional differences in the Search experience — aggregator units, supplier units, and carousels. The Search Central blog has published nothing newer than its Search Central Live Bogotá and Ciudad de México announcement. Any movement you see in rank tracking today has no announced cause behind it.
Unconfirmed: People Also Ask Reported at 97% AI Overviews
A figure is circulating that AlsoAsked's Mark Williams-Cook measured 19.2 million English-language 2026 queries and found 97% of People Also Ask answers in the first week of September were AI Overviews, up from 86% in August. It reached the trade press through Search Engine Roundtable on 9 September, sourced to LinkedIn posts. It could not be traced to a first-party page with a stable URL, so it is unconfirmed here and does not belong in a client deck or a board slide.
The direction is consistent with what Google has documented about AI Overviews reaching more surfaces, and the editorial instruction that follows from it holds regardless of whether the number is 97% or 86%: stop writing briefs whose stated goal is owning a People Also Ask box. Write the direct answer well enough to be the sentence a generated answer quotes, and put the reason to click — first-hand testing, original data, a decision the answer cannot make for the reader — somewhere a two-line summary cannot carry it.
Apply to Your Next Brief
- Before anything else today: confirm with engineering whether Cloudflare AI-training blocking is on for your zone, and whether the 15 September opt-out has been marked.
- For product briefs in AU, CA, IN, NZ, or US: pull the Merchant Center top-terms and popular-attributes lists for the category first, and write the description against them.
- Add a line to every product brief naming the shopping stage it serves — Discovery, Evaluation, or Ready to Buy — and answer that stage's question in the first paragraph.
- Treat a missing attribute named in the report as a copy gap to close with a real sentence, not a spec table to fabricate.
- Freeze any rewrite queue built from Search Console indexing statuses seen on 8 September; re-verify with a live URL Inspection test before restarting it.
- Drop “win the People Also Ask box” from brief objectives. Replace it with a quotable definition in the first 60 words, plus one thing on the page a generated answer cannot reproduce.
- Do not cite the 97% People Also Ask figure to a client until a first-party source publishes it.
2. SEO for Developers
The most consequential engineering item today is the one with a deadline on it: in five days Cloudflare's AI-bot defaults change, and every configuration that blocks AI training starts blocking Googlebot with it. The change that actually landed inside the window is Vercel's — protecting a production domain behind a sign-in wall stopped costing $150 a month and became a free toggle on every plan, which makes full-site deindexing a one-click mistake for the first time. Both failures look identical from outside: your canonical URL stops returning your HTML to a crawler.
Breaking on 15 September: Every AI-Training Block Will Also Block Googlebot
Rollout identifier and date: Cloudflare announced the new AI-bot defaults on 1 July 2026, effective 15 September 2026. The Block AI bots documentation states it directly: “On September 15, 2026, Cloudflare will set updated defaults for new domains: bots classified as Training or as Agent will be blocked on pages that display ads, and Search will remain allowed,” and “mixed-purpose crawlers that combine Search and Training will also be blocked by all configurations to block AI training, including the legacy ‘Block AI bots’ option.”
Breaking. The second sentence is the one that reaches existing zones, not just new domains. If your zone has the legacy “Block AI bots” toggle enabled — a setting many teams switched on in 2024 and have not looked at since — then from Monday Googlebot, Applebot, and BingBot are in scope. Symptom if ignored: crawl requests are challenged or blocked at the edge, Search Console crawl stats fall away, and pages drop out of the index over the following days. Nothing in your repository changes, and nothing in your build logs will show it.
The setting to change: Cloudflare dashboard → the zone → Security → Settings, in the AI crawler controls where “Block AI bots” lives. Cloudflare's own instruction is that “before September 15, all customers can opt out of these new defaults” from that screen. Mark the opt-out if you intend Search crawlers to stay allowed, then verify it from an address that is not on your network — an edge rule is invisible from the office IP, because the office IP is usually allowlisted.
#!/usr/bin/env bash
# Run this from OUTSIDE your own network. An edge block is invisible from
# an allowlisted office or CI IP, which is how these go unnoticed for days.
SITE="https://example.com"
GOOGLEBOT='Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)'
BROWSER='Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 Chrome/140.0.0.0'
# 1. What does the edge hand a Googlebot user agent right now?
curl -sS -o /dev/null -w 'googlebot %{http_code} %{redirect_url}\n' \
-A "$GOOGLEBOT" "$SITE/"
# 2. Same URL, ordinary browser UA. A different status is an edge rule,
# not an application bug. 403/503 to (1) and 200 to (2) is the tell.
curl -sS -o /dev/null -w 'browser %{http_code} %{redirect_url}\n' \
-A "$BROWSER" "$SITE/"
# 3. robots.txt as served, not as committed. The CDN can prepend to it.
curl -sS -A "$GOOGLEBOT" "$SITE/robots.txt" | head -40
# 4. Before trusting any log line claiming to be Googlebot, verify it:
# reverse DNS must end in googlebot.com/google.com AND resolve forward
# to the same IP. https://developers.google.com/search/docs/crawling-indexing/verifying-googlebot
IP=66.249.66.1
host "$IP" | awk '{print $NF}' | sed 's/\.$//' | xargs -r hostVercel Authentication Now Walls Off Production Domains at No Cost
Version and date: Vercel changelog, 9 September 2026. “Vercel Authentication can now protect all deployments in a project, including production, at no additional cost on every plan.” Covering production previously required the $150-per-month Advanced Deployment Protection add-on. Deployment Protection Exceptions became free on all plans in the same entry, and Pro teams can now enable Password Protection per project rather than team-wide.
Non-breaking as shipped, and severe if enabled by accident. Symptom: every request to the production domain gets a Vercel login redirect. Googlebot follows it to a sign-in page on vercel.com, never sees your HTML, and the site deindexes. The reason this now matters is purely economic — a $150/month add-on was a deliberate purchase that someone signed off on, whereas a free toggle is something a new team member flips while trying to hide a staging branch. The Vercel Authentication documentation lists exactly who can get through, and no search crawler is on that list.
The setting: Project → Deployment Protection → Vercel Authentication, where you pick the deployment environment. In the REST API it is the ssoProtection.deploymentType field, and the value that now reaches custom production domains for free is all. Pin every public project to preview and assert it in code, so the dashboard toggle cannot quietly win. Note that Protection Bypass for Automation is not an escape hatch here: it is a secret header you send from your own tooling, and there is no way to hand it to Googlebot.
// Pin Deployment Protection to previews only, so that a free dashboard
// toggle can never put a Vercel sign-in wall in front of Googlebot on the
// production domain. Run in CI on every deploy of a public project.
const PROJECT_ID = process.env.VERCEL_PROJECT_ID!
const res = await fetch(`https://api.vercel.com/v9/projects/${PROJECT_ID}`, {
method: 'PATCH',
headers: {
Authorization: `Bearer ${process.env.VERCEL_TOKEN}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
// 'all' -> includes custom production domains
// 'prod_deployment_urls_and_all_previews' -> generated prod URLs + previews
// 'preview' -> previews only (public sites)
// null -> Vercel Authentication off
ssoProtection: { deploymentType: 'preview' },
}),
})
if (!res.ok) throw new Error(`deployment protection pin failed: ${res.status}`)Next.js Canary Gives Every Parallel Route Its Own Metadata Chain
Version and date: pull request #97345 merged on 9 September 2026 and shipped the same day in v16.4.0-canary.25. The stable line is still 16.3.4 — there is no stable release to take here, and this is a watch item, not a same-day pull request.
Non-breaking; the new resolver sits behind experimental.parallelRouteMetadata. The symptom it addresses is present today with the flag off: parallel routes contribute to a single shared metadata chain, so traversal order decides which slot supplies the document head, and a rejected generateMetadata in one slot can interfere with another. If you run parallel routes and have ever seen the wrong slot's title win, or one slot's metadata error take out a page that should have rendered, that is this.
With the flag on, each branch resolves with its own accumulator. Head selection becomes deterministic at every fork — real routes are preferred over implicit fallbacks, and children slots are preferred when they contribute metadata — and each slot gets its own metadata outlet that replays only its own errors, regardless of which branch supplied the head. Eager generateMetadata and generateViewport invocation is preserved, so sibling branches are not serialised. Test it on canary in a branch, and diff the rendered head of every parallel-route page before and after.
import type { NextConfig } from 'next'
// Canary only (16.4.0-canary.25+). The stable line is 16.3.4 — do not
// upgrade production for this. With the flag off, parallel routes share
// one metadata chain and traversal order decides which slot wins <head>.
const nextConfig: NextConfig = {
experimental: {
parallelRouteMetadata: true,
},
}
export default nextConfigOne Monitor Catches Both of Today's Failure Modes
A Cloudflare edge block and an accidental Vercel sign-in wall produce the same observable: a crawler asks for your canonical URL and does not get your HTML back. Neither shows up in a build log, neither fails a test suite, and both are typically discovered from Search Console days later, after the deindexing has already started. The check that catches them is a crawler-user-agent request from off your network, asserted on status, redirect target, and the presence of a title.
Run it on a schedule rather than in the deploy pipeline — the failure modes here are configuration drift at the edge and in a dashboard, neither of which is tied to a deploy. The robots.txt assertion below is deliberately noisy: it fires on any global disallow, including a legitimate one scoped to a single user-agent group, which is a false alarm worth having.
#!/usr/bin/env bash
# Cron this from outside your network (GitHub Actions, an uptime worker).
set -euo pipefail
SITE="https://example.com"
UA='Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)'
fail() { echo "SEO GUARD FAIL: $1" >&2; exit 1; }
# 1. Canonical URL must return your HTML to a crawler, not a login redirect.
read -r CODE REDIR <<<"$(curl -sS -o /tmp/home.html \
-w '%{http_code} %{redirect_url}' -A "$UA" "$SITE/")"
[ "$CODE" = "200" ] || fail "home returned $CODE (redirect: ${REDIR:-none})"
case "${REDIR:-}" in
*vercel.com*|*login*|*sso*) fail "home redirects to a sign-in wall: $REDIR" ;;
esac
grep -qi '<title' /tmp/home.html || fail "home returned no <title>"
# 2. robots.txt as the edge serves it. Intentionally noisy on any global
# disallow — a CDN can prepend directives you never committed.
ROBOTS="$(curl -sS -A "$UA" "$SITE/robots.txt")"
grep -qiE '^[[:space:]]*Disallow:[[:space:]]*/[[:space:]]*$' <<<"$ROBOTS" \
&& fail "robots.txt contains a site-wide Disallow"
# 3. Sitemap must still be served to a crawler UA.
curl -sSf -o /dev/null -A "$UA" "$SITE/sitemap.xml" || fail "sitemap unreachable"
echo "SEO GUARD: ok"
exit 0Checked in the Window, No Action
Schema.org is still on v30.0, released 2026-03-19 — no vocabulary change to track. Lighthouse is still on 13.4.1; pin that version in CI so a future bump does not arrive as an unexplained score change. Google's crawling documentation changelog has logged nothing since 22 July 2026. OpenAI's bot documentation still lists the same four agents at the same versions — GPTBot/1.4, OAI-SearchBot/1.4, ChatGPT-User/1.0, OAI-AdsBot/1.0 — so yesterday's robots.txt decisions still hold.
The two entries in Cloudflare's changelog dated 9 September 2026 — cache-token rates in AI Gateway custom costs, and an iOS tap-to-type fix in Browser Isolation — touch no crawl, cache, or SEO surface. The GitHub Advisory Database batch published on 9–10 September 2026 carried no advisory against an SEO dependency: the Nuxt-adjacent entry, CVE-2026-59158, is against the nuxt-ollama package, not a Nuxt SEO module, and the smol-toml denial-of-service entry, CVE-2026-85730, sits in configuration parsing rather than in any sitemap, metadata, or structured-data path.
Ship Today
- Open the Cloudflare dashboard for every production zone, check whether AI-training blocking or the legacy “Block AI bots” option is enabled, and mark the opt-out before 15 September if Search crawlers must stay allowed.
- Run scripts/check-googlebot-access.sh from outside your network against every production hostname, and compare the Googlebot response to the browser response.
- Audit ssoProtection.deploymentType on every public Vercel project and pin it to “preview” in code, so the newly free production toggle cannot be enabled by accident.
- Schedule scripts/seo-guard.sh every 15 minutes from an off-network runner, and route its failures to the same channel as a production incident.
- Pin Lighthouse to 13.4.1 in CI, so the next release does not arrive as an unexplained score regression.
- Watch only, do not ship: experimental.parallelRouteMetadata on Next.js canary. The stable line is 16.3.4.
Comments
Share your thoughts and join the conversation