Indexing guard (non-production safety net)

A staging or local copy of your site leaking into Google is one of the most common — and most damaging — SEO mistakes: duplicate content competing with your real pages, a private environment sitting in the index, and weeks of cleanup with the URL-removal tool. The classic cause is a noindex that lives only in a .env you forgot to set, or a robots rule the deploy overwrote.

The indexing guard makes it structurally hard. Tie indexability to the Laravel environment instead of to a flag someone has to remember: when the app runs anywhere that isn't on your allow-list, every page is forced to noindex,nofollow, the managed robots.txt blocks all crawlers, and seo:audit says so out loud.

It is a free, core feature.

What it does when active

When app()->environment() is not in seo.indexing_guard.allowed_environments (and the guard is enabled), four things happen automatically:

1. The resolver forces noindex,nofollow on every page. This is applied above the whole precedence chain — it overrides even an explicit per-page robots value stored in seo_meta.
2. An X-Robots-Tag: noindex,nofollow HTTP header is sent on every response routed through the app — see Non-HTML responses below.
3. SEO::robotsTxt()->build() emits a disallow-all robots.txt (and ai.txt) — a plain User-agent: * / Disallow: /. This covers both the seo:robots-txt command and the optional dynamic route.
4. seo:audit prints a prominent banner, so the "everything is noindex" state is never a surprise when you read a report.

In allowed environments (by default, production) the guard is completely inert — zero changed output, byte-identical rendering.

Non-HTML responses (PDFs, feeds, images)

The forced robots meta tag only reaches crawlers that parse HTML — a PDF, an RSS/Atom feed, an image or any other non-HTML response carries no <head>. So while active, the guard also sends the same directive as an HTTP header, via a global middleware:

X-Robots-Tag: noindex,nofollow

The header and the meta tag come from the same source, so they can never disagree. It's on by default within the guard (the guard itself is opt-in and inert in allowed environments); turn it off to keep meta-only behaviour:

'indexing_guard' => [
    'send_header' => env('SEO_INDEXING_GUARD_HEADER', true),
],

The middleware is registered only when the guard is enabled, so a package with the guard off adds nothing to your stack.

Static files bypass PHP

A file your web server returns directly from public/ never enters Laravel, so it can't receive this header. Protect those at the edge (web-server config / CDN). This covers everything routed through the app.

Why it overrides an explicit robots value

Everywhere else in Rankbeam, an explicit stored value wins — that is the whole point of the precedence chain. The guard is the one deliberate exception, and it sits above the explicit layer, because the risk here is one-directional:

• A staging database is usually a clone of production, so a page that stored index,follow would carry that directive into staging and ask to be indexed.
• Wrongly indexing staging is a disaster; wrongly noindex-ing it is a no-op. So the guard is a floor the stored value cannot punch through, in exactly the environments where you never want indexing anyway.

Enabling it

The guard ships off, so installing or upgrading the package never changes what a non-production environment renders without your say-so (the same byte-identical-until-opt-in policy as the resolver's blank_is_unset and generated OG images). Arm it in one line:

SEO_INDEXING_GUARD=true

With the default allow-list, production is unaffected, so you can keep the guard enabled in shared configuration. Check that the allow-list includes every environment you intend to index. It is strongly recommended, and a candidate to default on in Core 4.

Disable it with the same one line:

SEO_INDEXING_GUARD=false

Choosing which environments may index

By default only production is allowed. Override the list with a comma-separated env var:

# Let a public preview environment index too
SEO_INDEXING_GUARD_ALLOWED="production,prod-eu"

Or in config/seo.php:

'indexing_guard' => [
    'enabled' => env('SEO_INDEXING_GUARD', false),
    'allowed_environments' => ['production', 'prod-eu'],
],

Entries are matched with Str::is(), so wildcards work — 'prod*' matches production and prod-eu:

'allowed_environments' => ['prod*'],

An empty list means no environment may index — the guard is active everywhere (the fail-safe direction). Note that an empty or blank SEO_INDEXING_GUARD_ALLOWED env value falls back to ['production'] so a typo can never silently un-index production; write an explicit [] in config if you truly want "everywhere".

Verifying it

seo:audit shows the banner and, in --json, carries the machine-readable state:

php artisan seo:audit --json

{
    "indexing_guard": {
        "active": true,
        "environment": "staging",
        "allowed_environments": ["production"],
        "directive": "noindex,nofollow"
    },
    "pages": [ /* ... */ ]
}

And the served/generated robots.txt on a guarded environment:

# robots.txt — managed by Rankbeam
# Indexing guard ACTIVE: this app is running in the "staging" environment,
# which is not in seo.indexing_guard.allowed_environments. Every crawler is
# disallowed so this non-production site stays out of search results.
# https://rankbeam.dev/guide/indexing-guard

User-agent: *
Disallow: /

Scope

The guard controls indexing directives — the robots meta tag, the X-Robots-Tag header, and robots.txt. It does not touch your titles, descriptions, canonicals, or schema, and it is independent of the robots rendering policy (seo.robots.emit_default): because noindex,nofollow deviates from the site default, it is always emitted as a tag.