X-Robots-Tag & Noindex Checker Tool Online

Last updated: Jul 12, 2026

Check whether a URL is indexable by reading both the HTTP X-Robots-Tag header and HTML robots meta tags. Built for technical SEO, GEO audits, and launch QA.

Enter a URL to check robots index directives:

How the X-Robots-Tag Checker Works

This checker fetches the URL and inspects the two places search engines use for index directives.

HTTP header check, the API reads the X-Robots-Tag response header, including rules that affect non-HTML files.
HTML robots check, the tool extracts meta robots and googlebot directives from the page head.
Directive parsing, values such as noindex, nofollow, none, noarchive, nosnippet, and max-snippet are normalized.
Effective verdict, the result combines header and meta signals to show whether the URL appears indexable.

Why Robots Directives Matter

A single noindex directive can remove a revenue page from search, even if the page content, schema, and backlinks are strong.

Launch protection, staging noindex rules sometimes ship to production by accident.
Header visibility, X-Robots-Tag is invisible in page source, so many basic SEO tools miss it.
AI search readiness, GEO visibility depends on pages being crawlable, indexable, and machine-readable.
Cleaner audits, this tool complements the Meta Tag Checker by focusing only on index directives and HTTP headers.

Robots Directive Reference

Directive	Meaning	SEO impact
noindex	Do not show this URL in search results.	Blocks organic visibility.
nofollow	Do not follow links on the page.	Can reduce discovery and link signal flow.
none	Equivalent to noindex, nofollow.	Strong index and link block.
noarchive	Do not show a cached copy.	Usually safe, but limits cache display.
nosnippet	Do not show text snippets or previews.	Can lower click-through rate.
max-snippet	Limit snippet text length.	Controls SERP display and AI extraction depth.

After checking directives, confirm real search visibility with the Google Index Checker and review crawl issues in our technical SEO audit guide.

X-Robots-Tag Directives Reference

These are the indexing directives you can send in an X-Robots-Tag HTTP response header. Because the header is delivered with the HTTP response, it works on any file type, including PDFs, images, and other non-HTML resources where a meta robots tag is not possible. Each value below is read by compliant search engines such as Google.

Directive	Meaning	Note
noindex	Do not show this URL in search results.	The main directive for removing a page or file from the index.
nofollow	Do not follow links on this page.	Affects link discovery; does not by itself remove the page from the index.
none	Equivalent to noindex, nofollow.	Shorthand that applies both blocks at once.
noarchive	Do not store or show a cached copy.	Page can still be indexed and ranked; only the cached snapshot is hidden.
nosnippet	Do not show a text or video snippet/preview.	Can reduce click-through because users see less context.
max-snippet:[n]	Limit the snippet to n characters.	Use 0 for no snippet, -1 for no limit (e.g. max-snippet:160).
max-image-preview:[setting]	Limit image preview size in results.	Accepts none, standard, or large.
noimageindex	Do not index images on this page.	The page can still rank; its images will not appear in image search.
unavailable_after:[date]	Stop showing the URL after a given date/time.	Useful for expiring offers or time-limited content.

Reading robots headers correctly is one piece of a clean crawl setup. If indexability problems keep recurring, our SEO services handle technical SEO end to end, from header rules to canonicals and crawl budget.

Next steps

X-Robots-Tag Checker related tools and articles

Continue with the closest follow-up checks and guides based on this tool's topic, crawl intent, and optimization workflow.

Google Index and Cache Checker Tool Online

Google Index & Cache Checker

Robots.txt Tester & Validator

Canonical URL Checker

Technical SEO Audit: What to Check and How to Fix It

How to Check and Test Your Robots.txt File: The Complete Guide

Google Search Console Guide for Business Owners

X-Robots-Tag Checker: FAQ

What directives does the checker inspect?

It reads the X-Robots-Tag response header plus robots and googlebot meta tags in an HTML response. Values are split on commas, normalized to lowercase, and duplicates are removed; none is also treated as noindex and nofollow.

How is the Indexable result calculated?

The combined directive list is labeled non-indexable when it contains noindex or none. This is a narrow directive check and does not account for the HTTP status, robots.txt, canonicalization, authentication, rendering, or search-engine-specific indexing decisions.

Does the result apply equally to every search engine?

Not necessarily. The tool combines generic robots, googlebot meta, and the header into one summary rather than calculating a verdict per crawler. A Google-specific directive may not apply to Bing, and other bot-specific meta tags are not inspected.

Can it interpret user-agent-specific X-Robots-Tag headers?

Only simple comma-separated directives are reliably summarized. Header forms such as googlebot: noindex remain scoped text and are not fully parsed into per-agent rules, so inspect the displayed raw header for advanced configurations.

Why are meta robots tags missing for a page I can see?

Meta tags are read only when the fetched response declares a text/html content type, and the parser expects quoted name and content attributes. Tags injected by JavaScript, unusual markup, consent variants, or bot-specific responses can be missed.

What does nofollow mean in this result?

It asks compliant crawlers not to follow or pass signals through links on the page; it does not prevent the URL itself from being indexed. Use noindex when the intent is to keep that page out of search results.

What should I do after finding an unexpected noindex?

Check the final URL, raw X-Robots-Tag header, page template, CDN or server rules, SEO plugin settings, and deployment environment. Remove the unintended directive, purge relevant caches, and request a fresh crawl after verifying the live response.

Is the checked URL or response stored?

The URL is sent to WebAloha's server for a live request with a 12-second timeout. This endpoint does not save the submitted URL, HTML, headers, directives, or result to a database or result cache.

Free 48-Hour Website Audit

Not sure what to fix first on your own website? We'll review it and tell you, in plain English. Free & non-obligatory.

Get My Free Audit 💪

Need a Full Indexability Audit?

We find crawl blocks, noindex mistakes, canonical conflicts, and technical SEO issues before they cost traffic.

Book an SEO Audit