Is a priority page marked noindex?

check 0.5Crawler accessa person does it

What the check looks at

The check reads each priority page's HTML for <meta name="robots" content="noindex"> and the response headers for X-Robots-Tag: noindex. Either one excludes the page from search indexes — and from the material retrieval systems draw on.

Why it matters

noindex is usually a leftover: a staging flag that shipped, a plugin setting, an SEO plugin's "discourage search engines" box. On a priority page it quietly removes the page from Google and from the indexes AI search engines retrieve from.

How to see it on your own site

View the page source and search for "noindex"; check the response headers for X-Robots-Tag. On WordPress, Settings → Reading → "Discourage search engines" sets it for the whole site.

How to fix it

Remove the meta tag or the header from pages you want found.
On WordPress, untick "Discourage search engines from indexing this site" and check the SEO plugin's per-page setting.
Keep noindex only where it belongs: thank-you pages, internal search results, duplicates.

What the agent does: the agent never flips noindex on its own — it names the page and the exact tag or setting, and you decide.

Related checks

Part of the 23 checks · group: Crawler access — robots.txt, CDN/bot filter, login walls, noindex, HTTP status, sitemap, canonical.

Questions people ask

Is noindex the same as blocking in robots.txt?

No. robots.txt stops the fetch; noindex lets the fetch happen and asks the engine not to index the result. Both remove the page from what AI retrieves.

Why does the agent not remove it itself?

Because noindex is sometimes intentional. A switch that could expose a page you meant to hide is not a switch worth having; the agent proposes, you decide.