Noindex

What is noindex

Noindex is a directive that tells search engines not to include a page in their index, which means the page will not appear in search results. It is most often delivered through a robots meta tag in the head of the page or through an X-Robots-Tag in the HTTP header. Noindex does not stop a page from being crawled, it stops it from being listed. This makes it the correct tool when a page needs to exist and be reachable for users, but should never surface in organic search.

How to implement noindex

The simplest method is a robots meta tag with the value noindex placed in the head of the HTML. To also prevent the page's links from being followed, you can combine it as noindex, nofollow, though noindex, follow is often preferable so that link equity still flows through the page. For non HTML resources such as PDFs, the X-Robots-Tag HTTP header applies the same directive. Crucially, the search engine must be able to crawl the page to see the noindex tag, so the page must not be blocked in robots.txt.

Noindex versus robots.txt and canonical

These are often confused. Robots.txt blocks crawling but does not reliably remove a page from the index, and a blocked page can still appear as a bare URL if it is linked elsewhere. Noindex reliably removes a page from results but requires the page to be crawlable. A canonical consolidates duplicates onto a preferred URL while keeping them all indexable in principle. The rule of thumb: use noindex to keep a page out of search, a canonical to choose between duplicates, and robots.txt to save crawl budget on URL spaces you never want fetched.

When to use noindex

Good candidates for noindex include thank you and confirmation pages, internal search results, thin tag or filter pages, staging or duplicate content, login and account pages, and low value archives. These pages serve a purpose for users but would clutter the index or compete with your real content if listed. Noindex keeps them functional while ensuring only your valuable, intentional pages represent you in search results.

Common noindex mistakes

The most damaging mistake is leaving a site wide noindex in place after a launch or migration, which can quietly deindex an entire site. Another is blocking a noindexed page in robots.txt, which prevents the crawler from ever seeing the noindex, so the page may linger in the index. Applying noindex to important, revenue driving pages by accident is equally costly. Regular audits with Ahrefs, Semrush, or a search console are essential to confirm that only the intended pages carry the directive.

Noindex, crawl budget and GEO

Used well, noindex keeps the index focused on your best pages, which indirectly supports crawl efficiency and the clarity of your site in the eyes of search engines. The same clarity benefits generative engines: keeping thin and utility pages out of the index reduces the chance that an answer engine surfaces or cites a low value page instead of your authoritative content. Deliberate use of noindex is part of a clean technical foundation for both classic SEO and generative engine optimisation.