The three stages of Google Search
Google says Search works in three stages, and not every page makes it through each one.
- Crawling: Google downloads text, images and videos from pages it found, using automated programs called crawlers.
- Indexing: Google analyses the text, images and video files on the page and stores the information in the Google index, which is a very large database.
- Serving: when someone searches, Google looks in the index for matching pages and shows the results it believes are of the highest quality.
During indexing, Google also groups pages that are the same or very similar and picks one as the main version. See canonical tag.
Not every page gets indexed
Google states that indexing is not guaranteed. Not every page it processes will be indexed. Low quality content, robots rules that say do not index, and some website designs can all stop it.
How to check your pages
Search Console has a Page indexing report. Google describes it as a way to see which pages Google can find and index on your site, and what went wrong with the others. Two common labels are worth knowing.
- Crawled, currently not indexed: Google fetched the page but did not add it. Google says it may or may not be indexed later, and you do not need to ask it to crawl the page again.
- Discovered, currently not indexed: Google found the page but has not crawled it yet. Google says it usually wanted to crawl, but expected that to overload the site.
The noindex rule
A noindex rule tells Google to drop a page from search results. It can be a robots meta tag in the page or an X-Robots-Tag header. Google must be able to crawl the page to see the rule. If robots.txt blocks the page, the crawler never sees the rule, and the page can still show up because other pages link to it. Google does not support a noindex rule written inside robots.txt.
Mobile-first indexing
Google says it uses the mobile version of a site's content, crawled with the smartphone agent, for indexing and ranking. This is called mobile-first indexing. Make sure the main content on your phone pages is the same as on desktop.
A made-up example
A furniture maker has a 40 page website. The Page indexing report shows 12 product pages as crawled but not indexed. The owner opens them and finds the same short text on each, with only the product name changed.
The owner adds real detail to each page: size, wood, finish and delivery time. The category page now links to every product. Google has not promised to index them. But the pages are now worth indexing, and Google can find them easily.
Why it matters for AI answers too
Google says that to be shown as a supporting link in AI Overviews or AI Mode, a page must be indexed and eligible to show with a snippet. Indexing comes first.
Related terms
- Crawl budget: how many pages a search engine will fetch from your site in a given time.
- Robots.txt: a file that tells crawlers which parts of a site not to fetch.
- Sitemap: a file that lists the pages you want search engines to find.
- Canonical tag: a line of code that names the main version of a page.
- Technical SEO (search engine optimisation): the setup work that lets search engines read your site.
To see how Google visits a site, read our Googlebot crawling guide. If pages of yours are stuck, see our SEO work.
Sources
- Google Search Central, "How Google Search works" (developers.google.com/search/docs/fundamentals/how-search-works), read 9 October 2026.
- Google Search Central, "Block Search indexing with noindex" (developers.google.com/search/docs/crawling-indexing/block-indexing), read 9 October 2026.
- Google Search Central, "Mobile-first indexing best practices" (developers.google.com/search/docs/crawling-indexing/mobile/mobile-sites-mobile-first-indexing), read 9 October 2026.
- Search Console Help, "Page indexing report" (support.google.com/webmasters/answer/7440203), read 9 October 2026.
- Google Search Central, "AI features and your website" (developers.google.com/search/docs/appearance/ai-features), read 9 October 2026.
