Indexing
The process by which search engines organize and store information from the web to display it in search results.
Detailed Explanation
A page that isn't indexed doesn't exist to search engine users. Indexing follows crawling. If a page has a 'noindex' tag, or if the content is low quality/duplicate, Google may choose not to index it. Developers can monitor indexing status using Google Search Console to ensure all tools are visible to the public.
Quick Summary
Indexing is the step *after* crawling: Google decides whether to store and serve a page in search results. Just because Googlebot fetched a page doesn't mean it'll appear in search, Google indexes maybe 60-80% of crawled pages, filtering low-quality, duplicate, or `noindex`-tagged content.
Key Takeaways
- Crawled ≠ indexed, Google can crawl but choose not to index.
- `<meta name="robots" content="noindex">` explicitly excludes a page.
- Duplicate content (without canonical) may be deindexed in favor of the 'original'.
- Thin/low-quality content increasingly excluded since Helpful Content Update.
- Verify via `site:yourdomain.com/page` or Search Console URL Inspection.
When to use it
- Diagnosing why a page doesn't appear in search (Search Console reveals exact reason).
- Strategically noindexing low-value pages (tag archives, search results) to focus crawl budget.
- Tracking indexation coverage as a leading indicator of SEO health.
- Detecting accidental noindex from CMS settings or staging deployments.
Common Mistakes
- Leaving `noindex` on a staging site that gets pushed to production.
- Assuming sitemap submission guarantees indexing, it doesn't.
- Massive thin content (auto-generated, AI-spammed) gets deindexed en masse.
- Confusing `noindex` (won't index) with `Disallow` in robots.txt (won't crawl), they solve different problems.
Indexing, Frequently Asked
How long does indexing take?
Hours for established sites with strong internal linking; days to weeks for new sites or weakly-linked pages.
How do I force-index a page?
Use 'Request Indexing' in Search Console URL Inspection. Not a guarantee but speeds things up.
Why was my page deindexed?
Common causes: thin content, duplicate of another URL, manual penalty, technical noindex tag, or robots.txt block. Check Search Console for the exact reason.