A web page can be publicly accessible and still be absent from a search engine’s index.
That distinction sounds technical until somebody tries to use search results as a census of the internet.
Search engines do not begin with every page on the web and then decide how to rank them. They first have to discover URLs, crawl them, process what they find, decide which versions are duplicates, and determine whether a page belongs in the index at all. Only after that can an indexed page compete for a search result.
Google’s own documentation is explicit about the limits. Its guide to how Search works says Googlebot does not crawl every page it discovers and that indexing is not guaranteed. Search Console’s Page indexing report documentation goes even further: Google does not guarantee that all pages everywhere will make it into its index.
Accessible is not the same as indexed
A page can return a perfectly ordinary 200 OK response in a browser and remain outside the index for several reasons.
The crawler may not know the URL exists. The page may be weakly linked. It may duplicate another page. A site may accidentally block crawling or indexing. Google may crawl the page and still decide not to index it. A URL may simply be waiting in the discovery queue.
Search Console even distinguishes between Discovered – currently not indexed and Crawled – currently not indexed. In the first case Google knows the URL but has not fetched it yet. In the second, it fetched the page but did not add it to the searchable index.
Neither state means the page is unavailable on the web.
Index size is not web size
This matters for Dead Internet Theory because search engines are often used as informal measuring instruments.
If a search for some obscure subject produces only twelve useful pages, several explanations are possible. Perhaps only twelve useful pages exist. Perhaps more exist but are poorly linked. Perhaps some are not indexed. Perhaps the search engine clustered similar material, ranked other pages higher, or failed to interpret the terminology used by a small community.
A result count therefore measures something closer to what a particular search system currently exposes for a query than the total amount of relevant material online.
Even Search Console’s site-level numbers only describe URLs Google knows about. A completely unknown URL is missing from both the indexed and non-indexed totals.
That is why estimating the size of the web from search indexes is slippery. The crawler’s frontier, the index, and the publicly accessible web are three overlapping but different things.
A search engine can be enormous without being complete.
The map can contain billions of roads and still leave towns off it.