AI Search Retrieves More URLs Than It Cites

In 1998, Larry Page and Sergey Brin, then at Stanford, published a paper on a large-scale web search engine.

By Career Ahead AI · Global — AI-assisted, fact-verified against cited sources. Readers should verify key facts.

PageRank made links a signal of importance

PageRank used the web’s link structure as a signal for importance. A page’s connections to other pages could therefore contribute to how a search engine judged it, rather than every page being treated as equal. The mechanism matters because it made a feature of the web itself useful for ranking. It offered a way to assess material through its relationships, not just through the words on a page. That is the concrete precedent behind the comparison with AI search: a system needs some basis for deciding which available material deserves to surface.^¹

For enterprise AI, the useful question is what can play a comparable role when a system retrieves information and then answers a user. Retrieval means material has been found; citation means the answer visibly credits some of that material. Those are separate outcomes. A ranking signal and a citation signal need not be the same thing, but both affect what becomes visible. The comparison with PageRank is therefore about the mechanism: can an observable feature help distinguish material that is merely available from material that earns a place in an answer? The available figures put that distinction in sharp relief.^¹

That is the concrete precedent behind the comparison with AI search: a system needs some basis for deciding which available material deserves to surface.¹

ChatGPT retrieves many URLs it does not cite

An Ahrefs study examined 1.4 million ChatGPT 5.2 prompts and tens of millions of URL retrieval and citation outcomes. It reported roughly 23.4 million cited URLs, but those citations represented only 49.98% of the URLs ChatGPT retrieved. In other words, finding a URL did not ensure that it appeared as a visible citation. The gap gives the comparison a practical edge: retrieval counts alone describe what a system accessed, while citations show which material it chose to expose in its answer. The two measurements capture different steps.^¹²

You may also like

News

Gen Z Opens Retirement Accounts Earlier Than Past Generations

Gen Z households with retirement accounts outnumber Gen X households at the same age by a factor of three.

Read More →

Reddit shows why the distinction matters. In the study, the platform had a large data footprint but a relatively low citation rate. A large presence in the material a system encounters does not, by itself, establish that the system often credits that material. The observation does not explain why Reddit’s citation rate was relatively low, and it does not identify a single rule that would raise it. It does show that footprint and citation rate describe different things. Any proposed AI equivalent of a ranking signal must be judged against the outcome it is meant to explain.^¹

Search access and distribution are separate layers

OpenAI’s crawler documentation distinguishes OAI-SearchBot, used for search functionality, from GPTBot, used for training data collection. That distinction separates two forms of access rather than treating all crawling as one event. For anyone trying to understand why material becomes visible in an answer, the difference matters: a crawler’s purpose is not itself a citation outcome. The documentation identifies separate functions, while the Ahrefs figures distinguish retrieval from visible citation. Together, they show why a single measure of access cannot stand in for every step between material and its appearance in an answer.^¹

The model field also includes different systems and modes. GPT-5, Claude 4.1, Gemini 2.5 and Grok 4 all shipped in mid-2025; Grok 4 Heavy is an example of a multi-agent variant. Those details matter to the PageRank comparison because the systems being considered are not one uniform search engine with one stated route to an answer. The evidence identifies multiple models and a multi-agent example, but does not establish that they use the same retrieval or citation rules. A useful signal would need to be assessed against the particular step it is supposed to clarify.^¹²

Distribution adds another layer. ChatGPT has reached 700 million users, while Microsoft has enterprise channels and a partnership with OpenAI. Those facts describe reach and a commercial relationship; they do not measure which URLs receive citations. Keeping the categories separate helps clarify what a PageRank-style moment would involve. Distribution can put a tool in front of users, and a crawler can access material, but neither fact alone shows what the system visibly credits in an answer. The unresolved measurement is the link between what an AI system retrieves and what it selects for users to see.^¹²

Track citations, not just retrieval

You may also like

News

Fed’s Logan Says More Hikes Needed, But Bond Moves Could Help

The Federal Reserve's commitment to further interest rate hikes aims to combat inflation, significantly impacting bond markets and investment strategies.

Read More →

When you evaluate an AI tool for your work, ask what it visibly credits, not just what it accesses. The Ahrefs study shows ChatGPT cited only 49.98% of the URLs it retrieved, so a system can pull material without surfacing it. For a role that depends on sourcing or verification, check whether a tool's citations match its retrieval. A large footprint, like Reddit's, does not guarantee a high citation rate, so judge a tool by the citations it actually shows.^¹

A useful signal would need to be assessed against the particular step it is supposed to clarify.¹²

The distinction between search and training access matters for your own content strategy. OpenAI's crawler documentation separates OAI-SearchBot from GPTBot, meaning your material can be used for training without appearing in search answers. If you create content, track whether it gets cited in AI responses, not just whether it is crawled. With ChatGPT at 700 million users, the visibility of your work in AI answers affects your reach, so plan for the gap between retrieval and citation.^¹²