How answer engines choose which sources to cite

What does Google say?

A page can be a supporting link in AI Overviews or AI Mode when it is indexed and eligible to be shown in Google Search with a snippet, and Google states that there are no additional technical requirements. It also states that meeting every requirement does not mean the page will be crawled, indexed or served. Both features may issue several related searches across subtopics and data sources, which Google says lets it show a broader and more varied group of links than a classic search, and AI Overviews appear only when Google's systems judge them additive to classic Search. No new machine-readable file, AI text file or special schema.org markup is needed. (Source: Google, AI features and your website (updated 10 December 2025), as of 20 October 2026.)

Crawling for Search is controlled with robots.txt rules for Googlebot, and what is shown from a page with nosnippet, data-nosnippet, max-snippet or noindex. Google-Extended is a separate token that controls use of crawled content for training future Gemini models and for grounding answers in Gemini Apps and in Vertex AI's search grounding; Google says it does not affect inclusion in Google Search and is not a ranking signal. Traffic from AI features is reported in the Performance report of Search Console, within the Web search type. (Sources: Google, AI features and your website (updated 10 December 2025), as of 20 October 2026; Google's common crawlers (last updated 15 October 2026), as of 20 October 2026.)

What does OpenAI say?

OpenAI documents two crawler settings that are independent of each other: OAI-SearchBot, which surfaces sites in ChatGPT's search features, and GPTBot, which crawls content that may be used to train its models. For a site that opts out of OAI-SearchBot, OpenAI says ChatGPT search answers will not show it, though it can still appear as a navigational link; if both are allowed, OpenAI may use one crawl for both purposes; and changes to robots.txt can take about a day to take effect for search. ChatGPT-User acts on a person's request, robots.txt rules may not apply to it, and it is not used to decide whether content appears in Search. (Source: OpenAI, Overview of OpenAI crawlers (no date shown), as of 20 October 2026.)

OpenAI's help page says ChatGPT ranks search results using multiple factors intended to help people find relevant, reliable information, and that placement is not assured. To be eligible a site must allow OAI-SearchBot and allow traffic from OpenAI's published IP addresses. The page also says that ChatGPT search sometimes partners with other search providers, that it may rewrite a question into several queries, and that results and citations can be incomplete, outdated or incorrect. It does not list the factors. (Source: OpenAI Help Center, Searching the web with ChatGPT (page showed an update on the day it was read), as of 20 October 2026.)

What does Anthropic say?

Anthropic documents three robots. ClaudeBot collects content that could contribute to training. Claude-SearchBot is the search robot: disabling it stops indexing for search, which Anthropic says may reduce a site's visibility in what users see when they search. Claude-User retrieves pages when a person asks a question, and disabling it may reduce visibility for user-directed web search. Anthropic says its robots honor robots.txt directives and support the Crawl-delay extension, and that blocking by IP address may not work as an opt-out. (Source: Anthropic Help Center, Does Anthropic crawl data from the web (dated April 8, 2026), as of 20 October 2026.)

Anthropic's help page on web search says that Claude invokes a search tool to ground its answers in content from the live web and that every response includes citations. It does not say how the sources behind a citation are chosen. (Source: Anthropic Help Center, Enable and use web search (shown as updated over a week before it was read), as of 20 October 2026.)

What does Perplexity say?

Perplexity documents PerplexityBot, which is designed to surface and link websites in its search results and is not used to crawl content for AI foundation models, and Perplexity-User, which supports user actions: when a person asks a question it might visit a page and include a link to it in the answer. Because a person requested the fetch, Perplexity says this fetcher generally ignores robots.txt rules. Changes can take up to a day to be reflected. (Source: Perplexity, Perplexity Crawlers (no date shown), as of 20 October 2026.)

Perplexity's help page says each answer includes numbered citations linking to the original sources, and describes the process in general terms: it searches the web, gathers information from sources it calls authoritative and compiles the most relevant insights. It gives no criteria for what it treats as authoritative or relevant. (Source: Perplexity Help Center, How does Perplexity work (last modified August 28, 2026), as of 20 October 2026.)

What do the providers leave unsaid?

In the pages read, none of the four providers gives criteria, weights or signals for choosing one eligible page over another. Google comes closest, stating an aim of a broader and more varied group of links. OpenAI says only that multiple factors are used, and Anthropic and Perplexity describe a process without criteria. The documents state who can be used and what a site owner can control; they do not state how a source is selected, and they do not describe how any provider decides which of several suitable pages to cite. Exhibit 2 sets this out. (Sources: Google, AI features and your website (updated 10 December 2025); OpenAI Help Center, Searching the web with ChatGPT (page showed an update on the day it was read); Anthropic Help Center, Enable and use web search (shown as updated over a week before it was read); Perplexity Help Center, How does Perplexity work (last modified August 28, 2026); each as of 20 October 2026.)

What does this mean for a firm that wants to be cited?

Our reading of the documents is that two things can be acted on. A page that cannot be crawled, or is not eligible, cannot be used, and the conditions for both are published, so the cheapest work is access: read robots.txt for each crawler name in Exhibit 1, check that the host or content delivery network allows the providers' published IP addresses, and check indexing and snippets for key pages. Beyond access, the providers publish no rule a firm can follow to be chosen, so advice that claims to know the rule is not supported by these documents. (Sources: Google, AI features and your website (updated 10 December 2025); OpenAI, Overview of OpenAI crawlers (no date shown); Perplexity, Perplexity Crawlers (no date shown); each as of 20 October 2026.)

A firm can still write clear pages that answer a buyer's question, describe itself consistently, and test its visibility by asking the questions, and it should treat the result as an observation, not a ranking, because OpenAI's own help page says citations can be incomplete, outdated or incorrect. (Source: OpenAI Help Center, Searching the web with ChatGPT (page showed an update on the day it was read), as of 20 October 2026.)

What should you watch for?

The documents change without notice. As read, Google's AI features page was last updated on 10 December 2025 and its crawler page on 15 October 2026, Anthropic's article is dated April 8, 2026, and Perplexity's help page was last modified on August 28, 2026. OpenAI's and Perplexity's crawler pages show no date. The address of Google's crawler page and of Perplexity's crawler page each redirected to a new address when opened, so links to them may move. Watch also for changes to the products that the documents do not describe, and for changes to what Search Console or other reports show about AI features. (Sources: Google, AI features and your website (updated 10 December 2025); Google's common crawlers (last updated 15 October 2026); Anthropic Help Center, Does Anthropic crawl data from the web (dated April 8, 2026); Perplexity Help Center, How does Perplexity work (last modified August 28, 2026); Perplexity, Perplexity Crawlers (no date shown); OpenAI, Overview of OpenAI crawlers (no date shown); each as of 20 October 2026.)