AI search guide
How AI assistants find and cite websites: the five compared
Each AI assistant searches the web through its own route: ChatGPT through its search crawler and partner search providers, Claude through its own search crawler, Perplexity through PerplexityBot, Copilot through the Bing index, and Gemini through Google Search. None publishes how it ranks sources. What you control is the same everywhere: let the right crawler in, be indexed where that assistant searches, and publish clear, consistent, verifiable pages.
The five at a glance
| Assistant | Where its web results come from | Crawler to allow | Documented control |
|---|---|---|---|
| ChatGPT | OpenAI’s search index plus partner search providers | OAI-SearchBot (search); ChatGPT-User (user requests) | robots.txt per bot; blocking GPTBot (training) is separate |
| Claude | Anthropic’s web search, with citations in every answer | Claude-SearchBot (search); Claude-User (user requests) | robots.txt per bot; ClaudeBot (training) is separate |
| Perplexity | Perplexity’s own index | PerplexityBot; Perplexity-User for live fetches | robots.txt for PerplexityBot |
| Microsoft Copilot | The Bing index | Bingbot (no separate Copilot crawler) | Bing Webmaster Tools; robots.txt for Bingbot |
| Google Gemini app | Google Search, used to ground answers | Googlebot to be indexed | Google-Extended governs grounding in the Gemini app and training |
Checked against the company’s own documentation on 16 November 2026. Ranking methods are not published by any of these companies; anything beyond the documented controls is informed estimation, not fact.
What works on all five
- Allow the search crawlers in robots.txt and make sure your firewall or CDN does not block them.
- Be indexed in both Google and Bing; submit sitemaps to both.
- Answer each buyer question directly near the top of a page, then give detail and evidence.
- Keep your name, services, prices and contact details identical everywhere you control.
- Earn mentions on sites these assistants already trust in your field; do not buy them.
- Measure with a fixed question set, on each assistant, on dated runs.
What nobody can tell you
How each assistant weighs one eligible page against another is not public. Anyone selling a guaranteed citation, or a single “AI visibility score” without the questions and dates behind it, is selling something they cannot control.
Related services
Sources
Primary references this guide relies on. Provider documentation changes, so check the source before acting.
Questions
The foundations are the same. The differences are which crawler to allow and which index matters: Bing for Copilot, Google for Gemini, each company’s own crawler for ChatGPT, Claude and Perplexity.
Ready to talk through your project?
Send a voice note, spreadsheet or a plain description of where your business is struggling. We’ll help you find the right next step.

