AI discovery

AI crawlers, search bots, and training agents.

A compact reference of crawler identities that matter for public SaaS docs and programmatic SEO pages — plus how ReelsRadar helps agents discover the product.

llms.txtDocsOpenAPI

AI answers

User-triggered fetches when an assistant opens a page to answer a person.

OpenAI

ChatGPT-User

User-triggered fetches when ChatGPT opens a page.

Anthropic

Claude-User

User-triggered fetches when Claude opens a page.

Perplexity

Perplexity-User

User-triggered fetches for answers and citations.

Google

Google-Agent

User-triggered fetcher for AI product experiences.

Microsoft

Copilot

User-triggered Copilot fetches for AI answers.

DuckDuckGo

DuckAssistBot

Real-time crawler for AI-assisted answers.

Search indexes

Crawlers that discover and refresh content for search / answer indexes.

Google

Googlebot

Primary Search indexing crawler.

OpenAI

OAI-SearchBot

Surfaces sites in ChatGPT search results.

Anthropic

Claude-SearchBot

Discovers content for Claude search.

Perplexity

PerplexityBot

Keeps Perplexity's index fresh.

Microsoft

Bingbot

Bing Search indexing crawler.

Training crawlers

Public-content crawlers associated with model training datasets.

OpenAI

GPTBot

May collect public content for future models.

Anthropic

ClaudeBot

Public content crawler for Claude.

Google

Google-Extended

Controls Gemini training/grounding usage.

Common Crawl

CCBot

Builds open web crawl datasets.

ByteDance

Bytespider

Public-content crawler.

Make your product agent-readable

ReelsRadar publishes llms.txt, markdown docs twins, OpenAPI, and programmatic SEO landings so answer engines can cite accurate product facts — not just crawl a homepage.

Read the agent docs