Ready for Agents

ClaudeBot

Anthropic's crawler for content that could contribute to training its models.

Operated byAnthropic
TypeAI training
robots.txt tokenClaudeBot
Official docsAnthropic's crawler documentation

What blocking it does

Blocking ClaudeBot signals that your future content should be excluded from Anthropic's training data. Anthropic says it respects robots.txt, including Crawl-delay.

robots.txt rules

To allow it everywhere:

User-agent: ClaudeBot
Allow: /

To block it everywhere:

User-agent: ClaudeBot
Disallow: /

Put these in the robots.txt file at the root of your site. A group that names ClaudeBot replaces your User-agent: * rules for that crawler, so copy any rules you still want into its group.

Questions

What is ClaudeBot?

Anthropic's crawler for content that could contribute to training its models.

What happens if I block ClaudeBot?

Blocking ClaudeBot signals that your future content should be excluded from Anthropic's training data. Anthropic says it respects robots.txt, including Crawl-delay.

Does allowing ClaudeBot guarantee my business shows up?

No. Allowing a crawler only lets it read your pages. Whether Anthropic's products show or recommend your business depends on many things nobody outside Anthropic can see.

Other Anthropic crawlers

Not affiliated with, endorsed by or sponsored by OpenAI, Anthropic, Google, Apple, Microsoft, Perplexity or Common Crawl. ChatGPT is a trademark of OpenAI. Other names belong to their owners and are used only to identify their crawlers.

Last checked against the vendors' public documentation on November 17, 2026. Crawler names and policies change; follow the source links for the current rules.