Claude-User
Fetches a page when a Claude user asks a question that needs it.
| Operated by | Anthropic |
|---|---|
| Type | User-requested fetch |
| robots.txt token | Claude-User |
| Official docs | Anthropic's crawler documentation |
What blocking it does
Anthropic says blocking it stops Claude from fetching your pages in response to user requests, which may reduce your visibility in user-directed answers.
robots.txt rules
To allow it everywhere:
User-agent: Claude-User
Allow: /To block it everywhere:
User-agent: Claude-User
Disallow: /
Put these in the robots.txt file at the root of your site. A group that names Claude-User replaces your User-agent: * rules for that crawler, so copy any rules you still want into its group.
Questions
What is Claude-User?
Fetches a page when a Claude user asks a question that needs it.
What happens if I block Claude-User?
Anthropic says blocking it stops Claude from fetching your pages in response to user requests, which may reduce your visibility in user-directed answers.
Does allowing Claude-User guarantee my business shows up?
No. Allowing a crawler only lets it read your pages. Whether Anthropic's products show or recommend your business depends on many things nobody outside Anthropic can see.
Other Anthropic crawlers
- Claude-SearchBot (AI search)
- ClaudeBot (AI training)
Not affiliated with, endorsed by or sponsored by OpenAI, Anthropic, Google, Apple, Microsoft, Perplexity or Common Crawl. ChatGPT is a trademark of OpenAI. Other names belong to their owners and are used only to identify their crawlers.
Last checked against the vendors' public documentation on November 17, 2026. Crawler names and policies change; follow the source links for the current rules.