AI Lab · Free tool
llms.txt and robots.txt generator for AI crawlers (GPTBot, ClaudeBot, PerplexityBot…)
Decide which AI robots may train their models on your site, cite you in their answers or read your pages on demand. Generate your robots.txt and llms.txt, and check what your current robots.txt really blocks.
🔒 Everything is computed in your browser: nothing is sent to or stored on a server.
| Robot | Company | Role | Rule |
|---|
Sections and links
One line per link: Title | https://… | short description
Why this tool
Three roles, not one
Training, search and citation, user request: every AI company separates these uses. Blocking the wrong robot makes you invisible in answers without protecting your content.
Your robots.txt, decoded
Paste your current file: the tool applies the official precedence rules (RFC 9309) and tells you, robot by robot, what is actually allowed.
A clean llms.txt
The format proposed by llmstxt.org, generated without errors, with an honest take on what it really brings today.
Frequently asked questions
Does blocking GPTBot remove me from ChatGPT?
No. GPTBot is used for training. Appearing in ChatGPT search depends on OAI-SearchBot. You can block one and allow the other: that is the tool's "cited but not trained" setting.
Does Google-Extended affect my Google rankings?
No. According to Google's documentation, this token controls the use of your content for training and grounding Gemini models, but has no effect on inclusion or ranking in Google Search.
Does Google read llms.txt?
Google has said it does not use it for Search. It is an open proposal, mostly useful to the tools and agents that read it. It is cheap to create; do not count on it for rankings.
Do "on-demand" robots respect robots.txt?
Not always. Perplexity and Meta state that user-triggered fetches may ignore robots.txt, and OpenAI notes the rules may not apply to ChatGPT-User. For strict blocking you need to act at the server level.
Going further
SEO for AI answers: getting cited by ChatGPT, Gemini and Perplexity
Read the guide →Play Store listing generator
Open the tool →Coding agent instructions file
Open the tool →Prompt library by profession
Open the tool →Automation ROI calculator
Open the tool →AI API cost calculator
Open the tool →Anonymiser before sending to an AI
Open the tool →Local AI hardware calculator
Open the tool →Business AI assistant designer
Open the tool →Company AI usage policy
Open the tool →Bring AI into your business, for real?
Automations, assistants, multilingual content, apps: I design solutions that are useful, measurable and respectful of your data.