How it works
A focused check,
not a mystery score.
01
Choose global disallow paths and add your sitemap.
02
Set optional rules for popular AI crawler tokens.
03
Copy the generated file and publish it at the root of your domain.
FAQ
Before you use it.
Can robots.txt remove a page from Google?+
Not reliably. Blocking crawling does not necessarily remove an already known URL. Use an indexable noindex directive or an appropriate removal workflow instead.
Where should robots.txt be uploaded?+
Publish it at the root of the exact host, for example https://example.com/robots.txt.