robots.txt tester for AI crawlers
Enter a domain or paste a robots.txt file. The tester shows, for each AI crawler the scanner checks, whether a path is allowed or blocked and which rule decides it.
Full scan
See the full AI readiness picture
robots.txt is one of 70 checks. The full scan also tests llms.txt, structured data, agent protocols and security headers.