SAYF & CO.TOOLS
Free audit

Robots.txt Checker

Read a site's robots.txt and test whether a path is allowed.

Free Live server check No sign-up

About the Robots.txt Checker

Load any domain's robots.txt, see its rules grouped by user-agent, and test whether a given path is allowed or blocked for a crawler.

How to use it

  1. Enter the domain.
  2. Click Fetch robots.txt.
  3. Test important paths to make sure they are not blocked by mistake.
  • Free, no sign-up
  • Live results from our server
  • Works on mobile and desktop

Frequently asked questions

Is the Robots.txt Checker free?

Yes. The Robots.txt Checker is completely free to use, with no sign-up or usage limits beyond fair-use protection.

Is my data private?

This tool fetches only the public URL or domain you enter, from our server. Results are not stored.

Sayf & Co.

Tools find the problems. We fix them.

Get a free audit of your website, search presence and social channels — with a clear, prioritised plan to grow.

Get your free audit