Robots.txt Generator

Create a robots.txt file with allow/disallow rules and sitemap URL.

User-agent

Allow pathsOne per line

Disallow pathsOne per line

Sitemap URL

Crawl delay (sec)

robots.txt

User-agent: *
Allow: /
Disallow: /admin
Disallow: /private
Sitemap: https://thetoolss.com/sitemap.xml

How to use Robots.txt Generator