Custom robot.txt generate free

Robots.txt Generator

Website Name (Optional):

Allow Paths (Comma-separated):

Defaults to empty.

Disallow Paths (Comma-separated):

Sitemap URL (Optional):

Crawl Delay (Optional, in seconds):

Notes

For a Post: If you’re generating a robots.txt for a specific post, include its path in “Allow Paths” (e.g., /post/my-article) and block sensitive areas (e.g., /drafts).

Testing: Test the generated file with tools like Google Search Console’s robots.txt Tester.

Deployment: Host the HTML file on a web server or use a service like GitHub Pages for online access.