Web & SEO Tools

Prepare the small web artifacts that are easy to get wrong and useful to preview before publishing.

No signup for many toolsPrivate where possible2h visitor downloads

Add text2Set details3Result

Step 1

Paste your text

Paste a sitemap.xml URL or website URL, then choose which crawlers can access the site.

Input

Step 2

Create a robots.txt file with sitemap, search crawler, AI crawler, crawl-delay, and blocked-path rules.

Private

Details

Crawler groupsAI accessSitemap URLBlocked pathsAllowed pathsCrawl delay secondsExtra crawler names

Text input1 MB limitrobots.txt

Presets