Robots.txt Generator
Build a valid robots.txt with allow/disallow rules, crawl-delay and a sitemap line. Quick presets to allow or block all robots, or write custom rules, copy or download instantly.
How it works
A robots.txt file sits at the root of your site (example.com/robots.txt)
and tells crawlers which paths they may or may not request. Each block names a
User-agent and lists Disallow and Allow rules.
An empty Disallow: means “crawl everything”; Disallow: / blocks the whole site.
Crawl-delay asks robots to wait between requests (not honoured by Google), and the Sitemap line points crawlers to your sitemap. robots.txt guides well-behaved bots. It is not a security control.
Generated entirely in your browser; nothing is uploaded.
What this does
A robots.txt generator builds a valid robots.txt with allow/disallow rules, crawl-delay, and a sitemap line, with quick presets to allow or block all robots.
How to use it
- Pick a preset or add custom rules.
- Set any crawl-delay.
- Add your sitemap URL.
- Copy or download robots.txt.
How it works
The file sits at your site root and tells crawlers which paths they may request — an empty Disallow means "crawl everything"; Disallow: / blocks the whole site. It only guides well-behaved bots and is public, so never use it to hide sensitive pages.
Example
Disallow: /admin/ keeps crawlers out of your admin path.
Sources & methodology
Last updated .
Frequently asked questions
What is robots.txt for?
It is a file at the root of your site (example.com/robots.txt) that tells crawlers which paths they may or may not request. An empty Disallow means “crawl everything”; Disallow: / blocks the whole site.
Does it improve security?
No. robots.txt only guides well-behaved bots and is publicly visible. Never use it to hide sensitive pages, use proper authentication for that.
How do I add my sitemap?
Enter your sitemap URL and the tool appends a Sitemap: line so crawlers can find it. You can copy the result or download it as robots.txt.
Related tools
Invoice Generator
Create a professional invoice with line items, tax, discounts and shipping, then print or save it as a PDF. Live totals, multiple currencies, everything stays in your browser.
Resume Builder
Build a clean, single-column résumé with live preview: add experience and education, list skills, then print to PDF or copy as Markdown. Private and in-browser.
Meta Tag Generator
Generate SEO meta tags plus Open Graph and Twitter Card tags from your title, description and image. Copy ready-to-paste HTML for your page’s head, built in your browser.
Email Signature Generator
Create a tidy HTML email signature from your name, title, contact details and an accent colour. Live preview, copy straight into Gmail, Outlook or Apple Mail, all in your browser.