Robots.txt Generator

Build a robots.txt to control search crawlers.

Save the output as robots.txt in your site's root folder (e.g. example.com/robots.txt).

How to use the Robots.txt Generator

Simple, free and instant - no sign-up required.

1

Fill in your details

Enter your page info in the fields - everything is generated in your browser, nothing is uploaded.

2

Adjust the options

Tweak the settings to generate exactly the markup or URL you need.

3

Get the code instantly

Your tags, code or URL build live as you type - no waiting and no processing queue.

4

Copy & paste into your site

Copy the output or download the file, then drop it into your page head, server config or CMS.

Advertisement · 970×90

Frequently asked questions

Everything you need to know about Robots.txt Generator.

robots.txt tells search-engine crawlers which parts of your site they may or may not crawl. This tool builds one with your allow/disallow rules and an optional sitemap reference.
In your site's root directory, so it's reachable at yourdomain.com/robots.txt. Download the file from this tool and upload it there.
No — it only stops crawling, not indexing. To keep a page out of search results, use a noindex meta tag instead (and don't block it in robots.txt, or Google can't see the noindex).

Related tools

More handy tools for your workflow.

Meta Tag Generator

Generate title, description & meta tags for SEO.

Open tool

Open Graph Generator

Create Open Graph & Twitter social preview tags.

Open tool

Sitemap Generator (XML)

Generate an XML sitemap from your URLs.

Open tool

Slug / URL Generator

Turn any text into a clean, SEO-friendly URL slug.

Open tool
View all tools
Robots.txt Generator | Free Online SEO & Web Tool - EveryFileTool