Configuration Settings

User Agents

Crawling Directives

Disallow all
Block all search engines from crawling your site
Allow all
Allow all search engines to crawl your entire site
Crawl delay
Set delay between crawls (in seconds)

Path Restrictions

Sitemap URL

Generated Robots.txt

# robots.txt generated by Robots.txt Generator # Created: 2024-01-15 User-agent: * Allow: / User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / Sitemap: https://example.com/sitemap.xml