Robots.txt Configuration

🤖 AI & LLM Bots

💡 Allowing AI bots enables your content to be used for training models and appear in AI responses

🔍 Search Engines

Helps search engines find all your pages
Delay between each bot request (0 = no delay)
Paths that bots should not explore
Useful for allowing subfolders within a blocked folder

⚙️ Custom Rules

Generated robots.txt File

# Robots.txt generated by BeBranded
# Date: ${new Date().toLocaleDateString('en-US')}

User-agent: *
Allow: /
⚠️ Place this file at the root of your site: yoursite.com/robots.txt
📝 In Webflow: Project Settings → SEO → Robots.txt → Paste the generated code
✓ Code copied to clipboard!