What is a Robots.txt?

Robots? A little bit confused? What it is, and what it is used for? How can I check my Robots.txt file?


  • It is a file for instructing bots (search engine's bots) to tell them which parts of your website are off the limits. It is crucial for SEO. It uses commands like "disallow" or "sitemap" to guide the bots. You can test your Robots.txt file with Google's robots.txt testing tool. You can also add /robots.txt to the end of your domain to see yours.

Sign In or Register to comment.