Blog: robots.txt
CHECKLIST for eCommerce Website #2: Robots.txt (Video Tutorial)
Not long ago we have launched a new category on our blog — CHECKLIST of the most common mistakes that you should take into consideration when launching / while running your online store.
Our CHECKLIST #1 related to HTTPS issues you can find here.
CHECKLIST #2 is devoted to Robots.txt problem points. In our previous post — Robots.txt Guide, we explain how essential is to understand the importance of robots.txt file and learn how to check whether you’re using it correctly.
Now, we are pleased to share our new step-by-step video tutorial series on the topic.
To Crawl or Not to Crawl: A Short Guide to Robots.txt
Magento is a leading and one of the most popular eCommerce platforms used by businesses from all over the world. Of course, for our readers, that is no news. It gained such popularity thanks to the variety of features available out of the box.
However, when it comes to Magento SEO and eCommerce, we see that so powerful solution doesn’t have a robots.txt file by default. Thus, the extremely common question is how a robots.txt file should look and what should be in it.
Newsletter