What is robots.txt?
A file telling search-engine crawlers which parts of a site they may or may not access.
Understanding robots.txt helps you make better decisions about your website. With Situs, the technical side is handled for you, but it's good to know the basics.
Get a premium website without the hassle
$100 once per website, $10/month hosting, optional add-ons. We build it, you edit it.