What is robots.txt?
A file telling search-engine crawlers which parts of a site they may or may not access.
Understanding robots.txt helps you make better decisions about your website. With Situs, the technical side is handled for you — but it's good to know the basics.
Get a premium website without the hassle
$100 once per website, $10/month hosting, optional add-ons. We build it, you edit it.