What is robots.txt?
A file telling search-engine crawlers which parts of a site they may or may not access.
Understanding robots.txt helps you make better decisions about your website. With Situs, the technical side is handled for you, but it's good to know the basics.
Get a premium website without the hassle
Every project is quoted. Tell us what you need and we design a free demo of your site first.