Crawler — definition

A bot that browses the web to discover and read pages for a search engine.