What is a spider?

A spider is a software application that browses web pages for the purpose of indexing content and providing the results to a search engine. (Also called search spider, search robots, or just bots.)