define Spider

Definition:

An automated program that follows links to visit web sites on behalf of search engines or directories. Robots then process and index the code and content of a web page to be stored in the search engine’s database.