What is a Web Spider?

A web spider is a program running on a computer connected to the Internet that automatically browses websites and follows all of the links on it. It can catalog the information, for example, to index sites for a search engine.