Web crawler

A Web crawler is a computer program that browses the world wide web in a methodical, automated manner. Other terms for Web crawlers are ants, automatic indexers, bots, and worms or Web spider or Web robot.