Web Crawler

Stim · Post by **Stim** » Thu 25. Jan 2007, 10:37

Anyone know if there is some software that can collects documents from the Web, like google, yahoo msn etc use? I would like to use some kind "search stuffs" that yahoo etc using but don’t know what software to looking for?

webwesen · Post by **webwesen** » Mon 29. Jan 2007, 21:39

smth to check out:
http://www-128.ibm.com/developerworks/l ... piderLinux

DeXXus · Post by **DeXXus** » Tue 30. Jan 2007, 04:47

http://www.google.com/search?num=50&hl= ... tnG=Search
http://www.google.com/search?hl=en&q=%2 ... scraper%22
http://www.google.com/search?hl=en&q=%2 ... +spider%22
http://www.google.com/search?num=50&hl= ... +spider%22
http://www.google.com/search?num=50&hl= ... crawler%22
http://www.google.com/search?num=50&hl= ... the+web%22