How to build a basic web-page-grabber in java
"Website Grabber is a utility that can be used to mirror, copy, synchronize, download, scrub or "steal" a web site." - from Epicware at http://www.epicware.com/webgrabber.html ( Important Warning - Web Grabber should be built with a lot of caution as it might cause problem to server and if not controlled to the web, in scenarios like crawler crawling all the web, following all the links on the web, so that might be the reason to not having so many Web-Grabber Application on internet. Because people might use them for stealing the website designs or stylesheets or framework, etc.) My purpose of publishing this blog is to make people aware, to make them literate about how do they (Grabbers) work and how they can be built, but using them wisely is an individual responsibility. Last semester, we had a project to build a fully functional web-crawler just like Googlebot or Yahoo! Slurp or msnbot or Bingbot, etc (right, not that powerful). And I knew previous...