I like architectural ideas behind Apache MINA (inspired by SEDA): for some (CPU-intensive) processing (such as parsing of content) we need single thread per single CPU core, for others (I/O bound) much more threads (waiting for response from network socket). It’s not just NIO...
-Fuad From: Lukáš Vlček [mailto:lukas.vl...@gmail.com] Sent: October-18-09 7:12 AM To: nutch-dev@lucene.apache.org; droids-...@incubator.apache.org Subject: Niocchi - java asynchronous crawl library released Hi, I just noticed that Niocchi has been released recently. http://www.niocchi.com/ Niocchi is a java asynchronous crawl library implemented with NIO. It is designed to crawl several thousands of hosts in parallel on a single low end server.It is currently being used in production by Enormo <http://www.enormo.com/> to crawl thousands of websites daily, and by Vitalprix <http://www.vitalprix.com/> . Regards, Lukas
