>From what I have noticed , implementing your own buffer manager, disk cache or using NIO or whatever you want to call it, does improve performance in "some" situations. It avoids using the Operating System Disk Cache which has some overheads. In the database world it can sometimes yield higher than 30% performance gains - assuming your index'es are much larger than available ram. This would make sense for a smaller implementation of nutch , say, for intranet/enterprise search rather than internet search.
Anyway, someone posted that a distributed filesystem would be better and I guess that's the reason google uses its own distributed filesystem. Here is a paper that I'm sure someone else posted here at some point. http://www.cs.rochester.edu/sosp2003/papers/p125-ghemawat.pdf -- Ab's --- Byron Miller <[EMAIL PROTECTED]> wrote: > Has anyone done any testing of converting > read/writes > & io to NIO or do you think that wouldn't offer much > gain? > > > ------------------------------------------------------- > This SF.Net email is sponsored by The 2004 > JavaOne(SM) Conference > Learn from the experts at JavaOne(SM), Sun's > Worldwide Java Developer > Conference, June 28 - July 1 at the Moscone Center > in San Francisco, CA > REGISTER AND SAVE! http://java.sun.com/javaone/sf > Priority Code NWMGYKND > _______________________________________________ > Nutch-developers mailing list > [EMAIL PROTECTED] > https://lists.sourceforge.net/lists/listinfo/nutch-developers > ------------------------------------------------------- This SF.Net email is sponsored by The 2004 JavaOne(SM) Conference Learn from the experts at JavaOne(SM), Sun's Worldwide Java Developer Conference, June 28 - July 1 at the Moscone Center in San Francisco, CA REGISTER AND SAVE! http://java.sun.com/javaone/sf Priority Code NWMGYKND _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers
