My tests still hold that the NioFile I submitted is significantly faster than the standard FSDirectory.
BUT, the memory mapped implementation is significantly faster than NioFile. I attribute this to the overhead of managing the soft references, and possible GC interaction. SO, I would like to use a memory mapped reader, but I encounter OOM errors when mapping large files, due to running out of address space. Has anyone found a solution for this? (A 2 gig index is not all that large...). -----Original Message----- From: Murat Yakici [mailto:[EMAIL PROTECTED] Sent: Tuesday, May 16, 2006 1:55 AM To: java-dev@lucene.apache.org Subject: Re: Nio File Caching & Performance Test Hi, According to my humble tests, there is no significant improvement either. NIO has buffer creation time costs compared to other Buffered IOs. However, a testbed would be ideal for benchmarks. Murat Doug Cutting wrote: > Robert Engels wrote: > >> The most important statistic is that the reading via the local cache, vs. >> going to the OS (where the block is cached) is 3x faster (22344 vs. >> 68578). >> With random reads, when the block may not be in the OS cache, it is 8x >> faster (72766 vs. 586391). > > [ ... ] > >> This test only demonstrates improvements in the low-level IO layer, >> but one >> could infer significant performance improvements for common searches >> and/or >> document retrievals. > > > That is not an inference I would make. There should be some > improvement, but whether it is significant is not clear to me. > >> Is there a standard Lucene search performance I could run both with and >> without the NioFSDirectory to demonstrate real world performance >> improvements? I have some internal tests that I am collating, but I would >> rather use a standard test if possible. > > > No, we don't have a standard benchmark suite. Folks have talked about > developing one, but I don't think one yet exists. > > Report what you have. Describe the collection, how it is indexed, how > you've selected queries, and the improvement in average response time. > > Doug > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
