Robert Engels wrote:
2. I agree that creating NioFSDirectory rather than modifying FSDirectory. I originally felt the memory mapped files would be the fastest, but it also requires OS calls, the "caching" code is CONSIDERABLY faster, since it does not need to do any JNI, or make OS calls.
On the contrary, mmap, if implemented well, should require fewer system calls and fewer buffer copies. However, in prior, single-threaded benchmarks, FSDirectory appeared to still be a bit faster than MmapDirectory, as have other nio-based directories that folks have tried. So nio's implementation may not have been optimal, although perhaps it is improving. But in multi-threaded benchmarks like yours, an nio-based Directory should shine.
As for creating a CachingDirectory that can cache any directory that should be feasible as well, but I am not sure it would perform as well as the direct internal cache version.
I would be surprised if it is measurably slower. It would add only a single additional method call in the case of cache misses.
If separate, this could be layered on: FSDirectory for best single-threaded performance; on an MMapDirectory for best multi-threaded performance on 64-bit machines or 32-bit machines with indexes less than a few hundred megabytes; or on an NioDirectory for best multi-threaded performance on a 32-bit machine with a large index.
Doug --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
