Thanks, Scott
Francesco Bellomi wrote:
Hi,
I developed a Directory implementation that accesses an index stored on the filesystem using memory-mapped files (as provided by the NIO API, introduced in Java 1.4).
You can download the complied jar and the source from here: www.fran.it/lucene-NIO.zip
Basically there are 3 new classes: NIODirectory, NIOInputStream and NIOOutputStream. They are heavily based on FSDirectory, FSInputStream and FSOutputStream.
NIOInputStram provides memory-mapped access to files. It does not rely on Lucene InputStream's caching feature, since direct access to the memory-mapped file should be faster. Also, cloned stream with independent positions are implemented using NIO buffer duplication (a buffer duplicate holds the same content but has its own position), and so the implementation logic is much simpler than FSInputStream's.
Some methods of Directory have been overridden to replace the caching feature. Some of then were final in Directory, so I have used a slightly modified version of Directory.java (BTW, I wonder why so many methods in Lucene are made final...)
These classes only works with the recently released Java 1.4.2. This is due to the fact that buffers connected with memory-mapped files could not be programmatically unmapped in previous releases, (they were unmapped only through finalization) and actively mapped files cannot be deleted. These issue are partially resolved with 1.4.2.
NIOOutputStream is the same as FSOutputStream; I don't know any way to take advantege of NIO for writing indexes (memory mapped buffers have a static size, so they are not useful if your file is growing).
I don't have a benchmarking suite for Lucene, so I can't accurately evaluate the speed of this implementation. I tested it on a small application I am developing and it seems to work well, but I think my test are not significative. Of course only the searching feature is expected to be faster, since the index writing is unchanged.
Francesco
- Francesco Bellomi "Use truth to show illusion, and illusion to show truth."
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
