2013/3/17 Michael McCandless <luc...@mikemccandless.com>: > Hi Michael (directly CC'd this time...), > > Maybe you're not subscribed to the list? Your first email got some > responses, eg: > > http://lucene.markmail.org/thread/lrv7miivzmjm3ml5
Indeed, he's not, I didn't auto-subscribe him when putting his message through to the list. Might be a good idea to subscribe; looks like interesting and promising work :D --dho > Net/net, these new directory implementations sound exciting! > > Mike McCandless > > http://blog.mikemccandless.com > > On Sun, Mar 17, 2013 at 4:03 PM, Michael Poindexter > <statics...@gmail.com> wrote: >> As part of a project using Lucene I have implemented a trio of Directories >> roughly corresponding to the FSDirectory implementations in core. These >> directory implementations use the NIO2 API's in JDK7 when opening files. >> This ensures that on Windows the files are opened in a mode that allows >> deletion even if the file is open elsewhere. >> >> 1.) JDK7MMapDirectory - Roughly the same as MMapDirectory. Uses >> FileChannel.open (instead of RandomAccessFile) to create a FileChannel that >> then has map() called on it to create the mapped buffers. >> 2.) JDK7NIOFSDirectory - Roughly the same as NIOFSDirectory, but uses >> FileChannel.open to create the file channel instead of RandomAccessFile. >> 3.) JDK7AsyncFSDirectory - This one is new and different. I needed a >> replacement for SimpleFSDirectory (that was not susceptible to problems if >> interrupt()'ed) and did not have the synchronization problems on Windows of >> NIOFSDirectory. This one is used where SimpleFSDirectory could have been >> used, but uses an AsynchronousFileChannel to do it's work. The actual >> operation is still synchronous, but on Windows AsynchronousFileChannel uses >> overlapped IO, and hence does not require synchronization on the position >> and should be safe for interrupts. >> >> A couple of questions: >> 1.) Is there any interest in me contributing these to Lucene? They >> require JDK7+, but perhaps they could go in a contrib module? >> >> 2.) While implementing these I noticed the implementation of >> FSDirectory.sync seems a little strange: It just opens a new >> RandomAccessFile and forces a sync using it. The JavaDocs seem to imply >> that this would force a sync on the file handle associated with the >> RandomAccessFile, but that's not the file handle that was written to as >> part of an IndexOutput. On Windows at least this won't matter, but it >> seems theoretically wrong...i.e. according to the JavaDoc on a given >> platform this style of operation could have no impact if I am understanding >> it correctly. It seems like maybe it would be better to have a sync() call >> on an IndexOutput that can be called before closing it...am I missing >> something here? >> >> 3.) What is the best way to go about benchmarking/testing these new >> implementations to compare against the core FSDirectory implementations? >> I've seen some references to randomized tests and benchmarks on the >> developer pages on the Lucene website, but I didn't see anything that was >> along the lines of "Here's how to run the benchmarks"...any pointers would >> be much appreciated. >> >> Thanks, >> Mike Poindexter > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
