[
https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael McCandless updated LUCENE-753:
--------------------------------------
Attachment: LUCENE-753.patch
New patch attached. Matthew if you could try this version out on your
index, that'd be awesome.
I didn't like how we were still copying the hairy readBytes & refill
methods from BufferedIndexInput, so I made some small additional mods
to BufferedIndexInput to notify subclass when a byte[] buffer gets
allocated, which then allowed us to fully inherit these methods.
But, then I realized we were duplicating alot of code from
FSIndexInput, so I switched to subclassing that instead and that made
things even simpler.
Some other things also fixed:
* We were ignoring bufferSize (eg setBufferSize).
* We weren't closing the FileChannel
* clone() now lazily clones the buffer again
To test this, I made NIOFSDirectory the default IMPL in
FSDirectory.getDirectory and ran all tests. One test failed at first
(because we were ignoring setBufferSize calls); with the new patch,
all tests pass.
I also built first 150K docs of wikipedia and ran various searches
using NIOFSDirectory and all seems good.
The class is quite a bit simpler now, however there's one thing I
don't like: when you use CFS, the NIOFSIndexInput.readInternal method
will wrap the CSIndexInput's byte[] (from it's parent
BufferedIndexInput class) for every call (every 1024 bytes read from
the file). I'd really like to find a clean way to reuse a single
ByteBuffer. Not yet sure how to do that though...
> Use NIO positional read to avoid synchronization in FSIndexInput
> ----------------------------------------------------------------
>
> Key: LUCENE-753
> URL: https://issues.apache.org/jira/browse/LUCENE-753
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Store
> Reporter: Yonik Seeley
> Assignee: Michael McCandless
> Fix For: 2.4
>
> Attachments: FileReadTest.java, FileReadTest.java, FileReadTest.java,
> FileReadTest.java, FileReadTest.java, FileReadTest.java, FileReadTest.java,
> FSDirectoryPool.patch, FSIndexInput.patch, FSIndexInput.patch,
> LUCENE-753.patch, LUCENE-753.patch, LUCENE-753.patch, LUCENE-753.patch,
> lucene-753.patch, lucene-753.patch
>
>
> As suggested by Doug, we could use NIO pread to avoid synchronization on the
> underlying file.
> This could mitigate any MT performance drop caused by reducing the number of
> files in the index format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
