[
https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12609401#action_12609401
]
bripink edited comment on LUCENE-753 at 6/30/08 4:32 PM:
-----------------------------------------------------------------
I was curious about the discrepancy between the ChannelPread implementation and
the SeparateFile implementation that Yonik saw. At least on Mac OS X, the
kernel implementation of read is virtually the same as pread, so there
shouldn't be any appreciable performance difference unless the VM is doing
something funny. Sure enough, the implementations of read() under
RandomAccessFile and read() under FileChannel are totally different. The
former relies on a buffer allocated either on the stack or by malloc, while the
latter allocates a native buffer and copies the results to the original array.
Switching to a native buffer in the benchmark yields identical results for
ChannelPread and SeparateFile on 1.5 and 1.6 on OS X. I'm attaching an
implementation of ChannelPreadDirect that uses a native buffer.
This may be a moot point because any implementation inside Lucene needs to
consume a byte[] and not a ByteBuffer, but at least it's informative.
was (Author: bripink):
I was curious about the discrepancy between the ChannelPread implementation
and the SeparateFile implementation that Yonik saw. At least on Mac OS X, the
kernel implementation of read is virtually the same as pread, so there
shouldn't be any appreciable performance difference unless the VM is doing
something funny. Sure enough, the implementations of read() under
RandomAccessFile and read() under FileChannel are totally different. The
former relies on either a buffer allocated either on the stack or by malloc,
while the latter allocates a native buffer and copies the results to the
original array.
Switching to a native buffer in the benchmark yields identical results for
ChannelPread and SeparateFile on 1.5 and 1.6 on OS X. I'm attaching an
implementation of ChannelPreadDirect that uses a native buffer.
This may be a moot point because any implementation inside Lucene needs to
consume a byte[] and not a ByteBuffer, but at least it's informative.
> Use NIO positional read to avoid synchronization in FSIndexInput
> ----------------------------------------------------------------
>
> Key: LUCENE-753
> URL: https://issues.apache.org/jira/browse/LUCENE-753
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Store
> Reporter: Yonik Seeley
> Attachments: FileReadTest.java, FileReadTest.java, FileReadTest.java,
> FileReadTest.java, FileReadTest.java, FileReadTest.java, FileReadTest.java,
> FSIndexInput.patch, FSIndexInput.patch, lucene-753.patch
>
>
> As suggested by Doug, we could use NIO pread to avoid synchronization on the
> underlying file.
> This could mitigate any MT performance drop caused by reducing the number of
> files in the index format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
