[
https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12550675
]
Michael McCandless commented on LUCENE-753:
-------------------------------------------
I ran Yonik's most recent FileReadTest.java on the platforms below,
testing single-threaded random access for fully cached 64 MB file.
I tested two Windows XP Pro machines and got opposite results from
Yonik. Yonik is your machine XP Home?
I'm showing ChannelTransfer to be much faster on all platforms except
Windows Server 2003 R2 Enterprise x64 where it's about the same as
ChannelPread and ChannelFile.
The ChannelTransfer test is giving the wrong checksum, but I think
just a bug in how checksum is computed (it's using "len" which with
ChannelTransfer is just the chunk size written on each call to
write). So I think the MB/sec is still correct.
Mac OS X 10.4 (Sun java 1.5)
config: impl=ClassicFile serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=-44611, ms=32565, MB/sec=412.15331797942576
config: impl=ChannelFile serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=-44611, ms=19512, MB/sec=687.8727347273473
config: impl=ChannelPread serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=-44611, ms=19492, MB/sec=688.5785347835009
config: impl=ChannelTransfer serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=147783, ms=16009, MB/sec=838.3892060715847
Linux 2.6.22.1 (Sun java 1.5)
config: impl=ClassicFile serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=-44611, ms=37879, MB/sec=354.33281765622115
config: impl=ChannelFile serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=-44611, ms=21845, MB/sec=614.4093751430535
config: impl=ChannelPread serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=-44611, ms=21902, MB/sec=612.8103734818737
config: impl=ChannelTransfer serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=147783, ms=15978, MB/sec=840.015821754913
Windows Server 2003 R2 Enterprise x64 (Sun java 1.6)
config: impl=ClassicFile serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=-44611, ms=32703, MB/sec=410.4141149130049
config: impl=ChannelFile serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=-44611, ms=23344, MB/sec=574.9559972583961
config: impl=ChannelPread serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=-44611, ms=23329, MB/sec=575.3256804835183
config: impl=ChannelTransfer serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=147783, ms=23422, MB/sec=573.0412774314747
Windows XP Pro SP2, laptop (Sun Java 1.5)
config: impl=ClassicFile serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=-44611, ms=71253, MB/sec=188.36782731955148
config: impl=ChannelFile serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=-44611, ms=57463, MB/sec=233.57243443607192
config: impl=ChannelPread serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=-44611, ms=58043, MB/sec=231.23844046655068
config: impl=ChannelTransfer serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=147783, ms=20039, MB/sec=669.7825640001995
Windows XP Pro SP2, older desktop (Sun Java 1.6)
config: impl=ClassicFile serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=-44611, ms=53047, MB/sec=253.01662299470283
config: impl=ChannelFile serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=-44611, ms=34047, MB/sec=394.2130819161747
config: impl=ChannelPread serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=-44611, ms=34078, MB/sec=393.8544750278772
config: impl=ChannelTransfer serial=false nThreads=1 iterations=200
bufsize=6518936 filelen=67108864
answer=147783, ms=18781, MB/sec=714.6463340610192
> Use NIO positional read to avoid synchronization in FSIndexInput
> ----------------------------------------------------------------
>
> Key: LUCENE-753
> URL: https://issues.apache.org/jira/browse/LUCENE-753
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Store
> Reporter: Yonik Seeley
> Attachments: FileReadTest.java, FileReadTest.java,
> FSIndexInput.patch, FSIndexInput.patch
>
>
> As suggested by Doug, we could use NIO pread to avoid synchronization on the
> underlying file.
> This could mitigate any MT performance drop caused by reducing the number of
> files in the index format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
