[
https://issues.apache.org/jira/browse/HADOOP-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HADOOP-6166:
--------------------------------
Attachment: graph.r
Rplots.pdf
Attaching PDF of the concurrency benchmark from HADOOP-5318 run on all of these
implementations. The first page also includes java.util.zip.CRC32 to verify
that the benchmark script really is testing something and that we've fixed the
problem originally reported in that ticket. The second page of the PDF is the
same graph after dropping the super-slow one.
I've normalized all of the results as percentages of the very fastest. It looks
like they're all within 5-10% of each other for this benchmark, and with the
error bars it's pretty obvious that there is no clear winner for the
1-byte-write case at any concurrency level. If anything Crc32_6_6 seems to come
out a little ahead, but really barely discernible.
I'd upload the script I used to run the benchmarks, but I had to add a silly
little patch to Hadoop to be able to switch between implementations easily, so
it's a hassle to run.
For reference, this system is a Nehalem box running 64-bit JDK 1.6.0u14, 8
cores w/ hyperthreading (16 logical).
> Improve PureJavaCrc32
> ---------------------
>
> Key: HADOOP-6166
> URL: https://issues.apache.org/jira/browse/HADOOP-6166
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: c6166_20090722.patch, c6166_20090722_benchmark_32VM.txt,
> c6166_20090722_benchmark_64VM.txt, c6166_20090727.patch,
> c6166_20090728.patch, c6166_20090810.patch, c6166_20090811.patch, graph.r,
> Rplots.pdf
>
>
> Got some ideas to improve CRC32 calculation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
