[
https://issues.apache.org/jira/browse/CASSANDRA-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175050#comment-13175050
]
Vijay commented on CASSANDRA-3610:
----------------------------------
I might have miss commuicated the results... here is the results without any
setting. SunJDK vs OpenJDK is the only difference.
[vijay_tcasstest@vijay_tcass--1a-i-aad629c8 ~]$ java -jar
TestCRC32Performance.jar
||bytes||PureJava MB/sec||Native MB/sec||Random PureJava MB/sec||Native MB/sec||
| 1 |40.582 |11.678 |
| 2 |152.297 |23.060 |
| 4 |206.689 |46.501 |
| 8 |311.654 |87.895 |
| 16 |333.174 |151.716 |
| 32 |386.252 |246.780 |
| 64 |414.639 |353.623 || PureJava |0-64 |256.513
MB/sec|| Native |0-64 |196.319 MB/sec|
| 128 |429.995 |449.165 || PureJava |0-128 |324.051
MB/sec|| Native |0-128 |306.220 MB/sec|
| 256 |438.206 |519.658 || PureJava |0-256 |377.582
MB/sec|| Native |0-256 |410.256 MB/sec|
| 512 |442.210 |565.027 || PureJava |0-512 |408.946
MB/sec|| Native |0-512 |494.210 MB/sec|
| 1024 |453.627 |591.082 || PureJava |0-1024 |428.094
MB/sec|| Native |0-1024 |551.725 MB/sec|
| 2048 |455.738 |604.581 || PureJava |0-2048 |436.861
MB/sec|| Native |0-2048 |573.995 MB/sec|
| 4096 |443.916 |610.058 || PureJava |0-4096 |444.456
MB/sec|| Native |0-4096 |595.364 MB/sec|
| 8192 |449.964 |613.482 || PureJava |0-8192 |446.008
MB/sec|| Native |0-8192 |606.647 MB/sec|
| 16384 |447.484 |615.583 || PureJava |0-16384
|446.013 MB/sec|| Native |0-16384 |609.534 MB/sec|
| 32768 |446.452 |616.336 || PureJava |0-32768
|447.556 MB/sec|| Native |0-32768 |612.477 MB/sec|
| 65536 |446.351 |616.431 || PureJava |0-65536
|446.171 MB/sec|| Native |0-65536 |612.508 MB/sec|
| 131072 |446.457 |616.208 || PureJava |0-131072
|444.713 MB/sec|| Native |0-131072 |615.538 MB/sec|
| 262144 |445.521 |613.368 || PureJava |0-262144
|446.434 MB/sec|| Native |0-262144 |615.559 MB/sec|
| 524288 |445.233 |612.942 || PureJava |0-524288
|444.488 MB/sec|| Native |0-524288 |613.167 MB/sec|
| 1048576 |445.278 |613.044 || PureJava |0-1048576
|445.561 MB/sec|| Native |0-1048576 |614.500 MB/sec|
| 2097152 |443.896 |611.457 || PureJava |0-2097152
|440.685 MB/sec|| Native |0-2097152 |601.230 MB/sec|
| 4194304 |425.962 |595.604 || PureJava |0-4194304
|440.787 MB/sec|| Native |0-4194304 |610.587 MB/sec|
| 8388608 |430.688 |608.036 || PureJava |0-8388608
|439.621 MB/sec|| Native |0-8388608 |608.256 MB/sec|
| 16777216 |436.099 |599.706 || PureJava |0-16777216
|435.695 MB/sec|| Native |0-16777216 |603.244 MB/sec|
[vijay_tcasstest@vijay_tcass--1a-i-aad629c8 ~]$ /usr/java/latest/jre/bin/java
-jar TestCRC32Performance.jar ||bytes||PureJava MB/sec||Native MB/sec||Random
PureJava MB/sec||Native MB/sec||
| 1 |78.345 |10.897 |
| 2 |157.171 |20.341 |
| 4 |197.572 |40.119 |
| 8 |295.368 |67.051 |
| 16 |379.664 |104.782 |
| 32 |440.257 |147.490 |
| 64 |478.970 |187.356 || PureJava |0-64 |268.908
MB/sec|| Native |0-64 |141.436 MB/sec|
| 128 |495.235 |211.237 || PureJava |0-128 |363.636
MB/sec|| Native |0-128 |182.596 MB/sec|
| 256 |511.961 |231.697 || PureJava |0-256 |432.433
MB/sec|| Native |0-256 |211.921 MB/sec|
| 512 |517.852 |240.495 || PureJava |0-512 |475.837
MB/sec|| Native |0-512 |229.803 MB/sec|
| 1024 |516.815 |246.001 || PureJava |0-1024 |496.127
MB/sec|| Native |0-1024 |240.152 MB/sec|
| 2048 |520.283 |248.616 || PureJava |0-2048 |509.963
MB/sec|| Native |0-2048 |245.212 MB/sec|
| 4096 |521.950 |249.806 || PureJava |0-4096 |518.219
MB/sec|| Native |0-4096 |247.586 MB/sec|
| 8192 |516.635 |249.594 || PureJava |0-8192 |520.342
MB/sec|| Native |0-8192 |249.513 MB/sec|
| 16384 |522.743 |250.757 || PureJava |0-16384
|522.467 MB/sec|| Native |0-16384 |250.493 MB/sec|
| 32768 |521.264 |250.807 || PureJava |0-32768
|520.336 MB/sec|| Native |0-32768 |250.503 MB/sec|
| 65536 |521.192 |250.916 || PureJava |0-65536
|520.443 MB/sec|| Native |0-65536 |250.505 MB/sec|
| 131072 |520.264 |249.842 || PureJava |0-131072
|514.098 MB/sec|| Native |0-131072 |250.517 MB/sec|
| 262144 |519.953 |250.719 || PureJava |0-262144
|520.510 MB/sec|| Native |0-262144 |249.682 MB/sec|
| 524288 |512.457 |250.282 || PureJava |0-524288
|516.266 MB/sec|| Native |0-524288 |250.072 MB/sec|
| 1048576 |514.005 |249.813 || PureJava |0-1048576
|516.444 MB/sec|| Native |0-1048576 |250.548 MB/sec|
| 2097152 |519.207 |250.680 || PureJava |0-2097152
|519.260 MB/sec|| Native |0-2097152 |250.526 MB/sec|
| 4194304 |518.882 |250.608 || PureJava |0-4194304
|517.022 MB/sec|| Native |0-4194304 |250.538 MB/sec|
| 8388608 |517.444 |250.182 || PureJava |0-8388608
|518.502 MB/sec|| Native |0-8388608 |249.994 MB/sec|
| 16777216 |511.875 |249.008 || PureJava |0-16777216
|514.823 MB/sec|| Native |0-16777216 |249.477 MB/sec|
[vijay_tcasstest@vijay_tcass--1a-i-aad629c8 ~]$ java -version
java version "1.6.0_17"
OpenJDK Runtime Environment (IcedTea6 1.7.10) (rhel-1.21.b17.el5-x86_64)
OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)
[vijay_tcasstest@vijay_tcass--1a-i-aad629c8 ~]$ /usr/java/latest/jre/bin/java
-versionjava version "1.6.0_27"
Java(TM) SE Runtime Environment (build 1.6.0_27-b07)
Java HotSpot(TM) 64-Bit Server VM (build 20.2-b06, mixed mode)
[vijay_tcasstest@vijay_tcass--1a-i-aad629c8 ~]$
> Checksum improvement for CompressedRandomAccessReader
> -----------------------------------------------------
>
> Key: CASSANDRA-3610
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3610
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.1
> Environment: JVM
> Reporter: Vijay
> Assignee: Vijay
> Priority: Minor
> Fix For: 1.1
>
> Attachments: 0001-use-pure-java-CRC32-v2.patch,
> 0001-use-pure-java-CRC32.patch, TestCrc32Performance.java
>
>
> When compression is on, Currently we see checksum taking up about 40% of the
> CPU more than snappy library.
> Looks like hadoop solved it by implementing their own checksum, we can either
> use it or implement something like that.
> http://images.slidesharecdn.com/1toddlipconyanpeichen-cloudera-hadoopandperformance-final-111110132228-phpapp01-slide-15-768.jpg?1321043717
> in our test env it provided 50% improvement over native implementation which
> uses jni to call the OS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
