Andrey Yarovoy created HDDS-15128:
-------------------------------------
Summary: switch default checksum to CRC32C since its providing
better throughput on JRE17+
Key: HDDS-15128
URL: https://issues.apache.org/jira/browse/HDDS-15128
Project: Apache Ozone
Issue Type: Bug
Reporter: Andrey Yarovoy
According to the benchmark JVM native CRC32C provide slightly better throughout
vs CRC32 on JRE17+. It would also make it consistent with HDFS default settings.
{noformat}
Benchmark (checksumSize) (crcImpl) Mode Cnt Score
Error Units
BenchMarkCRCStreaming.runCRC 512 pureCRC32 thrpt 5 14.731
± 0.214 ops/s
BenchMarkCRCStreaming.runCRC 512 pureCRC32C thrpt 5 14.427
± 0.199 ops/s
BenchMarkCRCStreaming.runCRC 512 hadoopCRC32C thrpt 5 24.204
± 0.349 ops/s
BenchMarkCRCStreaming.runCRC 512 hadoopCRC32 thrpt 5 26.560
± 0.450 ops/s
BenchMarkCRCStreaming.runCRC 512 zipCRC32 thrpt 5 284.283
± 13.557 ops/s
BenchMarkCRCStreaming.runCRC 512 zipCRC32C thrpt 5 283.479
± 6.288 ops/s
BenchMarkCRCStreaming.runCRC 1024 pureCRC32 thrpt 5 14.462
± 0.223 ops/s
BenchMarkCRCStreaming.runCRC 1024 pureCRC32C thrpt 5 14.801
± 0.289 ops/s
BenchMarkCRCStreaming.runCRC 1024 hadoopCRC32C thrpt 5 24.162
± 0.382 ops/s
BenchMarkCRCStreaming.runCRC 1024 hadoopCRC32 thrpt 5 25.943
± 0.406 ops/s
BenchMarkCRCStreaming.runCRC 1024 zipCRC32 thrpt 5 266.230
± 13.065 ops/s
BenchMarkCRCStreaming.runCRC 1024 zipCRC32C thrpt 5 254.955
± 10.738 ops/s
BenchMarkCRCStreaming.runCRC 2048 pureCRC32 thrpt 5 14.411
± 0.217 ops/s
BenchMarkCRCStreaming.runCRC 2048 pureCRC32C thrpt 5 14.752
± 0.244 ops/s
BenchMarkCRCStreaming.runCRC 2048 hadoopCRC32C thrpt 5 21.074
± 0.371 ops/s
BenchMarkCRCStreaming.runCRC 2048 hadoopCRC32 thrpt 5 22.141
± 0.360 ops/s
BenchMarkCRCStreaming.runCRC 2048 zipCRC32 thrpt 5 268.253
± 10.607 ops/s
BenchMarkCRCStreaming.runCRC 2048 zipCRC32C thrpt 5 259.033
± 12.080 ops/s
BenchMarkCRCStreaming.runCRC 4096 pureCRC32 thrpt 5 14.817
± 0.257 ops/s
BenchMarkCRCStreaming.runCRC 4096 pureCRC32C thrpt 5 14.455
± 0.227 ops/s
BenchMarkCRCStreaming.runCRC 4096 hadoopCRC32C thrpt 5 21.138
± 0.362 ops/s
BenchMarkCRCStreaming.runCRC 4096 hadoopCRC32 thrpt 5 21.984
± 0.352 ops/s
BenchMarkCRCStreaming.runCRC 4096 zipCRC32 thrpt 5 280.286
± 8.329 ops/s
BenchMarkCRCStreaming.runCRC 4096 zipCRC32C thrpt 5 274.376
± 12.497 ops/s
BenchMarkCRCStreaming.runCRC 32768 pureCRC32 thrpt 5 17.933
± 0.269 ops/s
BenchMarkCRCStreaming.runCRC 32768 pureCRC32C thrpt 5 17.933
± 0.268 ops/s
BenchMarkCRCStreaming.runCRC 32768 hadoopCRC32C thrpt 5 21.276
± 0.289 ops/s
BenchMarkCRCStreaming.runCRC 32768 hadoopCRC32 thrpt 5 21.953
± 0.327 ops/s
BenchMarkCRCStreaming.runCRC 32768 zipCRC32 thrpt 5 270.707
± 20.560 ops/s
BenchMarkCRCStreaming.runCRC 32768 zipCRC32C thrpt 5 276.425
± 8.271 ops/s
BenchMarkCRCStreaming.runCRC 1048576 pureCRC32 thrpt 5 17.579
± 0.294 ops/s
BenchMarkCRCStreaming.runCRC 1048576 pureCRC32C thrpt 5 17.937
± 0.283 ops/s
BenchMarkCRCStreaming.runCRC 1048576 hadoopCRC32C thrpt 5 21.141
± 0.352 ops/s
BenchMarkCRCStreaming.runCRC 1048576 hadoopCRC32 thrpt 5 21.948
± 0.379 ops/s
BenchMarkCRCStreaming.runCRC 1048576 zipCRC32 thrpt 5 277.069
± 7.166 ops/s
BenchMarkCRCStreaming.runCRC 1048576 zipCRC32C thrpt 5 280.083
± 7.043 ops/s{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
