[
https://issues.apache.org/jira/browse/HDDS-15128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Yarovoy reassigned HDDS-15128:
-------------------------------------
Assignee: Andrey Yarovoy
> switch default checksum to CRC32C since its providing better throughput on
> JRE17+
> ---------------------------------------------------------------------------------
>
> Key: HDDS-15128
> URL: https://issues.apache.org/jira/browse/HDDS-15128
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Andrey Yarovoy
> Assignee: Andrey Yarovoy
> Priority: Major
>
> According to the benchmark JVM native CRC32C provide slightly better
> throughout vs CRC32 on JRE17+. It would also make it consistent with HDFS
> default settings.
> {noformat}
> Benchmark (checksumSize) (crcImpl) Mode Cnt
> Score Error Units
> BenchMarkCRCStreaming.runCRC 512 pureCRC32 thrpt 5
> 14.731 ± 0.214 ops/s
> BenchMarkCRCStreaming.runCRC 512 pureCRC32C thrpt 5
> 14.427 ± 0.199 ops/s
> BenchMarkCRCStreaming.runCRC 512 hadoopCRC32C thrpt 5
> 24.204 ± 0.349 ops/s
> BenchMarkCRCStreaming.runCRC 512 hadoopCRC32 thrpt 5
> 26.560 ± 0.450 ops/s
> BenchMarkCRCStreaming.runCRC 512 zipCRC32 thrpt 5
> 284.283 ± 13.557 ops/s
> BenchMarkCRCStreaming.runCRC 512 zipCRC32C thrpt 5
> 283.479 ± 6.288 ops/s
> BenchMarkCRCStreaming.runCRC 1024 pureCRC32 thrpt 5
> 14.462 ± 0.223 ops/s
> BenchMarkCRCStreaming.runCRC 1024 pureCRC32C thrpt 5
> 14.801 ± 0.289 ops/s
> BenchMarkCRCStreaming.runCRC 1024 hadoopCRC32C thrpt 5
> 24.162 ± 0.382 ops/s
> BenchMarkCRCStreaming.runCRC 1024 hadoopCRC32 thrpt 5
> 25.943 ± 0.406 ops/s
> BenchMarkCRCStreaming.runCRC 1024 zipCRC32 thrpt 5
> 266.230 ± 13.065 ops/s
> BenchMarkCRCStreaming.runCRC 1024 zipCRC32C thrpt 5
> 254.955 ± 10.738 ops/s
> BenchMarkCRCStreaming.runCRC 2048 pureCRC32 thrpt 5
> 14.411 ± 0.217 ops/s
> BenchMarkCRCStreaming.runCRC 2048 pureCRC32C thrpt 5
> 14.752 ± 0.244 ops/s
> BenchMarkCRCStreaming.runCRC 2048 hadoopCRC32C thrpt 5
> 21.074 ± 0.371 ops/s
> BenchMarkCRCStreaming.runCRC 2048 hadoopCRC32 thrpt 5
> 22.141 ± 0.360 ops/s
> BenchMarkCRCStreaming.runCRC 2048 zipCRC32 thrpt 5
> 268.253 ± 10.607 ops/s
> BenchMarkCRCStreaming.runCRC 2048 zipCRC32C thrpt 5
> 259.033 ± 12.080 ops/s
> BenchMarkCRCStreaming.runCRC 4096 pureCRC32 thrpt 5
> 14.817 ± 0.257 ops/s
> BenchMarkCRCStreaming.runCRC 4096 pureCRC32C thrpt 5
> 14.455 ± 0.227 ops/s
> BenchMarkCRCStreaming.runCRC 4096 hadoopCRC32C thrpt 5
> 21.138 ± 0.362 ops/s
> BenchMarkCRCStreaming.runCRC 4096 hadoopCRC32 thrpt 5
> 21.984 ± 0.352 ops/s
> BenchMarkCRCStreaming.runCRC 4096 zipCRC32 thrpt 5
> 280.286 ± 8.329 ops/s
> BenchMarkCRCStreaming.runCRC 4096 zipCRC32C thrpt 5
> 274.376 ± 12.497 ops/s
> BenchMarkCRCStreaming.runCRC 32768 pureCRC32 thrpt 5
> 17.933 ± 0.269 ops/s
> BenchMarkCRCStreaming.runCRC 32768 pureCRC32C thrpt 5
> 17.933 ± 0.268 ops/s
> BenchMarkCRCStreaming.runCRC 32768 hadoopCRC32C thrpt 5
> 21.276 ± 0.289 ops/s
> BenchMarkCRCStreaming.runCRC 32768 hadoopCRC32 thrpt 5
> 21.953 ± 0.327 ops/s
> BenchMarkCRCStreaming.runCRC 32768 zipCRC32 thrpt 5
> 270.707 ± 20.560 ops/s
> BenchMarkCRCStreaming.runCRC 32768 zipCRC32C thrpt 5
> 276.425 ± 8.271 ops/s
> BenchMarkCRCStreaming.runCRC 1048576 pureCRC32 thrpt 5
> 17.579 ± 0.294 ops/s
> BenchMarkCRCStreaming.runCRC 1048576 pureCRC32C thrpt 5
> 17.937 ± 0.283 ops/s
> BenchMarkCRCStreaming.runCRC 1048576 hadoopCRC32C thrpt 5
> 21.141 ± 0.352 ops/s
> BenchMarkCRCStreaming.runCRC 1048576 hadoopCRC32 thrpt 5
> 21.948 ± 0.379 ops/s
> BenchMarkCRCStreaming.runCRC 1048576 zipCRC32 thrpt 5
> 277.069 ± 7.166 ops/s
> BenchMarkCRCStreaming.runCRC 1048576 zipCRC32C thrpt 5
> 280.083 ± 7.043 ops/s{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
