PeterPtroc opened a new pull request, #8371:
URL: https://github.com/apache/hadoop/pull/8371
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines:
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g.,
'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
Before introducing RVV optimizations for CRC32, it is useful to first close
the optimization gap for CRC32C on Zbc. This change addresses that prerequisite
and helps establish a cleaner baseline for subsequent RVV-focused tuning work.
The following summarizes the work included in this commit.
- Adds a riscv64 native CRC32C acceleration path using Zbc CLMUL in the
existing bulk CRC implementation.
- Adds CRC32C bitwise fallback for small/misaligned data and pipelined
CRC32C processing for larger blocks.
- Hooks the CRC32C function pointer at runtime when Zbc is detected on
linux-riscv64.
- Keeps existing CRC32 (zlib) behavior unchanged.
For small chunks (bpc <= 32), throughput regresses; for medium/large chunks
(bpc >= 256), throughput improves significantly.
| bpc | #T | NativeC (origin) | NativeC (new) | Delta (MB/s) | Delta% |
|---:|---:|---:|---:|---:|---:|
| 32 | 1 | 695.1 | 70.4 | -624.7 | -89.9% |
| 32 | 2 | 699.4 | 70.3 | -629.1 | -89.9% |
| 32 | 4 | 701.0 | 70.5 | -630.5 | -89.9% |
| 32 | 8 | 682.8 | 70.4 | -612.4 | -89.7% |
| 32 | 16 | 690.6 | 70.3 | -620.3 | -89.8% |
| 64 | 1 | 830.0 | 717.7 | -112.3 | -13.5% |
| 64 | 2 | 791.0 | 751.1 | -39.9 | -5.0% |
| 64 | 4 | 817.0 | 738.5 | -78.5 | -9.6% |
| 64 | 8 | 813.1 | 752.7 | -60.4 | -7.4% |
| 64 | 16 | 807.9 | 746.5 | -61.4 | -7.6% |
| 128 | 1 | 921.8 | 679.0 | -242.8 | -26.3% |
| 128 | 2 | 865.5 | 627.4 | -238.1 | -27.5% |
| 128 | 4 | 902.6 | 693.3 | -209.3 | -23.2% |
| 128 | 8 | 916.8 | 698.0 | -218.8 | -23.9% |
| 128 | 16 | 895.2 | 672.9 | -222.3 | -24.8% |
| 256 | 1 | 872.0 | 2982.3 | +2110.3 | +242.0% |
| 256 | 2 | 756.8 | 3023.4 | +2266.6 | +299.5% |
| 256 | 4 | 879.4 | 2518.3 | +1638.9 | +186.4% |
| 256 | 8 | 847.2 | 2529.9 | +1682.7 | +198.6% |
| 256 | 16 | 892.8 | 2518.4 | +1625.6 | +182.1% |
| 512 | 1 | 969.1 | 3698.0 | +2728.9 | +281.6% |
| 512 | 2 | 988.9 | 3737.5 | +2748.6 | +277.9% |
| 512 | 4 | 955.8 | 2996.8 | +2041.0 | +213.5% |
| 512 | 8 | 936.1 | 2645.3 | +1709.2 | +182.6% |
| 512 | 16 | 952.3 | 3002.4 | +2050.1 | +215.3% |
| 1024 | 1 | 1006.7 | 3596.7 | +2590.0 | +257.3% |
| 1024 | 2 | 947.1 | 3588.8 | +2641.7 | +278.9% |
| 1024 | 4 | 1001.0 | 3025.9 | +2024.9 | +202.3% |
| 1024 | 8 | 980.3 | 2751.4 | +1771.1 | +180.7% |
| 1024 | 16 | 982.2 | 3027.8 | +2045.6 | +208.3% |
| 2048 | 1 | 1015.5 | 3529.7 | +2514.2 | +247.6% |
| 2048 | 2 | 1033.3 | 3575.9 | +2542.6 | +246.1% |
| 2048 | 4 | 1016.1 | 3070.5 | +2054.4 | +202.2% |
| 2048 | 8 | 1006.1 | 2819.4 | +1813.3 | +180.2% |
| 2048 | 16 | 991.3 | 3056.3 | +2065.0 | +208.3% |
| 4096 | 1 | 1029.0 | 4485.6 | +3456.6 | +335.9% |
| 4096 | 2 | 1026.8 | 4562.3 | +3535.5 | +344.3% |
| 4096 | 4 | 1005.9 | 3617.4 | +2611.5 | +259.6% |
| 4096 | 8 | 1006.4 | 3634.1 | +2627.7 | +261.1% |
| 4096 | 16 | 1015.6 | 3630.8 | +2615.2 | +257.5% |
| 8192 | 1 | 1032.9 | 5100.4 | +4067.5 | +393.8% |
| 8192 | 2 | 1043.6 | 5057.9 | +4014.3 | +384.7% |
| 8192 | 4 | 1025.3 | 3954.0 | +2928.7 | +285.6% |
| 8192 | 8 | 1024.9 | 3415.2 | +2390.3 | +233.2% |
| 8192 | 16 | 1007.1 | 3899.5 | +2892.4 | +287.2% |
| 16384 | 1 | 1034.7 | 5706.5 | +4671.8 | +451.5% |
| 16384 | 2 | 1045.7 | 5694.4 | +4648.7 | +444.6% |
| 16384 | 4 | 1012.9 | 4325.6 | +3312.7 | +327.1% |
| 16384 | 8 | 1036.8 | 4261.1 | +3224.3 | +311.0% |
| 16384 | 16 | 1011.5 | 4556.3 | +3544.8 | +350.4% |
| 32768 | 1 | 1037.7 | 6255.9 | +5218.2 | +502.9% |
| 32768 | 2 | 975.2 | 6236.5 | +5261.3 | +539.5% |
| 32768 | 4 | 1030.5 | 4543.7 | +3513.2 | +340.9% |
| 32768 | 8 | 1007.3 | 4311.6 | +3304.3 | +328.0% |
| 32768 | 16 | 1010.7 | 4823.0 | +3812.3 | +377.2% |
| 65536 | 1 | 1037.5 | 6060.3 | +5022.8 | +484.1% |
| 65536 | 2 | 974.2 | 6235.9 | +5261.7 | +540.1% |
| 65536 | 4 | 1001.4 | 4620.2 | +3618.8 | +361.4% |
| 65536 | 8 | 1007.5 | 4624.2 | +3616.7 | +359.0% |
| 65536 | 16 | 1011.0 | 4091.1 | +3080.1 | +304.7% |
### How was this patch tested?
Built `hadoop-common` with native profile on riscv64; verified functionality
with `TestNativeCrc32`.
Ran Hadoop's CRC benchmark on riscv64 with JDK 17.
Summary:
- Unit/native tests: pass (`Tests run: 22, Failures: 0, Errors: 0, Skipped:
0`)
- Build: success
Command:
```
mvn -Pnative \
-Dtest=org.apache.hadoop.util.TestNativeCrc32 \
-Djava.library.path="$HADOOP_COMMON_LIB_NATIVE_DIR" \
test
```
Results:
```
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.util.TestNativeCrc32
[INFO] Tests run: 22, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
1.725 s -- in org.apache.hadoop.util.TestNativeCrc32
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 22, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] Executed tasks
[INFO]
[INFO] --- hadoop-maven-plugins:3.6.0-SNAPSHOT:cmake-test (test_bulk_crc32)
@ hadoop-common ---
[INFO]
[INFO] --- hadoop-maven-plugins:3.6.0-SNAPSHOT:cmake-test
(erasure_code_test) @ hadoop-common ---
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
```
Command:
```
export
HADOOP_COMMON_LIB_NATIVE_DIR="$PWD/hadoop-common-project/hadoop-common/target/native/target/usr/local/lib"
export LD_LIBRARY_PATH="$HADOOP_COMMON_LIB_NATIVE_DIR:$LD_LIBRARY_PATH"
mvn -Pnative -DskipTests -Dexec.classpathScope=test \
-Dexec.mainClass=org.apache.hadoop.util.Crc32PerformanceTest \
-Djava.library.path="$HADOOP_COMMON_LIB_NATIVE_DIR" exec:java
```
Results (Origin)
```
[INFO] Scanning for projects...
[INFO]
------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO]
------------------------------------------------------------------------
[INFO] os.detected.name: linux
[INFO] os.detected.arch: riscv64
[INFO] os.detected.bitness: 64
[INFO] os.detected.version: 6.12
[INFO] os.detected.version.major: 6
[INFO] os.detected.version.minor: 12
[INFO] os.detected.release: PolyOS
[INFO] os.detected.release.version: 24.03
[INFO] os.detected.release.like.PolyOS: true
[INFO] os.detected.classifier: linux-riscv64
[INFO]
[INFO] ------------------< org.apache.hadoop:hadoop-common
>-------------------
[INFO] Building Apache Hadoop Common 3.6.0-SNAPSHOT
[INFO] --------------------------------[ jar
]---------------------------------
[INFO]
[INFO] --- exec-maven-plugin:1.3.1:java (default-cli) @ hadoop-common ---
[WARNING] Warning: killAfter is now deprecated. Do you need it ? Please
comment on MEXEC-6.
java.version = 17.0.11
java.runtime.name = OpenJDK Runtime Environment
java.runtime.version = 17.0.11+9
java.vm.version = 17.0.11+9
java.vm.vendor = BiSheng
java.vm.name = OpenJDK 64-Bit Server VM
java.vm.specification.version = 17
java.specification.version = 17
os.arch = riscv64
os.name = Linux
os.version = 6.12.74-0.0.0.0.riscv64
Data Length = 64 MB
Trials = 5
Direct Buffer Performance Table (bpc: byte-per-crc in MB/sec; #T: #Theads)
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 32 | 1 | 49.2 | 158.2 | 221.4% | 183.7 | 16.1% |
179.7 | -2.2% | 504.4 | 180.6% | 695.1 | 37.8% |
| 32 | 2 | 50.1 | 159.0 | 217.2% | 186.1 | 17.1% |
183.6 | -1.3% | 527.4 | 187.3% | 699.4 | 32.6% |
| 32 | 4 | 45.4 | 137.5 | 202.8% | 159.9 | 16.3% |
157.6 | -1.4% | 509.8 | 223.3% | 701.0 | 37.5% |
| 32 | 8 | 44.7 | 126.4 | 182.7% | 157.0 | 24.2% |
138.3 | -11.9% | 506.7 | 266.3% | 682.8 | 34.7% |
| 32 | 16 | 40.5 | 130.7 | 222.9% | 148.7 | 13.7% |
148.8 | 0.1% | 516.0 | 246.7% | 690.6 | 33.8% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 64 | 1 | 92.4 | 266.5 | 188.4% | 293.6 | 10.2% |
293.9 | 0.1% | 295.7 | 0.6% | 830.0 | 180.7% |
| 64 | 2 | 91.2 | 265.3 | 190.8% | 288.4 | 8.7% |
286.4 | -0.7% | 294.8 | 3.0% | 791.0 | 168.3% |
| 64 | 4 | 86.6 | 264.1 | 204.8% | 274.2 | 3.8% |
275.8 | 0.6% | 287.5 | 4.2% | 817.0 | 184.2% |
| 64 | 8 | 82.9 | 237.5 | 186.5% | 258.5 | 8.8% |
267.2 | 3.4% | 294.7 | 10.3% | 813.1 | 175.9% |
| 64 | 16 | 79.9 | 195.1 | 144.2% | 234.5 | 20.2% |
240.7 | 2.7% | 295.0 | 22.5% | 807.9 | 173.9% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 128 | 1 | 151.1 | 394.9 | 161.3% | 414.9 | 5.1% |
419.4 | 1.1% | 2372.3 | 465.6% | 921.8 | -61.1% |
| 128 | 2 | 147.3 | 389.9 | 164.8% | 419.0 | 7.5% |
416.1 | -0.7% | 1621.5 | 289.7% | 865.5 | -46.6% |
| 128 | 4 | 151.8 | 371.3 | 144.5% | 361.2 | -2.7% |
387.0 | 7.1% | 1991.1 | 414.5% | 902.6 | -54.7% |
| 128 | 8 | 140.0 | 343.8 | 145.7% | 368.1 | 7.1% |
374.0 | 1.6% | 2183.1 | 483.7% | 916.8 | -58.0% |
| 128 | 16 | 146.7 | 349.7 | 138.4% | 374.3 | 7.0% |
358.6 | -4.2% | 1896.5 | 428.9% | 895.2 | -52.8% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 256 | 1 | 252.2 | 526.4 | 108.7% | 519.4 | -1.3% |
534.9 | 3.0% | 2801.8 | 423.8% | 872.0 | -68.9% |
| 256 | 2 | 244.7 | 523.5 | 114.0% | 524.4 | 0.2% |
520.2 | -0.8% | 2745.8 | 427.8% | 756.8 | -72.4% |
| 256 | 4 | 232.1 | 523.6 | 125.5% | 493.4 | -5.8% |
526.7 | 6.7% | 1734.8 | 229.4% | 879.4 | -49.3% |
| 256 | 8 | 236.1 | 479.7 | 103.2% | 472.6 | -1.5% |
485.3 | 2.7% | 1852.7 | 281.8% | 847.2 | -54.3% |
| 256 | 16 | 228.6 | 474.8 | 107.7% | 482.8 | 1.7% |
489.0 | 1.3% | 1598.7 | 227.0% | 892.8 | -44.2% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 512 | 1 | 348.0 | 636.1 | 82.8% | 616.4 | -3.1% |
629.7 | 2.2% | 2935.3 | 366.1% | 969.1 | -67.0% |
| 512 | 2 | 353.5 | 622.2 | 76.0% | 590.8 | -5.1% |
597.8 | 1.2% | 2204.9 | 268.8% | 988.9 | -55.2% |
| 512 | 4 | 353.3 | 609.1 | 72.4% | 583.6 | -4.2% |
604.0 | 3.5% | 2234.2 | 269.9% | 955.8 | -57.2% |
| 512 | 8 | 326.4 | 591.5 | 81.2% | 584.0 | -1.3% |
594.5 | 1.8% | 2205.3 | 271.0% | 936.1 | -57.6% |
| 512 | 16 | 333.4 | 587.1 | 76.1% | 574.3 | -2.2% |
589.8 | 2.7% | 1989.8 | 237.4% | 952.3 | -52.1% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 1024 | 1 | 461.8 | 700.0 | 51.6% | 636.3 | -9.1% |
649.6 | 2.1% | 3020.7 | 365.0% | 1006.7 | -66.7% |
| 1024 | 2 | 452.3 | 703.9 | 55.6% | 652.1 | -7.4% |
622.9 | -4.5% | 2919.2 | 368.6% | 947.1 | -67.6% |
| 1024 | 4 | 443.4 | 670.7 | 51.2% | 636.9 | -5.0% |
634.9 | -0.3% | 2618.8 | 312.5% | 1001.0 | -61.8% |
| 1024 | 8 | 441.1 | 655.5 | 48.6% | 608.7 | -7.1% |
635.5 | 4.4% | 2186.3 | 244.0% | 980.3 | -55.2% |
| 1024 | 16 | 436.6 | 652.1 | 49.3% | 625.9 | -4.0% |
635.9 | 1.6% | 2173.4 | 241.8% | 982.2 | -54.8% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 2048 | 1 | 514.2 | 719.6 | 39.9% | 660.6 | -8.2% |
677.1 | 2.5% | 3188.5 | 370.9% | 1015.5 | -68.1% |
| 2048 | 2 | 543.6 | 693.8 | 27.6% | 634.3 | -8.6% |
645.7 | 1.8% | 3076.5 | 376.5% | 1033.3 | -66.4% |
| 2048 | 4 | 504.4 | 697.5 | 38.3% | 650.2 | -6.8% |
658.0 | 1.2% | 2444.8 | 271.5% | 1016.1 | -58.4% |
| 2048 | 8 | 515.2 | 694.3 | 34.8% | 637.8 | -8.1% |
663.0 | 3.9% | 2807.4 | 323.5% | 1006.1 | -64.2% |
| 2048 | 16 | 502.8 | 689.4 | 37.1% | 639.7 | -7.2% |
663.5 | 3.7% | 2617.8 | 294.5% | 991.3 | -62.1% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 4096 | 1 | 570.9 | 743.2 | 30.2% | 681.9 | -8.3% |
694.4 | 1.8% | 3620.6 | 421.4% | 1029.0 | -71.6% |
| 4096 | 2 | 561.3 | 748.1 | 33.3% | 651.4 | -12.9% |
664.9 | 2.1% | 2827.3 | 325.2% | 1026.8 | -63.7% |
| 4096 | 4 | 583.4 | 719.6 | 23.3% | 662.4 | -7.9% |
677.8 | 2.3% | 2816.8 | 315.6% | 1005.9 | -64.3% |
| 4096 | 8 | 562.7 | 722.8 | 28.4% | 670.1 | -7.3% |
680.6 | 1.6% | 3002.3 | 341.1% | 1006.4 | -66.5% |
| 4096 | 16 | 567.6 | 713.4 | 25.7% | 661.2 | -7.3% |
682.0 | 3.1% | 2700.2 | 295.9% | 1015.6 | -62.4% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 8192 | 1 | 621.1 | 753.5 | 21.3% | 687.7 | -8.7% |
701.1 | 1.9% | 4044.4 | 476.9% | 1032.9 | -74.5% |
| 8192 | 2 | 579.4 | 710.1 | 22.6% | 652.6 | -8.1% |
707.9 | 8.5% | 4040.9 | 470.8% | 1043.6 | -74.2% |
| 8192 | 4 | 603.6 | 721.5 | 19.5% | 677.2 | -6.1% |
675.5 | -0.3% | 3049.6 | 351.5% | 1025.3 | -66.4% |
| 8192 | 8 | 558.3 | 714.2 | 27.9% | 660.4 | -7.5% |
684.0 | 3.6% | 3189.0 | 366.2% | 1024.9 | -67.9% |
| 8192 | 16 | 587.3 | 730.7 | 24.4% | 657.7 | -10.0% |
682.0 | 3.7% | 2822.2 | 313.8% | 1007.1 | -64.3% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 16384 | 1 | 633.3 | 754.9 | 19.2% | 689.3 | -8.7% |
704.5 | 2.2% | 4114.2 | 484.0% | 1034.7 | -74.8% |
| 16384 | 2 | 568.3 | 681.1 | 19.9% | 631.5 | -7.3% |
646.8 | 2.4% | 3135.5 | 384.8% | 1045.7 | -66.7% |
| 16384 | 4 | 586.9 | 688.3 | 17.3% | 669.5 | -2.7% |
652.3 | -2.6% | 3083.4 | 372.7% | 1012.9 | -67.1% |
| 16384 | 8 | 597.9 | 726.2 | 21.5% | 635.5 | -12.5% |
654.5 | 3.0% | 2851.8 | 335.7% | 1036.8 | -63.6% |
| 16384 | 16 | 592.1 | 698.1 | 17.9% | 639.2 | -8.4% |
655.6 | 2.6% | 3064.9 | 367.5% | 1011.5 | -67.0% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 32768 | 1 | 640.9 | 756.8 | 18.1% | 688.7 | -9.0% |
703.6 | 2.2% | 4401.6 | 525.6% | 1037.7 | -76.4% |
| 32768 | 2 | 592.4 | 668.7 | 12.9% | 691.1 | 3.3% |
642.6 | -7.0% | 3277.6 | 410.1% | 975.2 | -70.2% |
| 32768 | 4 | 565.4 | 724.9 | 28.2% | 633.0 | -12.7% |
646.1 | 2.1% | 3265.6 | 405.4% | 1030.5 | -68.4% |
| 32768 | 8 | 601.3 | 683.1 | 13.6% | 623.8 | -8.7% |
639.9 | 2.6% | 3460.8 | 440.9% | 1007.3 | -70.9% |
| 32768 | 16 | 590.8 | 696.0 | 17.8% | 625.5 | -10.1% |
645.1 | 3.1% | 2790.5 | 332.6% | 1010.7 | -63.8% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 65536 | 1 | 630.3 | 761.5 | 20.8% | 692.2 | -9.1% |
709.6 | 2.5% | 4314.6 | 508.0% | 1037.5 | -76.0% |
| 65536 | 2 | 603.9 | 690.4 | 14.3% | 687.9 | -0.4% |
621.9 | -9.6% | 3258.4 | 423.9% | 974.2 | -70.1% |
| 65536 | 4 | 603.0 | 692.8 | 14.9% | 618.4 | -10.7% |
616.6 | -0.3% | 3278.9 | 431.8% | 1001.4 | -69.5% |
| 65536 | 8 | 606.3 | 697.7 | 15.1% | 646.2 | -7.4% |
639.9 | -1.0% | 3559.0 | 456.1% | 1007.5 | -71.7% |
| 65536 | 16 | 587.2 | 703.6 | 19.8% | 655.4 | -6.9% |
637.0 | -2.8% | 3083.7 | 384.1% | 1011.0 | -67.2% |
Elapsed 464.6s
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
```
Results (With this commit)
```
[INFO] Scanning for projects...
[INFO]
------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO]
------------------------------------------------------------------------
[INFO] os.detected.name: linux
[INFO] os.detected.arch: riscv64
[INFO] os.detected.bitness: 64
[INFO] os.detected.version: 6.12
[INFO] os.detected.version.major: 6
[INFO] os.detected.version.minor: 12
[INFO] os.detected.release: PolyOS
[INFO] os.detected.release.version: 24.03
[INFO] os.detected.release.like.PolyOS: true
[INFO] os.detected.classifier: linux-riscv64
[INFO]
[INFO] ------------------< org.apache.hadoop:hadoop-common
>-------------------
[INFO] Building Apache Hadoop Common 3.6.0-SNAPSHOT
[INFO] --------------------------------[ jar
]---------------------------------
[INFO]
[INFO] --- exec-maven-plugin:1.3.1:java (default-cli) @ hadoop-common ---
[WARNING] Warning: killAfter is now deprecated. Do you need it ? Please
comment on MEXEC-6.
java.version = 17.0.11
java.runtime.name = OpenJDK Runtime Environment
java.runtime.version = 17.0.11+9
java.vm.version = 17.0.11+9
java.vm.vendor = BiSheng
java.vm.name = OpenJDK 64-Bit Server VM
java.vm.specification.version = 17
java.specification.version = 17
os.arch = riscv64
os.name = Linux
os.version = 6.12.74-0.0.0.0.riscv64
Data Length = 64 MB
Trials = 5
Direct Buffer Performance Table (bpc: byte-per-crc in MB/sec; #T: #Theads)
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 32 | 1 | 52.5 | 161.6 | 207.8% | 187.5 | 16.0% |
185.6 | -1.0% | 450.3 | 142.7% | 70.4 | -84.4% |
| 32 | 2 | 52.8 | 162.1 | 206.9% | 185.1 | 14.2% |
185.4 | 0.2% | 505.0 | 172.4% | 70.3 | -86.1% |
| 32 | 4 | 52.5 | 161.3 | 206.9% | 186.6 | 15.7% |
184.0 | -1.4% | 471.6 | 156.3% | 70.5 | -85.1% |
| 32 | 8 | 52.6 | 122.7 | 133.4% | 186.2 | 51.7% |
184.8 | -0.7% | 485.7 | 162.7% | 70.4 | -85.5% |
| 32 | 16 | 51.9 | 159.8 | 207.8% | 184.9 | 15.7% |
182.9 | -1.1% | 483.9 | 164.6% | 70.3 | -85.5% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 64 | 1 | 97.3 | 271.0 | 178.6% | 294.5 | 8.7% |
296.4 | 0.7% | 285.5 | -3.7% | 717.7 | 151.4% |
| 64 | 2 | 97.8 | 271.4 | 177.4% | 292.3 | 7.7% |
297.3 | 1.7% | 290.0 | -2.5% | 751.1 | 159.0% |
| 64 | 4 | 96.5 | 269.1 | 178.9% | 293.3 | 9.0% |
296.2 | 1.0% | 291.2 | -1.7% | 738.5 | 153.6% |
| 64 | 8 | 96.0 | 270.8 | 182.3% | 290.5 | 7.3% |
295.7 | 1.8% | 293.8 | -0.7% | 752.7 | 156.2% |
| 64 | 16 | 94.8 | 265.6 | 180.1% | 291.7 | 9.9% |
293.7 | 0.7% | 300.5 | 2.3% | 746.5 | 148.4% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 128 | 1 | 165.5 | 406.5 | 145.6% | 415.9 | 2.3% |
422.5 | 1.6% | 2391.2 | 465.9% | 679.0 | -71.6% |
| 128 | 2 | 163.9 | 407.0 | 148.3% | 417.5 | 2.6% |
424.7 | 1.7% | 1645.0 | 287.3% | 627.4 | -61.9% |
| 128 | 4 | 166.1 | 402.7 | 142.4% | 415.9 | 3.3% |
420.4 | 1.1% | 1998.6 | 375.4% | 693.3 | -65.3% |
| 128 | 8 | 148.1 | 393.6 | 165.7% | 404.7 | 2.8% |
417.7 | 3.2% | 2014.1 | 382.2% | 698.0 | -65.3% |
| 128 | 16 | 159.0 | 393.6 | 147.5% | 413.8 | 5.1% |
418.3 | 1.1% | 1878.0 | 349.0% | 672.9 | -64.2% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 256 | 1 | 258.3 | 535.9 | 107.5% | 522.9 | -2.4% |
537.6 | 2.8% | 2924.1 | 443.9% | 2982.3 | 2.0% |
| 256 | 2 | 258.6 | 540.1 | 108.8% | 529.2 | -2.0% |
540.5 | 2.1% | 2890.8 | 434.9% | 3023.4 | 4.6% |
| 256 | 4 | 254.4 | 523.2 | 105.6% | 525.7 | 0.5% |
536.9 | 2.1% | 2350.8 | 337.9% | 2518.3 | 7.1% |
| 256 | 8 | 250.6 | 523.3 | 108.8% | 516.4 | -1.3% |
532.8 | 3.2% | 2176.6 | 308.5% | 2529.9 | 16.2% |
| 256 | 16 | 242.5 | 517.0 | 113.2% | 520.4 | 0.7% |
530.6 | 2.0% | 2334.7 | 340.0% | 2518.4 | 7.9% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 512 | 1 | 366.5 | 642.4 | 75.3% | 614.5 | -4.3% |
632.2 | 2.9% | 3498.9 | 453.5% | 3698.0 | 5.7% |
| 512 | 2 | 374.7 | 638.4 | 70.4% | 617.5 | -3.3% |
615.5 | -0.3% | 2350.3 | 281.9% | 3737.5 | 59.0% |
| 512 | 4 | 360.3 | 624.6 | 73.3% | 614.3 | -1.7% |
622.6 | 1.4% | 2908.9 | 367.2% | 2996.8 | 3.0% |
| 512 | 8 | 335.3 | 624.8 | 86.4% | 605.2 | -3.1% |
620.8 | 2.6% | 2607.8 | 320.1% | 2645.3 | 1.4% |
| 512 | 16 | 323.5 | 621.0 | 92.0% | 605.6 | -2.5% |
618.4 | 2.1% | 2762.1 | 346.6% | 3002.4 | 8.7% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 1024 | 1 | 441.4 | 703.1 | 59.3% | 655.2 | -6.8% |
665.0 | 1.5% | 3036.7 | 356.7% | 3596.7 | 18.4% |
| 1024 | 2 | 451.0 | 698.2 | 54.8% | 658.4 | -5.7% |
668.6 | 1.5% | 3030.4 | 353.3% | 3588.8 | 18.4% |
| 1024 | 4 | 431.2 | 680.3 | 57.8% | 656.7 | -3.5% |
652.2 | -0.7% | 2657.5 | 307.5% | 3025.9 | 13.9% |
| 1024 | 8 | 450.4 | 681.5 | 51.3% | 648.9 | -4.8% |
652.5 | 0.6% | 2653.0 | 306.6% | 2751.4 | 3.7% |
| 1024 | 16 | 439.5 | 681.8 | 55.1% | 643.6 | -5.6% |
641.3 | -0.4% | 2493.0 | 288.7% | 3027.8 | 21.5% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 2048 | 1 | 503.4 | 721.2 | 43.3% | 662.7 | -8.1% |
676.9 | 2.1% | 3179.3 | 369.7% | 3529.7 | 11.0% |
| 2048 | 2 | 504.8 | 718.7 | 42.4% | 664.2 | -7.6% |
680.9 | 2.5% | 3175.8 | 366.4% | 3575.9 | 12.6% |
| 2048 | 4 | 508.9 | 700.5 | 37.6% | 653.3 | -6.7% |
667.2 | 2.1% | 2736.8 | 310.2% | 3070.5 | 12.2% |
| 2048 | 8 | 515.9 | 706.8 | 37.0% | 649.2 | -8.1% |
671.5 | 3.4% | 2577.8 | 283.9% | 2819.4 | 9.4% |
| 2048 | 16 | 500.9 | 703.8 | 40.5% | 655.3 | -6.9% |
668.0 | 1.9% | 2592.2 | 288.0% | 3056.3 | 17.9% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 4096 | 1 | 516.6 | 738.7 | 43.0% | 679.8 | -8.0% |
695.8 | 2.4% | 3673.4 | 427.9% | 4485.6 | 22.1% |
| 4096 | 2 | 524.9 | 744.0 | 41.7% | 683.5 | -8.1% |
702.7 | 2.8% | 3670.5 | 422.4% | 4562.3 | 24.3% |
| 4096 | 4 | 561.4 | 733.6 | 30.7% | 674.0 | -8.1% |
700.3 | 3.9% | 3189.5 | 355.5% | 3617.4 | 13.4% |
| 4096 | 8 | 543.8 | 731.5 | 34.5% | 668.2 | -8.7% |
684.7 | 2.5% | 2952.8 | 331.3% | 3634.1 | 23.1% |
| 4096 | 16 | 553.6 | 727.7 | 31.4% | 675.2 | -7.2% |
688.9 | 2.0% | 3063.0 | 344.6% | 3630.8 | 18.5% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 8192 | 1 | 584.1 | 750.2 | 28.4% | 680.6 | -9.3% |
700.2 | 2.9% | 3973.5 | 467.5% | 5100.4 | 28.4% |
| 8192 | 2 | 592.4 | 752.6 | 27.0% | 687.1 | -8.7% |
705.2 | 2.6% | 3986.6 | 465.3% | 5057.9 | 26.9% |
| 8192 | 4 | 563.8 | 737.5 | 30.8% | 665.8 | -9.7% |
688.5 | 3.4% | 3457.2 | 402.2% | 3954.0 | 14.4% |
| 8192 | 8 | 580.8 | 727.3 | 25.2% | 665.0 | -8.6% |
679.0 | 2.1% | 3202.9 | 371.7% | 3415.2 | 6.6% |
| 8192 | 16 | 574.4 | 731.8 | 27.4% | 675.0 | -7.8% |
685.0 | 1.5% | 3512.6 | 412.8% | 3899.5 | 11.0% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 16384 | 1 | 617.4 | 751.0 | 21.6% | 682.4 | -9.1% |
695.0 | 1.8% | 4139.6 | 495.6% | 5706.5 | 37.9% |
| 16384 | 2 | 611.9 | 755.6 | 23.5% | 687.1 | -9.1% |
703.5 | 2.4% | 4158.4 | 491.1% | 5694.4 | 36.9% |
| 16384 | 4 | 583.7 | 727.0 | 24.5% | 664.8 | -8.6% |
683.0 | 2.7% | 3629.5 | 431.4% | 4325.6 | 19.2% |
| 16384 | 8 | 585.0 | 719.5 | 23.0% | 660.9 | -8.2% |
671.3 | 1.6% | 3784.6 | 463.8% | 4261.1 | 12.6% |
| 16384 | 16 | 589.9 | 721.6 | 22.3% | 662.4 | -8.2% |
676.2 | 2.1% | 3693.9 | 446.3% | 4556.3 | 23.3% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 32768 | 1 | 641.5 | 754.5 | 17.6% | 686.4 | -9.0% |
699.1 | 1.9% | 4389.4 | 527.8% | 6255.9 | 42.5% |
| 32768 | 2 | 579.3 | 756.5 | 30.6% | 689.7 | -8.8% |
704.8 | 2.2% | 4318.7 | 512.8% | 6236.5 | 44.4% |
| 32768 | 4 | 609.3 | 722.7 | 18.6% | 655.0 | -9.4% |
690.3 | 5.4% | 3799.8 | 450.5% | 4543.7 | 19.6% |
| 32768 | 8 | 596.2 | 730.5 | 22.5% | 664.9 | -9.0% |
668.0 | 0.5% | 4009.4 | 500.2% | 4311.6 | 7.5% |
| 32768 | 16 | 594.9 | 725.2 | 21.9% | 651.5 | -10.2% |
671.2 | 3.0% | 3869.9 | 476.6% | 4823.0 | 24.6% |
| bpc | #T || Zip || ZipC | % diff || PureJava | % diff ||
PureJavaC | % diff || Native | % diff || NativeC | % diff |
| 65536 | 1 | 649.6 | 758.6 | 16.8% | 690.7 | -8.9% |
707.2 | 2.4% | 4433.7 | 527.0% | 6060.3 | 36.7% |
| 65536 | 2 | 606.3 | 758.3 | 25.1% | 637.3 | -16.0% |
708.3 | 11.1% | 4437.3 | 526.5% | 6235.9 | 40.5% |
| 65536 | 4 | 601.2 | 723.8 | 20.4% | 660.5 | -8.7% |
679.3 | 2.8% | 3832.5 | 464.2% | 4620.2 | 20.6% |
| 65536 | 8 | 608.1 | 718.5 | 18.2% | 676.1 | -5.9% |
676.5 | 0.1% | 4424.1 | 554.0% | 4624.2 | 4.5% |
| 65536 | 16 | 596.1 | 727.7 | 22.1% | 651.9 | -10.4% |
674.5 | 3.5% | 3911.9 | 480.0% | 4091.1 | 4.6% |
Elapsed 460.2s
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
```
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
