This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new bd9a4b9b3863 [SPARK-46528][BUILD] Upgrade `zstd-jni` to 1.5.5-11
bd9a4b9b3863 is described below
commit bd9a4b9b38633005d583ccc3f425b4f05aac92ba
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Thu Dec 28 09:23:48 2023 +0900
[SPARK-46528][BUILD] Upgrade `zstd-jni` to 1.5.5-11
### What changes were proposed in this pull request?
This PR aims to update `zstd-jni` to 1.5.5-11.
### Why are the changes needed?
This version has a few bug fixes and the following improvement.
- https://github.com/luben/zstd-jni/pull/287
Full commit list.
- https://github.com/luben/zstd-jni/compare/v1.5.5-10...v1.5.5-11
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #44515 from dongjoon-hyun/SPARK-46528.
Authored-by: Dongjoon Hyun <dh...@apple.com>
Signed-off-by: Hyukjin Kwon <gurwls...@apache.org>
---
.../ZStandardBenchmark-jdk21-results.txt | 56 +++++++++++-----------
core/benchmarks/ZStandardBenchmark-results.txt | 56 +++++++++++-----------
dev/deps/spark-deps-hadoop-3-hive-2.3 | 2 +-
pom.xml | 2 +-
4 files changed, 58 insertions(+), 58 deletions(-)
diff --git a/core/benchmarks/ZStandardBenchmark-jdk21-results.txt
b/core/benchmarks/ZStandardBenchmark-jdk21-results.txt
index f5383af55c3a..e0928df7f3c6 100644
--- a/core/benchmarks/ZStandardBenchmark-jdk21-results.txt
+++ b/core/benchmarks/ZStandardBenchmark-jdk21-results.txt
@@ -2,48 +2,48 @@
Benchmark ZStandardCompressionCodec
================================================================================================
-OpenJDK 64-Bit Server VM 21.0.1+12-LTS on Linux 5.15.0-1051-azure
+OpenJDK 64-Bit Server VM 21.0.1+12-LTS on Linux 5.15.0-1053-azure
AMD EPYC 7763 64-Core Processor
Benchmark ZStandardCompressionCodec: Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------
-Compression 10000 times at level 1 without buffer pool 672
681 10 0.0 67171.0 1.0X
-Compression 10000 times at level 2 without buffer pool 715
718 4 0.0 71458.8 0.9X
-Compression 10000 times at level 3 without buffer pool 831
835 4 0.0 83139.1 0.8X
-Compression 10000 times at level 1 with buffer pool 609
611 2 0.0 60881.5 1.1X
-Compression 10000 times at level 2 with buffer pool 648
649 1 0.0 64791.0 1.0X
-Compression 10000 times at level 3 with buffer pool 744
751 6 0.0 74392.4 0.9X
+Compression 10000 times at level 1 without buffer pool 674
920 293 0.0 67406.4 1.0X
+Compression 10000 times at level 2 without buffer pool 882
884 3 0.0 88195.1 0.8X
+Compression 10000 times at level 3 without buffer pool 973
978 4 0.0 97301.3 0.7X
+Compression 10000 times at level 1 with buffer pool 955
955 1 0.0 95452.0 0.7X
+Compression 10000 times at level 2 with buffer pool 994
996 2 0.0 99432.1 0.7X
+Compression 10000 times at level 3 with buffer pool 1093
1101 12 0.0 109300.9 0.6X
-OpenJDK 64-Bit Server VM 21.0.1+12-LTS on Linux 5.15.0-1051-azure
+OpenJDK 64-Bit Server VM 21.0.1+12-LTS on Linux 5.15.0-1053-azure
AMD EPYC 7763 64-Core Processor
Benchmark ZStandardCompressionCodec: Best Time(ms)
Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------
-Decompression 10000 times from level 1 without buffer pool 842
849 12 0.0 84240.0 1.0X
-Decompression 10000 times from level 2 without buffer pool 842
846 6 0.0 84185.2 1.0X
-Decompression 10000 times from level 3 without buffer pool 843
844 1 0.0 84285.4 1.0X
-Decompression 10000 times from level 1 with buffer pool 770
771 1 0.0 77024.9 1.1X
-Decompression 10000 times from level 2 with buffer pool 771
771 0 0.0 77120.4 1.1X
-Decompression 10000 times from level 3 with buffer pool 770
771 0 0.0 77031.9 1.1X
+Decompression 10000 times from level 1 without buffer pool 826
829 3 0.0 82591.4 1.0X
+Decompression 10000 times from level 2 without buffer pool 825
826 1 0.0 82533.4 1.0X
+Decompression 10000 times from level 3 without buffer pool 827
830 5 0.0 82715.3 1.0X
+Decompression 10000 times from level 1 with buffer pool 763
764 1 0.0 76271.6 1.1X
+Decompression 10000 times from level 2 with buffer pool 763
777 23 0.0 76321.2 1.1X
+Decompression 10000 times from level 3 with buffer pool 763
765 2 0.0 76286.1 1.1X
-OpenJDK 64-Bit Server VM 21.0.1+12-LTS on Linux 5.15.0-1051-azure
+OpenJDK 64-Bit Server VM 21.0.1+12-LTS on Linux 5.15.0-1053-azure
AMD EPYC 7763 64-Core Processor
Parallel Compression at level 3: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parallel Compression with 0 workers 48 50
3 0.0 376597.0 1.0X
-Parallel Compression with 1 workers 41 42
3 0.0 318927.3 1.2X
-Parallel Compression with 2 workers 38 40
2 0.0 297410.2 1.3X
-Parallel Compression with 4 workers 37 39
1 0.0 287605.8 1.3X
-Parallel Compression with 8 workers 39 40
1 0.0 301948.1 1.2X
-Parallel Compression with 16 workers 41 43
1 0.0 317095.6 1.2X
+Parallel Compression with 0 workers 49 50
1 0.0 384188.1 1.0X
+Parallel Compression with 1 workers 42 44
4 0.0 328139.4 1.2X
+Parallel Compression with 2 workers 40 42
1 0.0 309013.2 1.2X
+Parallel Compression with 4 workers 40 41
1 0.0 309732.2 1.2X
+Parallel Compression with 8 workers 41 43
2 0.0 319730.2 1.2X
+Parallel Compression with 16 workers 43 45
1 0.0 337944.2 1.1X
-OpenJDK 64-Bit Server VM 21.0.1+12-LTS on Linux 5.15.0-1051-azure
+OpenJDK 64-Bit Server VM 21.0.1+12-LTS on Linux 5.15.0-1053-azure
AMD EPYC 7763 64-Core Processor
Parallel Compression at level 9: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parallel Compression with 0 workers 174 175
1 0.0 1360596.3 1.0X
-Parallel Compression with 1 workers 189 228
24 0.0 1477060.7 0.9X
-Parallel Compression with 2 workers 109 118
15 0.0 851455.9 1.6X
-Parallel Compression with 4 workers 114 118
3 0.0 891964.9 1.5X
-Parallel Compression with 8 workers 115 122
4 0.0 899748.7 1.5X
-Parallel Compression with 16 workers 119 123
2 0.0 931210.7 1.5X
+Parallel Compression with 0 workers 160 161
1 0.0 1250203.7 1.0X
+Parallel Compression with 1 workers 196 197
2 0.0 1529028.2 0.8X
+Parallel Compression with 2 workers 114 121
10 0.0 892592.4 1.4X
+Parallel Compression with 4 workers 111 113
1 0.0 865617.7 1.4X
+Parallel Compression with 8 workers 112 117
2 0.0 878723.8 1.4X
+Parallel Compression with 16 workers 114 117
2 0.0 889199.7 1.4X
diff --git a/core/benchmarks/ZStandardBenchmark-results.txt
b/core/benchmarks/ZStandardBenchmark-results.txt
index 64375b7a379a..bbedcda0f160 100644
--- a/core/benchmarks/ZStandardBenchmark-results.txt
+++ b/core/benchmarks/ZStandardBenchmark-results.txt
@@ -2,48 +2,48 @@
Benchmark ZStandardCompressionCodec
================================================================================================
-OpenJDK 64-Bit Server VM 17.0.9+9-LTS on Linux 5.15.0-1051-azure
+OpenJDK 64-Bit Server VM 17.0.9+9-LTS on Linux 5.15.0-1053-azure
AMD EPYC 7763 64-Core Processor
Benchmark ZStandardCompressionCodec: Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------
-Compression 10000 times at level 1 without buffer pool 666
669 3 0.0 66598.6 1.0X
-Compression 10000 times at level 2 without buffer pool 711
711 1 0.0 71077.5 0.9X
-Compression 10000 times at level 3 without buffer pool 816
816 0 0.0 81575.8 0.8X
-Compression 10000 times at level 1 with buffer pool 591
592 1 0.0 59095.6 1.1X
-Compression 10000 times at level 2 with buffer pool 630
632 1 0.0 62995.1 1.1X
-Compression 10000 times at level 3 with buffer pool 742
742 0 0.0 74180.7 0.9X
+Compression 10000 times at level 1 without buffer pool 666
669 3 0.0 66613.9 1.0X
+Compression 10000 times at level 2 without buffer pool 708
709 1 0.0 70817.8 0.9X
+Compression 10000 times at level 3 without buffer pool 818
819 1 0.0 81828.4 0.8X
+Compression 10000 times at level 1 with buffer pool 591
596 9 0.0 59119.9 1.1X
+Compression 10000 times at level 2 with buffer pool 629
630 1 0.0 62943.8 1.1X
+Compression 10000 times at level 3 with buffer pool 746
747 2 0.0 74593.2 0.9X
-OpenJDK 64-Bit Server VM 17.0.9+9-LTS on Linux 5.15.0-1051-azure
+OpenJDK 64-Bit Server VM 17.0.9+9-LTS on Linux 5.15.0-1053-azure
AMD EPYC 7763 64-Core Processor
Benchmark ZStandardCompressionCodec: Best Time(ms)
Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------
-Decompression 10000 times from level 1 without buffer pool 600
602 1 0.0 60024.1 1.0X
-Decompression 10000 times from level 2 without buffer pool 600
603 2 0.0 59973.0 1.0X
-Decompression 10000 times from level 3 without buffer pool 601
602 1 0.0 60075.9 1.0X
-Decompression 10000 times from level 1 with buffer pool 553
553 0 0.0 55316.4 1.1X
-Decompression 10000 times from level 2 with buffer pool 553
554 1 0.0 55271.5 1.1X
-Decompression 10000 times from level 3 with buffer pool 553
553 0 0.0 55261.4 1.1X
+Decompression 10000 times from level 1 without buffer pool 600
603 2 0.0 60027.9 1.0X
+Decompression 10000 times from level 2 without buffer pool 603
604 1 0.0 60270.8 1.0X
+Decompression 10000 times from level 3 without buffer pool 602
604 2 0.0 60224.9 1.0X
+Decompression 10000 times from level 1 with buffer pool 548
548 0 0.0 54774.5 1.1X
+Decompression 10000 times from level 2 with buffer pool 548
548 1 0.0 54763.3 1.1X
+Decompression 10000 times from level 3 with buffer pool 548
548 0 0.0 54751.8 1.1X
-OpenJDK 64-Bit Server VM 17.0.9+9-LTS on Linux 5.15.0-1051-azure
+OpenJDK 64-Bit Server VM 17.0.9+9-LTS on Linux 5.15.0-1053-azure
AMD EPYC 7763 64-Core Processor
Parallel Compression at level 3: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parallel Compression with 0 workers 49 50
1 0.0 380070.1 1.0X
-Parallel Compression with 1 workers 41 42
4 0.0 319807.1 1.2X
-Parallel Compression with 2 workers 38 41
2 0.0 297706.4 1.3X
-Parallel Compression with 4 workers 38 40
2 0.0 296505.8 1.3X
-Parallel Compression with 8 workers 39 41
1 0.0 305793.6 1.2X
-Parallel Compression with 16 workers 43 44
1 0.0 332233.1 1.1X
+Parallel Compression with 0 workers 49 50
1 0.0 382234.6 1.0X
+Parallel Compression with 1 workers 42 43
1 0.0 327419.1 1.2X
+Parallel Compression with 2 workers 38 41
2 0.0 299638.6 1.3X
+Parallel Compression with 4 workers 38 40
2 0.0 294671.3 1.3X
+Parallel Compression with 8 workers 40 42
1 0.0 308641.0 1.2X
+Parallel Compression with 16 workers 43 45
2 0.0 335800.5 1.1X
-OpenJDK 64-Bit Server VM 17.0.9+9-LTS on Linux 5.15.0-1051-azure
+OpenJDK 64-Bit Server VM 17.0.9+9-LTS on Linux 5.15.0-1053-azure
AMD EPYC 7763 64-Core Processor
Parallel Compression at level 9: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parallel Compression with 0 workers 175 175
0 0.0 1363800.8 1.0X
-Parallel Compression with 1 workers 186 187
3 0.0 1455096.4 0.9X
-Parallel Compression with 2 workers 110 115
6 0.0 863272.6 1.6X
-Parallel Compression with 4 workers 104 108
2 0.0 810721.1 1.7X
-Parallel Compression with 8 workers 110 112
2 0.0 859303.5 1.6X
-Parallel Compression with 16 workers 109 112
2 0.0 847863.6 1.6X
+Parallel Compression with 0 workers 159 161
1 0.0 1242039.9 1.0X
+Parallel Compression with 1 workers 201 203
4 0.0 1568116.4 0.8X
+Parallel Compression with 2 workers 115 122
12 0.0 900801.9 1.4X
+Parallel Compression with 4 workers 111 114
3 0.0 868535.8 1.4X
+Parallel Compression with 8 workers 113 117
2 0.0 886200.6 1.4X
+Parallel Compression with 16 workers 113 117
2 0.0 880790.0 1.4X
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3
b/dev/deps/spark-deps-hadoop-3-hive-2.3
index 1ed23f257d73..bebc63bae72b 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -267,4 +267,4 @@ xz/1.9//xz-1.9.jar
zjsonpatch/0.3.0//zjsonpatch-0.3.0.jar
zookeeper-jute/3.9.1//zookeeper-jute-3.9.1.jar
zookeeper/3.9.1//zookeeper-3.9.1.jar
-zstd-jni/1.5.5-10//zstd-jni-1.5.5-10.jar
+zstd-jni/1.5.5-11//zstd-jni-1.5.5-11.jar
diff --git a/pom.xml b/pom.xml
index 09a1d3d49998..d2f8ff5777e7 100644
--- a/pom.xml
+++ b/pom.xml
@@ -800,7 +800,7 @@
<dependency>
<groupId>com.github.luben</groupId>
<artifactId>zstd-jni</artifactId>
- <version>1.5.5-10</version>
+ <version>1.5.5-11</version>
</dependency>
<dependency>
<groupId>com.clearspring.analytics</groupId>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
