This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 8cba16aca26 [SPARK-39161][BUILD] Upgrade rocksdbjni to 7.2.2
8cba16aca26 is described below
commit 8cba16aca260e145b5620b284122108351343818
Author: yangjie01 <yangji...@baidu.com>
AuthorDate: Thu May 12 10:58:03 2022 -0700
[SPARK-39161][BUILD] Upgrade rocksdbjni to 7.2.2
### What changes were proposed in this pull request?
This PR aims to upgrade RocksDB JNI library from 7.1.2 to 7.2.2.
### Why are the changes needed?
This will bring improvements and bug fix of RocksDB JNI.
- https://github.com/facebook/rocksdb/releases/tag/v7.2.2
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass GA
- The benchmark result :
**Before 7.1.2**
```
[INFO] Running org.apache.spark.util.kvstore.RocksDBBenchmark
count mean min max
95th
dbClose 4 0.285 0.199
0.492 0.492
dbCreation 4 76.967 3.187
300.938 300.938
naturalIndexCreateIterator 1024 0.006 0.002
1.391 0.007
naturalIndexDescendingCreateIterator 1024 0.005 0.005
0.063 0.007
naturalIndexDescendingIteration 1024 0.006 0.004
0.247 0.008
naturalIndexIteration 1024 0.006 0.004
0.053 0.009
randomDeleteIndexed 1024 0.027 0.019
0.277 0.037
randomDeletesNoIndex 1024 0.015 0.013
0.031 0.017
randomUpdatesIndexed 1024 0.081 0.033
30.561 0.085
randomUpdatesNoIndex 1024 0.036 0.033
0.465 0.039
randomWritesIndexed 1024 0.118 0.035
52.557 0.123
randomWritesNoIndex 1024 0.042 0.036
1.439 0.048
refIndexCreateIterator 1024 0.004 0.004
0.017 0.006
refIndexDescendingCreateIterator 1024 0.003 0.003
0.028 0.004
refIndexDescendingIteration 1024 0.006 0.005
0.045 0.008
refIndexIteration 1024 0.007 0.005
0.073 0.009
sequentialDeleteIndexed 1024 0.021 0.018
0.108 0.025
sequentialDeleteNoIndex 1024 0.015 0.012
0.039 0.017
sequentialUpdatesIndexed 1024 0.045 0.039
0.732 0.056
sequentialUpdatesNoIndex 1024 0.042 0.031
0.812 0.054
sequentialWritesIndexed 1024 0.050 0.044
1.881 0.057
sequentialWritesNoIndex 1024 0.039 0.032
2.160 0.042
```
**After 7.2.2**
```
[INFO] Running org.apache.spark.util.kvstore.RocksDBBenchmark
count mean min max
95th
dbClose 4 0.397 0.280
0.719 0.719
dbCreation 4 77.931 3.410
300.959 300.959
naturalIndexCreateIterator 1024 0.006 0.002
1.433 0.007
naturalIndexDescendingCreateIterator 1024 0.005 0.005
0.064 0.007
naturalIndexDescendingIteration 1024 0.006 0.004
0.253 0.009
naturalIndexIteration 1024 0.006 0.004
0.069 0.010
randomDeleteIndexed 1024 0.027 0.019
0.243 0.037
randomDeletesNoIndex 1024 0.015 0.013
0.042 0.017
randomUpdatesIndexed 1024 0.082 0.033
30.571 0.087
randomUpdatesNoIndex 1024 0.034 0.031
0.625 0.037
randomWritesIndexed 1024 0.118 0.034
52.687 0.128
randomWritesNoIndex 1024 0.040 0.034
1.923 0.045
refIndexCreateIterator 1024 0.004 0.004
0.018 0.006
refIndexDescendingCreateIterator 1024 0.003 0.002
0.033 0.004
refIndexDescendingIteration 1024 0.007 0.005
0.047 0.010
refIndexIteration 1024 0.007 0.005
0.077 0.010
sequentialDeleteIndexed 1024 0.021 0.017
0.103 0.025
sequentialDeleteNoIndex 1024 0.015 0.012
0.067 0.017
sequentialUpdatesIndexed 1024 0.042 0.037
0.817 0.052
sequentialUpdatesNoIndex 1024 0.040 0.030
1.067 0.049
sequentialWritesIndexed 1024 0.048 0.039
2.049 0.055
sequentialWritesNoIndex 1024 0.039 0.032
2.421 0.041
```
Closes #36522 from LuciferYang/SPARK-39161.
Authored-by: yangjie01 <yangji...@baidu.com>
Signed-off-by: Dongjoon Hyun <dongj...@apache.org>
---
dev/deps/spark-deps-hadoop-2-hive-2.3 | 2 +-
dev/deps/spark-deps-hadoop-3-hive-2.3 | 2 +-
pom.xml | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/dev/deps/spark-deps-hadoop-2-hive-2.3
b/dev/deps/spark-deps-hadoop-2-hive-2.3
index 091ef889d9d..18a090c8db9 100644
--- a/dev/deps/spark-deps-hadoop-2-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-2-hive-2.3
@@ -235,7 +235,7 @@ pickle/1.2//pickle-1.2.jar
protobuf-java/2.5.0//protobuf-java-2.5.0.jar
py4j/0.10.9.5//py4j-0.10.9.5.jar
remotetea-oncrpc/1.1.2//remotetea-oncrpc-1.1.2.jar
-rocksdbjni/7.1.2//rocksdbjni-7.1.2.jar
+rocksdbjni/7.2.2//rocksdbjni-7.2.2.jar
scala-collection-compat_2.12/2.1.1//scala-collection-compat_2.12-2.1.1.jar
scala-compiler/2.12.15//scala-compiler-2.12.15.jar
scala-library/2.12.15//scala-library-2.12.15.jar
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3
b/dev/deps/spark-deps-hadoop-3-hive-2.3
index 88df21498b4..f187a671d95 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -224,7 +224,7 @@ pickle/1.2//pickle-1.2.jar
protobuf-java/2.5.0//protobuf-java-2.5.0.jar
py4j/0.10.9.5//py4j-0.10.9.5.jar
remotetea-oncrpc/1.1.2//remotetea-oncrpc-1.1.2.jar
-rocksdbjni/7.1.2//rocksdbjni-7.1.2.jar
+rocksdbjni/7.2.2//rocksdbjni-7.2.2.jar
scala-collection-compat_2.12/2.1.1//scala-collection-compat_2.12-2.1.1.jar
scala-compiler/2.12.15//scala-compiler-2.12.15.jar
scala-library/2.12.15//scala-library-2.12.15.jar
diff --git a/pom.xml b/pom.xml
index 68bd7cd5cb0..2ec0ea3ee7c 100644
--- a/pom.xml
+++ b/pom.xml
@@ -675,7 +675,7 @@
<dependency>
<groupId>org.rocksdb</groupId>
<artifactId>rocksdbjni</artifactId>
- <version>7.1.2</version>
+ <version>7.2.2</version>
</dependency>
<dependency>
<groupId>${leveldbjni.group}</groupId>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
