This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new e79b9f05360 [SPARK-39882][BUILD] Upgrade rocksdbjni to 7.4.3 e79b9f05360 is described below commit e79b9f053605369a60f15c606f1a4bc0b4f31329 Author: yangjie01 <yangji...@baidu.com> AuthorDate: Tue Jul 26 21:52:23 2022 -0700 [SPARK-39882][BUILD] Upgrade rocksdbjni to 7.4.3 ### What changes were proposed in this pull request? This PR aims to upgrade RocksDB JNI library from 7.3.1 to 7.4.3. ### Why are the changes needed? This version brings some bug fix, the release note as follows: - https://github.com/facebook/rocksdb/releases/tag/v7.4.3 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Pass GA - The benchmark result : **Before 7.3.1** ``` [INFO] Running org.apache.spark.util.kvstore.RocksDBBenchmark count mean min max 95th dbClose 4 0.346 0.266 0.539 0.539 dbCreation 4 78.389 3.583 301.763 301.763 naturalIndexCreateIterator 1024 0.005 0.002 1.469 0.007 naturalIndexDescendingCreateIterator 1024 0.007 0.006 0.064 0.008 naturalIndexDescendingIteration 1024 0.005 0.004 0.024 0.007 naturalIndexIteration 1024 0.006 0.004 0.053 0.008 randomDeleteIndexed 1024 0.026 0.019 0.293 0.035 randomDeletesNoIndex 1024 0.015 0.012 0.053 0.017 randomUpdatesIndexed 1024 0.081 0.033 30.752 0.083 randomUpdatesNoIndex 1024 0.037 0.034 0.502 0.041 randomWritesIndexed 1024 0.116 0.035 52.128 0.121 randomWritesNoIndex 1024 0.042 0.036 1.869 0.046 refIndexCreateIterator 1024 0.004 0.004 0.019 0.006 refIndexDescendingCreateIterator 1024 0.003 0.002 0.026 0.004 refIndexDescendingIteration 1024 0.006 0.005 0.042 0.008 refIndexIteration 1024 0.007 0.005 0.314 0.009 sequentialDeleteIndexed 1024 0.021 0.018 0.104 0.026 sequentialDeleteNoIndex 1024 0.015 0.012 0.053 0.019 sequentialUpdatesIndexed 1024 0.044 0.038 0.802 0.053 sequentialUpdatesNoIndex 1024 0.045 0.032 1.552 0.054 sequentialWritesIndexed 1024 0.048 0.040 1.862 0.055 sequentialWritesNoIndex 1024 0.044 0.032 2.929 0.048 ``` **After 7.4.3** ``` [INFO] Running org.apache.spark.util.kvstore.RocksDBBenchmark count mean min max 95th dbClose 4 0.364 0.305 0.525 0.525 dbCreation 4 80.553 3.635 310.735 310.735 naturalIndexCreateIterator 1024 0.006 0.002 1.569 0.007 naturalIndexDescendingCreateIterator 1024 0.006 0.005 0.073 0.008 naturalIndexDescendingIteration 1024 0.006 0.004 0.269 0.008 naturalIndexIteration 1024 0.006 0.004 0.059 0.009 randomDeleteIndexed 1024 0.027 0.020 0.339 0.036 randomDeletesNoIndex 1024 0.015 0.012 0.037 0.018 randomUpdatesIndexed 1024 0.084 0.032 32.286 0.090 randomUpdatesNoIndex 1024 0.023 0.019 0.584 0.026 randomWritesIndexed 1024 0.122 0.035 53.989 0.129 randomWritesNoIndex 1024 0.027 0.022 1.547 0.031 refIndexCreateIterator 1024 0.005 0.005 0.027 0.007 refIndexDescendingCreateIterator 1024 0.003 0.003 0.035 0.005 refIndexDescendingIteration 1024 0.006 0.005 0.051 0.008 refIndexIteration 1024 0.007 0.005 0.096 0.009 sequentialDeleteIndexed 1024 0.023 0.018 1.396 0.027 sequentialDeleteNoIndex 1024 0.015 0.012 0.054 0.019 sequentialUpdatesIndexed 1024 0.045 0.040 0.773 0.053 sequentialUpdatesNoIndex 1024 0.027 0.021 0.867 0.034 sequentialWritesIndexed 1024 0.051 0.039 1.916 0.059 sequentialWritesNoIndex 1024 0.031 0.024 3.339 0.033 ``` Closes #37302 from LuciferYang/SPARK-39882. Authored-by: yangjie01 <yangji...@baidu.com> Signed-off-by: Dongjoon Hyun <dongj...@apache.org> --- dev/deps/spark-deps-hadoop-2-hive-2.3 | 2 +- dev/deps/spark-deps-hadoop-3-hive-2.3 | 2 +- pom.xml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/dev/deps/spark-deps-hadoop-2-hive-2.3 b/dev/deps/spark-deps-hadoop-2-hive-2.3 index d8521647522..f26fb36675b 100644 --- a/dev/deps/spark-deps-hadoop-2-hive-2.3 +++ b/dev/deps/spark-deps-hadoop-2-hive-2.3 @@ -234,7 +234,7 @@ pickle/1.2//pickle-1.2.jar protobuf-java/2.5.0//protobuf-java-2.5.0.jar py4j/0.10.9.5//py4j-0.10.9.5.jar remotetea-oncrpc/1.1.2//remotetea-oncrpc-1.1.2.jar -rocksdbjni/7.3.1//rocksdbjni-7.3.1.jar +rocksdbjni/7.4.3//rocksdbjni-7.4.3.jar scala-collection-compat_2.12/2.2.0//scala-collection-compat_2.12-2.2.0.jar scala-compiler/2.12.16//scala-compiler-2.12.16.jar scala-library/2.12.16//scala-library-2.12.16.jar diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3 b/dev/deps/spark-deps-hadoop-3-hive-2.3 index c50d8c66c21..2bc30a9f523 100644 --- a/dev/deps/spark-deps-hadoop-3-hive-2.3 +++ b/dev/deps/spark-deps-hadoop-3-hive-2.3 @@ -223,7 +223,7 @@ pickle/1.2//pickle-1.2.jar protobuf-java/2.5.0//protobuf-java-2.5.0.jar py4j/0.10.9.5//py4j-0.10.9.5.jar remotetea-oncrpc/1.1.2//remotetea-oncrpc-1.1.2.jar -rocksdbjni/7.3.1//rocksdbjni-7.3.1.jar +rocksdbjni/7.4.3//rocksdbjni-7.4.3.jar scala-collection-compat_2.12/2.2.0//scala-collection-compat_2.12-2.2.0.jar scala-compiler/2.12.16//scala-compiler-2.12.16.jar scala-library/2.12.16//scala-library-2.12.16.jar diff --git a/pom.xml b/pom.xml index 40d82261e43..34dbbf8b46d 100644 --- a/pom.xml +++ b/pom.xml @@ -678,7 +678,7 @@ <dependency> <groupId>org.rocksdb</groupId> <artifactId>rocksdbjni</artifactId> - <version>7.3.1</version> + <version>7.4.3</version> </dependency> <dependency> <groupId>${leveldbjni.group}</groupId> --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org