This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new ab6338e09aa0 [SPARK-47838][BUILD] Upgrade `rocksdbjni` to 8.11.4
ab6338e09aa0 is described below
commit ab6338e09aa0fe06aef1c753eaaf677f766e9490
Author: Neil Ramaswamy <neil.ramasw...@databricks.com>
AuthorDate: Tue Apr 16 20:11:16 2024 -0700
[SPARK-47838][BUILD] Upgrade `rocksdbjni` to 8.11.4
### What changes were proposed in this pull request?
Upgrades `rocksdbjni` dependency to 8.11.4.
### Why are the changes needed?
8.11.4 has Java-related RocksDB fixes:
https://github.com/facebook/rocksdb/releases/tag/v8.11.4
- Fixed CMake Javadoc build
- Fixed Java SstFileMetaData to prevent throwing java.lang.NoSuchMethodError
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
- All existing UTs should pass
- [In progress] Performance benchmarks
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #46065 from neilramaswamy/spark-47838.
Authored-by: Neil Ramaswamy <neil.ramasw...@databricks.com>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
dev/deps/spark-deps-hadoop-3-hive-2.3 | 2 +-
pom.xml | 2 +-
...StoreBasicOperationsBenchmark-jdk21-results.txt | 122 +++++++++++----------
.../StateStoreBasicOperationsBenchmark-results.txt | 122 +++++++++++----------
4 files changed, 126 insertions(+), 122 deletions(-)
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3
b/dev/deps/spark-deps-hadoop-3-hive-2.3
index 466e8d09d89e..54e54a108904 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -247,7 +247,7 @@ parquet-jackson/1.13.1//parquet-jackson-1.13.1.jar
pickle/1.3//pickle-1.3.jar
py4j/0.10.9.7//py4j-0.10.9.7.jar
remotetea-oncrpc/1.1.2//remotetea-oncrpc-1.1.2.jar
-rocksdbjni/8.11.3//rocksdbjni-8.11.3.jar
+rocksdbjni/8.11.4//rocksdbjni-8.11.4.jar
scala-collection-compat_2.13/2.7.0//scala-collection-compat_2.13-2.7.0.jar
scala-compiler/2.13.13//scala-compiler-2.13.13.jar
scala-library/2.13.13//scala-library-2.13.13.jar
diff --git a/pom.xml b/pom.xml
index bf8d4f1b417d..7ded74b9f9df 100644
--- a/pom.xml
+++ b/pom.xml
@@ -687,7 +687,7 @@
<dependency>
<groupId>org.rocksdb</groupId>
<artifactId>rocksdbjni</artifactId>
- <version>8.11.3</version>
+ <version>8.11.4</version>
</dependency>
<dependency>
<groupId>${leveldbjni.group}</groupId>
diff --git
a/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-jdk21-results.txt
b/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-jdk21-results.txt
index 0317e6116375..953031fc1daf 100644
--- a/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-jdk21-results.txt
+++ b/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-jdk21-results.txt
@@ -2,141 +2,143 @@
put rows
================================================================================================
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
putting 10000 rows (10000 rows to overwrite - rate 100): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
10 1 1.1 936.2 1.0X
-RocksDB (trackTotalNumberOfRows: true) 41
42 1 0.2 4068.9 0.2X
-RocksDB (trackTotalNumberOfRows: false) 15
16 1 0.7 1500.4 0.6X
+In-memory 9
10 1 1.1 938.9 1.0X
+RocksDB (trackTotalNumberOfRows: true) 42
44 2 0.2 4215.2 0.2X
+RocksDB (trackTotalNumberOfRows: false) 15
16 1 0.7 1535.3 0.6X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
putting 10000 rows (5000 rows to overwrite - rate 50): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
11 1 1.1 929.8 1.0X
-RocksDB (trackTotalNumberOfRows: true) 40
41 1 0.3 3955.7 0.2X
-RocksDB (trackTotalNumberOfRows: false) 15
16 1 0.7 1497.3 0.6X
+In-memory 9
11 1 1.1 943.3 1.0X
+RocksDB (trackTotalNumberOfRows: true) 41
43 2 0.2 4083.3 0.2X
+RocksDB (trackTotalNumberOfRows: false) 15
16 1 0.7 1534.4 0.6X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
putting 10000 rows (1000 rows to overwrite - rate 10): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
10 1 1.1 907.5 1.0X
-RocksDB (trackTotalNumberOfRows: true) 39
40 1 0.3 3886.5 0.2X
-RocksDB (trackTotalNumberOfRows: false) 15
16 1 0.7 1497.2 0.6X
+In-memory 9
10 1 1.1 909.9 1.0X
+RocksDB (trackTotalNumberOfRows: true) 40
41 1 0.2 4017.6 0.2X
+RocksDB (trackTotalNumberOfRows: false) 15
16 1 0.7 1533.0 0.6X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
putting 10000 rows (0 rows to overwrite - rate 0): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
10 1 1.1 904.0 1.0X
-RocksDB (trackTotalNumberOfRows: true) 39
40 1 0.3 3859.8 0.2X
-RocksDB (trackTotalNumberOfRows: false) 15
16 0 0.7 1497.2 0.6X
+In-memory 9
10 0 1.1 900.2 1.0X
+RocksDB (trackTotalNumberOfRows: true) 40
42 1 0.3 3993.3 0.2X
+RocksDB (trackTotalNumberOfRows: false) 15
16 1 0.6 1547.1 0.6X
================================================================================================
merge rows
================================================================================================
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
merging 10000 rows with 10 values per key (10000 rows to overwrite - rate
100): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
--------------------------------------------------------------------------------------------------------------------------------------------------------------
-RocksDB (trackTotalNumberOfRows: true)
519 533 7 0.0 51916.6 1.0X
-RocksDB (trackTotalNumberOfRows: false)
171 177 3 0.1 17083.9 3.0X
+RocksDB (trackTotalNumberOfRows: true)
544 561 6 0.0 54379.3 1.0X
+RocksDB (trackTotalNumberOfRows: false)
178 184 3 0.1 17764.5 3.1X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
merging 10000 rows with 10 values per key (5000 rows to overwrite - rate 50):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------------
-RocksDB (trackTotalNumberOfRows: true)
506 521 7 0.0 50644.0 1.0X
-RocksDB (trackTotalNumberOfRows: false)
170 176 3 0.1 17022.0 3.0X
+RocksDB (trackTotalNumberOfRows: true)
528 543 6 0.0 52813.5 1.0X
+RocksDB (trackTotalNumberOfRows: false)
179 184 3 0.1 17896.3 3.0X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
merging 10000 rows with 10 values per key (1000 rows to overwrite - rate 10):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------------
-RocksDB (trackTotalNumberOfRows: true)
493 508 6 0.0 49319.3 1.0X
-RocksDB (trackTotalNumberOfRows: false)
169 175 3 0.1 16897.6 2.9X
+RocksDB (trackTotalNumberOfRows: true)
515 530 6 0.0 51526.1 1.0X
+RocksDB (trackTotalNumberOfRows: false)
178 183 3 0.1 17774.8 2.9X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
merging 10000 rows with 10 values per key (0 rows to overwrite - rate 0):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------------------------
-RocksDB (trackTotalNumberOfRows: true)
495 508 6 0.0 49462.5 1.0X
-RocksDB (trackTotalNumberOfRows: false)
169 175 3 0.1 16896.6 2.9X
+RocksDB (trackTotalNumberOfRows: true)
513 529 6 0.0 51333.1 1.0X
+RocksDB (trackTotalNumberOfRows: false)
177 183 3 0.1 17713.4 2.9X
================================================================================================
delete rows
================================================================================================
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
trying to delete 10000 rows from 10000 rows(10000 rows are non-existing - rate
100): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
0 1 0 26.3 38.0
1.0X
-RocksDB (trackTotalNumberOfRows: true)
39 41 1 0.3 3942.0
0.0X
-RocksDB (trackTotalNumberOfRows: false)
15 16 1 0.7 1529.2
0.0X
+In-memory
0 1 0 22.0 45.5
1.0X
+RocksDB (trackTotalNumberOfRows: true)
40 42 1 0.2 4030.3
0.0X
+RocksDB (trackTotalNumberOfRows: false)
16 17 1 0.6 1576.1
0.0X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
trying to delete 10000 rows from 10000 rows(5000 rows are non-existing - rate
50): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 9 1 1.3 790.4
1.0X
-RocksDB (trackTotalNumberOfRows: true)
40 41 1 0.2 4036.7
0.2X
-RocksDB (trackTotalNumberOfRows: false)
15 16 0 0.7 1536.9
0.5X
+In-memory
8 9 0 1.3 797.7
1.0X
+RocksDB (trackTotalNumberOfRows: true)
42 43 1 0.2 4184.7
0.2X
+RocksDB (trackTotalNumberOfRows: false)
16 16 0 0.6 1579.2
0.5X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
trying to delete 10000 rows from 10000 rows(1000 rows are non-existing - rate
10): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 10 1 1.2 847.0
1.0X
-RocksDB (trackTotalNumberOfRows: true)
41 42 1 0.2 4099.8
0.2X
-RocksDB (trackTotalNumberOfRows: false)
16 16 0 0.6 1563.3
0.5X
+In-memory
8 9 0 1.2 848.5
1.0X
+RocksDB (trackTotalNumberOfRows: true)
43 44 1 0.2 4268.3
0.2X
+RocksDB (trackTotalNumberOfRows: false)
16 17 0 0.6 1582.7
0.5X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
trying to delete 10000 rows from 10000 rows(0 rows are non-existing - rate 0):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
9 10 1 1.2 859.4 1.0X
-RocksDB (trackTotalNumberOfRows: true)
41 42 1 0.2 4118.9 0.2X
-RocksDB (trackTotalNumberOfRows: false)
15 16 1 0.7 1507.8 0.6X
+In-memory
9 9 0 1.2 856.7 1.0X
+RocksDB (trackTotalNumberOfRows: true)
42 44 1 0.2 4244.3 0.2X
+RocksDB (trackTotalNumberOfRows: false)
16 16 0 0.6 1570.4 0.5X
================================================================================================
evict rows
================================================================================================
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
evicting 10000 rows (maxTimestampToEvictInMillis: 9999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 9 1 1.2 831.0 1.0X
-RocksDB (trackTotalNumberOfRows: true)
40 40 1 0.3 3956.6 0.2X
-RocksDB (trackTotalNumberOfRows: false)
16 16 0 0.6 1571.3 0.5X
+In-memory
8 9 0 1.2 829.5 1.0X
+RocksDB (trackTotalNumberOfRows: true)
41 42 1 0.2 4110.7 0.2X
+RocksDB (trackTotalNumberOfRows: false)
17 17 0 0.6 1652.2 0.5X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
evicting 5000 rows (maxTimestampToEvictInMillis: 4999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 8 1 1.3 787.6 1.0X
-RocksDB (trackTotalNumberOfRows: true)
21 22 0 0.5 2112.6 0.4X
-RocksDB (trackTotalNumberOfRows: false)
9 9 0 1.1 932.9 0.8X
+In-memory
8 8 0 1.3 782.2 1.0X
+RocksDB (trackTotalNumberOfRows: true)
22 23 0 0.5 2207.2 0.4X
+RocksDB (trackTotalNumberOfRows: false)
10 10 0 1.0 955.0 0.8X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
evicting 1000 rows (maxTimestampToEvictInMillis: 999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
7 8 0 1.4 715.7 1.0X
-RocksDB (trackTotalNumberOfRows: true)
7 7 0 1.5 676.3 1.1X
-RocksDB (trackTotalNumberOfRows: false)
4 5 0 2.3 442.3 1.6X
+In-memory
7 8 0 1.4 729.9 1.0X
+RocksDB (trackTotalNumberOfRows: true)
7 7 0 1.4 710.1 1.0X
+RocksDB (trackTotalNumberOfRows: false)
5 5 0 2.2 465.1 1.6X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
evicting 0 rows (maxTimestampToEvictInMillis: -1) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
0 0 0 23.8 41.9 1.0X
-RocksDB (trackTotalNumberOfRows: true)
3 3 0 3.2 309.5 0.1X
-RocksDB (trackTotalNumberOfRows: false)
3 3 0 3.2 309.9 0.1X
+In-memory
0 0 0 24.0 41.7 1.0X
+RocksDB (trackTotalNumberOfRows: true)
3 3 0 3.0 329.6 0.1X
+RocksDB (trackTotalNumberOfRows: false)
3 3 0 3.0 330.2 0.1X
+
+
diff --git a/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-results.txt
b/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-results.txt
index d2aa646d5ec1..af7e11e786ef 100644
--- a/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-results.txt
+++ b/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-results.txt
@@ -2,141 +2,143 @@
put rows
================================================================================================
-OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
putting 10000 rows (10000 rows to overwrite - rate 100): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------------------
-In-memory 10
12 1 1.0 960.1 1.0X
-RocksDB (trackTotalNumberOfRows: true) 42
43 2 0.2 4173.9 0.2X
-RocksDB (trackTotalNumberOfRows: false) 16
16 1 0.6 1551.6 0.6X
+In-memory 9
10 0 1.1 934.9 1.0X
+RocksDB (trackTotalNumberOfRows: true) 40
41 2 0.2 4018.6 0.2X
+RocksDB (trackTotalNumberOfRows: false) 15
16 1 0.7 1499.5 0.6X
-OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
putting 10000 rows (5000 rows to overwrite - rate 50): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------
-In-memory 10
12 1 1.0 970.1 1.0X
-RocksDB (trackTotalNumberOfRows: true) 41
42 1 0.2 4095.8 0.2X
-RocksDB (trackTotalNumberOfRows: false) 15
17 1 0.6 1544.6 0.6X
+In-memory 9
10 1 1.1 922.2 1.0X
+RocksDB (trackTotalNumberOfRows: true) 39
40 1 0.3 3924.1 0.2X
+RocksDB (trackTotalNumberOfRows: false) 15
15 1 0.7 1504.0 0.6X
-OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
putting 10000 rows (1000 rows to overwrite - rate 10): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
11 1 1.1 933.3 1.0X
-RocksDB (trackTotalNumberOfRows: true) 40
41 1 0.3 3966.2 0.2X
-RocksDB (trackTotalNumberOfRows: false) 15
16 1 0.6 1540.2 0.6X
+In-memory 9
10 1 1.1 902.7 1.0X
+RocksDB (trackTotalNumberOfRows: true) 38
39 1 0.3 3846.6 0.2X
+RocksDB (trackTotalNumberOfRows: false) 15
15 1 0.7 1501.3 0.6X
-OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
putting 10000 rows (0 rows to overwrite - rate 0): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
11 1 1.1 936.1 1.0X
-RocksDB (trackTotalNumberOfRows: true) 39
41 1 0.3 3942.4 0.2X
-RocksDB (trackTotalNumberOfRows: false) 15
16 0 0.7 1530.1 0.6X
+In-memory 9
10 0 1.1 896.1 1.0X
+RocksDB (trackTotalNumberOfRows: true) 38
39 1 0.3 3816.6 0.2X
+RocksDB (trackTotalNumberOfRows: false) 15
15 0 0.7 1496.4 0.6X
================================================================================================
merge rows
================================================================================================
-OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
merging 10000 rows with 10 values per key (10000 rows to overwrite - rate
100): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
--------------------------------------------------------------------------------------------------------------------------------------------------------------
-RocksDB (trackTotalNumberOfRows: true)
525 538 6 0.0 52516.4 1.0X
-RocksDB (trackTotalNumberOfRows: false)
170 177 4 0.1 16960.4 3.1X
+RocksDB (trackTotalNumberOfRows: true)
508 519 5 0.0 50831.2 1.0X
+RocksDB (trackTotalNumberOfRows: false)
169 175 3 0.1 16943.3 3.0X
-OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
merging 10000 rows with 10 values per key (5000 rows to overwrite - rate 50):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------------
-RocksDB (trackTotalNumberOfRows: true)
514 528 6 0.0 51351.9 1.0X
-RocksDB (trackTotalNumberOfRows: false)
168 174 4 0.1 16794.0 3.1X
+RocksDB (trackTotalNumberOfRows: true)
495 505 6 0.0 49515.8 1.0X
+RocksDB (trackTotalNumberOfRows: false)
166 171 3 0.1 16606.9 3.0X
-OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
merging 10000 rows with 10 values per key (1000 rows to overwrite - rate 10):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------------
-RocksDB (trackTotalNumberOfRows: true)
500 513 6 0.0 49955.1 1.0X
-RocksDB (trackTotalNumberOfRows: false)
169 174 2 0.1 16867.1 3.0X
+RocksDB (trackTotalNumberOfRows: true)
487 501 7 0.0 48742.3 1.0X
+RocksDB (trackTotalNumberOfRows: false)
167 174 3 0.1 16654.6 2.9X
-OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
merging 10000 rows with 10 values per key (0 rows to overwrite - rate 0):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------------------------
-RocksDB (trackTotalNumberOfRows: true)
492 508 8 0.0 49225.8 1.0X
-RocksDB (trackTotalNumberOfRows: false)
168 173 3 0.1 16757.2 2.9X
+RocksDB (trackTotalNumberOfRows: true)
480 496 6 0.0 48046.7 1.0X
+RocksDB (trackTotalNumberOfRows: false)
166 173 3 0.1 16591.0 2.9X
================================================================================================
delete rows
================================================================================================
-OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
trying to delete 10000 rows from 10000 rows(10000 rows are non-existing - rate
100): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
0 1 0 26.1 38.3
1.0X
-RocksDB (trackTotalNumberOfRows: true)
38 40 1 0.3 3835.6
0.0X
-RocksDB (trackTotalNumberOfRows: false)
15 16 1 0.7 1455.7
0.0X
+In-memory
0 0 0 25.8 38.8
1.0X
+RocksDB (trackTotalNumberOfRows: true)
38 39 1 0.3 3772.3
0.0X
+RocksDB (trackTotalNumberOfRows: false)
15 15 0 0.7 1466.1
0.0X
-OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
trying to delete 10000 rows from 10000 rows(5000 rows are non-existing - rate
50): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 9 1 1.3 793.9
1.0X
-RocksDB (trackTotalNumberOfRows: true)
40 41 1 0.2 4018.1
0.2X
-RocksDB (trackTotalNumberOfRows: false)
15 16 0 0.7 1505.6
0.5X
+In-memory
8 8 1 1.3 777.7
1.0X
+RocksDB (trackTotalNumberOfRows: true)
39 40 1 0.3 3898.8
0.2X
+RocksDB (trackTotalNumberOfRows: false)
15 15 1 0.7 1482.2
0.5X
-OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
trying to delete 10000 rows from 10000 rows(1000 rows are non-existing - rate
10): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 10 1 1.2 837.2
1.0X
-RocksDB (trackTotalNumberOfRows: true)
41 42 1 0.2 4073.9
0.2X
-RocksDB (trackTotalNumberOfRows: false)
15 16 1 0.7 1470.6
0.6X
+In-memory
8 9 0 1.2 823.3
1.0X
+RocksDB (trackTotalNumberOfRows: true)
40 41 1 0.3 3960.8
0.2X
+RocksDB (trackTotalNumberOfRows: false)
15 15 0 0.7 1489.0
0.6X
-OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
trying to delete 10000 rows from 10000 rows(0 rows are non-existing - rate 0):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 9 0 1.2 843.6 1.0X
-RocksDB (trackTotalNumberOfRows: true)
41 42 1 0.2 4088.7 0.2X
-RocksDB (trackTotalNumberOfRows: false)
15 15 0 0.7 1466.1 0.6X
+In-memory
8 9 0 1.2 832.0 1.0X
+RocksDB (trackTotalNumberOfRows: true)
40 41 1 0.3 3986.7 0.2X
+RocksDB (trackTotalNumberOfRows: false)
15 16 0 0.7 1480.5 0.6X
================================================================================================
evict rows
================================================================================================
-OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
evicting 10000 rows (maxTimestampToEvictInMillis: 9999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 9 0 1.2 833.5 1.0X
-RocksDB (trackTotalNumberOfRows: true)
40 41 0 0.3 3976.5 0.2X
-RocksDB (trackTotalNumberOfRows: false)
16 16 0 0.6 1588.1 0.5X
+In-memory
8 9 0 1.2 838.3 1.0X
+RocksDB (trackTotalNumberOfRows: true)
39 39 1 0.3 3867.4 0.2X
+RocksDB (trackTotalNumberOfRows: false)
16 16 1 0.6 1564.6 0.5X
-OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
evicting 5000 rows (maxTimestampToEvictInMillis: 4999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 8 0 1.3 784.3 1.0X
-RocksDB (trackTotalNumberOfRows: true)
22 22 0 0.5 2155.1 0.4X
-RocksDB (trackTotalNumberOfRows: false)
10 10 0 1.0 986.9 0.8X
+In-memory
8 8 0 1.3 785.3 1.0X
+RocksDB (trackTotalNumberOfRows: true)
21 22 1 0.5 2098.1 0.4X
+RocksDB (trackTotalNumberOfRows: false)
10 10 0 1.0 957.2 0.8X
-OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
evicting 1000 rows (maxTimestampToEvictInMillis: 999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
7 8 0 1.4 722.3 1.0X
-RocksDB (trackTotalNumberOfRows: true)
7 7 0 1.4 718.8 1.0X
-RocksDB (trackTotalNumberOfRows: false)
5 5 0 2.0 488.7 1.5X
+In-memory
7 8 0 1.4 725.4 1.0X
+RocksDB (trackTotalNumberOfRows: true)
7 7 0 1.5 682.3 1.1X
+RocksDB (trackTotalNumberOfRows: false)
5 5 0 2.2 458.3 1.6X
-OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
+OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1017-azure
AMD EPYC 7763 64-Core Processor
evicting 0 rows (maxTimestampToEvictInMillis: -1) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
0 1 0 21.3 46.9 1.0X
-RocksDB (trackTotalNumberOfRows: true)
4 4 0 2.8 358.9 0.1X
-RocksDB (trackTotalNumberOfRows: false)
4 4 0 2.8 358.7 0.1X
+In-memory
0 1 0 21.2 47.2 1.0X
+RocksDB (trackTotalNumberOfRows: true)
3 3 0 3.1 326.5 0.1X
+RocksDB (trackTotalNumberOfRows: false)
3 3 0 3.1 326.6 0.1X
+
+
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
