This is an automated email from the ASF dual-hosted git repository.
dbtsai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 3221bf4 [SPARK-27034][SPARK-27123][SQL][FOLLOWUP] Update Nested
Schema Pruning BM result with EC2
3221bf4 is described below
commit 3221bf4cd5199a366aa1041e1a8bf9de9ed2eb89
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Wed Mar 13 20:27:10 2019 +0000
[SPARK-27034][SPARK-27123][SQL][FOLLOWUP] Update Nested Schema Pruning BM
result with EC2
## What changes were proposed in this pull request?
This is a follow up PR for #23943 in order to update the benchmark result
with EC2 `r3.xlarge` instance.
## How was this patch tested?
N/A. (Manually compare the diff)
Closes #24078 from dongjoon-hyun/SPARK-27034.
Authored-by: Dongjoon Hyun <dh...@apple.com>
Signed-off-by: DB Tsai <d_t...@apple.com>
---
.../OrcNestedSchemaPruningBenchmark-results.txt | 40 +++++++++++-----------
.../OrcV2NestedSchemaPruningBenchmark-results.txt | 40 +++++++++++-----------
...ParquetNestedSchemaPruningBenchmark-results.txt | 40 +++++++++++-----------
3 files changed, 60 insertions(+), 60 deletions(-)
diff --git a/sql/core/benchmarks/OrcNestedSchemaPruningBenchmark-results.txt
b/sql/core/benchmarks/OrcNestedSchemaPruningBenchmark-results.txt
index fdd35cd..078fbbb 100644
--- a/sql/core/benchmarks/OrcNestedSchemaPruningBenchmark-results.txt
+++ b/sql/core/benchmarks/OrcNestedSchemaPruningBenchmark-results.txt
@@ -2,39 +2,39 @@
Nested Schema Pruning Benchmark For ORC v1
================================================================================================
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Selection: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 116 151
36 8.6 116.3 1.0X
-Nested column 544 604
31 1.8 544.5 0.2X
+Top-level column 117 154
23 8.5 117.5 1.0X
+Nested column 1271 1295
26 0.8 1270.5 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Limiting: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 360 397
32 2.8 360.4 1.0X
-Nested column 3322 3503
166 0.3 3322.4 0.1X
+Top-level column 431 488
73 2.3 431.2 1.0X
+Nested column 1738 1777
24 0.6 1738.3 0.2X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Repartitioning: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 292 334
32 3.4 291.8 1.0X
-Nested column 3306 3489
123 0.3 3305.7 0.1X
+Top-level column 349 381
87 2.9 348.7 1.0X
+Nested column 4374 4456
125 0.2 4373.6 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Repartitioning by exprs: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 302 333
27 3.3 302.0 1.0X
-Nested column 2697 3347
390 0.4 2697.4 0.1X
+Top-level column 358 410
85 2.8 358.5 1.0X
+Nested column 4447 4517
125 0.2 4447.3 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Sorting: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 316 440
146 3.2 315.8 1.0X
-Nested column 2728 2928
205 0.4 2727.9 0.1X
+Top-level column 553 566
13 1.8 552.8 1.0X
+Nested column 5115 5248
84 0.2 5115.1 0.1X
diff --git a/sql/core/benchmarks/OrcV2NestedSchemaPruningBenchmark-results.txt
b/sql/core/benchmarks/OrcV2NestedSchemaPruningBenchmark-results.txt
index ad43ffb..eaaf3b0 100644
--- a/sql/core/benchmarks/OrcV2NestedSchemaPruningBenchmark-results.txt
+++ b/sql/core/benchmarks/OrcV2NestedSchemaPruningBenchmark-results.txt
@@ -2,39 +2,39 @@
Nested Schema Pruning Benchmark For ORC v2
================================================================================================
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Selection: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 91 102
9 11.0 91.2 1.0X
-Nested column 1459 1548
80 0.7 1458.5 0.1X
+Top-level column 120 148
24 8.3 120.0 1.0X
+Nested column 2367 2415
43 0.4 2367.0 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Limiting: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 101 112
10 9.9 100.7 1.0X
-Nested column 1459 1619
109 0.7 1458.9 0.1X
+Top-level column 129 153
16 7.8 128.5 1.0X
+Nested column 2368 2400
32 0.4 2367.7 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Repartitioning: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 268 284
12 3.7 268.2 1.0X
-Nested column 2781 2865
73 0.4 2780.8 0.1X
+Top-level column 359 396
59 2.8 358.9 1.0X
+Nested column 4100 4147
59 0.2 4099.9 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Repartitioning by exprs: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 309 318
6 3.2 309.2 1.0X
-Nested column 2426 2891
253 0.4 2425.8 0.1X
+Top-level column 367 394
56 2.7 366.8 1.0X
+Nested column 4182 4236
64 0.2 4181.5 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Sorting: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 179 194
8 5.6 179.3 1.0X
-Nested column 2084 2277
243 0.5 2083.7 0.1X
+Top-level column 262 269
6 3.8 262.5 1.0X
+Nested column 2950 3021
65 0.3 2949.8 0.1X
diff --git
a/sql/core/benchmarks/ParquetNestedSchemaPruningBenchmark-results.txt
b/sql/core/benchmarks/ParquetNestedSchemaPruningBenchmark-results.txt
index d51ebc6..1f1296e 100644
--- a/sql/core/benchmarks/ParquetNestedSchemaPruningBenchmark-results.txt
+++ b/sql/core/benchmarks/ParquetNestedSchemaPruningBenchmark-results.txt
@@ -2,39 +2,39 @@
Nested Schema Pruning Benchmark For Parquet
================================================================================================
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Selection: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 88 114
16 11.4 87.5 1.0X
-Nested column 201 223
27 5.0 200.5 0.4X
+Top-level column 145 174
23 6.9 145.1 1.0X
+Nested column 325 346
19 3.1 324.8 0.4X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Limiting: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 263 315
36 3.8 263.2 1.0X
-Nested column 2111 2622
613 0.5 2111.1 0.1X
+Top-level column 434 508
108 2.3 434.3 1.0X
+Nested column 625 647
23 1.6 624.8 0.7X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Repartitioning: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 222 250
34 4.5 222.2 1.0X
-Nested column 2084 2339
266 0.5 2084.2 0.1X
+Top-level column 357 368
9 2.8 356.9 1.0X
+Nested column 2897 2976
88 0.3 2897.4 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Repartitioning by exprs: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 238 306
96 4.2 238.1 1.0X
-Nested column 2080 2373
218 0.5 2079.5 0.1X
+Top-level column 365 413
77 2.7 364.9 1.0X
+Nested column 2902 2969
99 0.3 2902.4 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Sorting: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 328 383
57 3.1 327.6 1.0X
-Nested column 2595 3136
638 0.4 2595.1 0.1X
+Top-level column 555 600
80 1.8 555.4 1.0X
+Nested column 3448 3490
45 0.3 3447.9 0.2X
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
