Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22501#discussion_r226442573
--- Diff: sql/core/benchmarks/WideSchemaBenchmark-results.txt ---
@@ -1,117 +1,145 @@
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_92-b14 on Mac OS X 10.11.6
-Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz
+================================================================================================
+parsing large select expressions
+================================================================================================
+Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
+Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz
parsing large select: Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
------------------------------------------------------------------------------------------------
-1 select expressions 2 / 4 0.0
2050147.0 1.0X
-100 select expressions 6 / 7 0.0
6123412.0 0.3X
-2500 select expressions 135 / 141 0.0
134623148.0 0.0X
+1 select expressions 2 / 4 0.0
1934953.0 1.0X
+100 select expressions 4 / 5 0.0
3659399.0 0.5X
+2500 select expressions 68 / 76 0.0
68278937.0 0.0X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_92-b14 on Mac OS X 10.11.6
-Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz
+================================================================================================
+many column field read and write
+================================================================================================
+
+Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
+Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz
many column field r/w: Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
------------------------------------------------------------------------------------------------
-1 cols x 100000 rows (read in-mem) 16 / 18 6.3
158.6 1.0X
-1 cols x 100000 rows (exec in-mem) 17 / 19 6.0
166.7 1.0X
-1 cols x 100000 rows (read parquet) 24 / 26 4.3
235.1 0.7X
-1 cols x 100000 rows (write parquet) 81 / 85 1.2
811.3 0.2X
-100 cols x 1000 rows (read in-mem) 17 / 19 6.0
166.2 1.0X
-100 cols x 1000 rows (exec in-mem) 25 / 27 4.0
249.2 0.6X
-100 cols x 1000 rows (read parquet) 23 / 25 4.4
226.0 0.7X
-100 cols x 1000 rows (write parquet) 83 / 87 1.2
831.0 0.2X
-2500 cols x 40 rows (read in-mem) 132 / 137 0.8
1322.9 0.1X
-2500 cols x 40 rows (exec in-mem) 326 / 330 0.3
3260.6 0.0X
-2500 cols x 40 rows (read parquet) 831 / 839 0.1
8305.8 0.0X
-2500 cols x 40 rows (write parquet) 237 / 245 0.4
2372.6 0.1X
-
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_92-b14 on Mac OS X 10.11.6
-Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz
+1 cols x 100000 rows (read in-mem) 22 / 25 4.6
219.4 1.0X
+1 cols x 100000 rows (exec in-mem) 22 / 28 4.5
223.8 1.0X
+1 cols x 100000 rows (read parquet) 45 / 49 2.2
449.6 0.5X
+1 cols x 100000 rows (write parquet) 204 / 223 0.5
2044.4 0.1X
+100 cols x 1000 rows (read in-mem) 26 / 28 3.9
255.8 0.9X
+100 cols x 1000 rows (exec in-mem) 32 / 35 3.1
319.3 0.7X
+100 cols x 1000 rows (read parquet) 45 / 52 2.2
445.9 0.5X
+100 cols x 1000 rows (write parquet) 275 / 536 0.4
2746.1 0.1X
+2500 cols x 40 rows (read in-mem) 261 / 434 0.4
2607.3 0.1X
+2500 cols x 40 rows (exec in-mem) 624 / 701 0.2
6240.5 0.0X
+2500 cols x 40 rows (read parquet) 196 / 301 0.5
1963.4 0.1X
+2500 cols x 40 rows (write parquet) 687 / 1049 0.1
6870.6 0.0X
+
+
+================================================================================================
+wide shallowly nested struct field read and write
+================================================================================================
+Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
+Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz
wide shallowly nested struct field r/w: Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
------------------------------------------------------------------------------------------------
-1 wide x 100000 rows (read in-mem) 15 / 17 6.6
151.0 1.0X
-1 wide x 100000 rows (exec in-mem) 20 / 22 5.1
196.6 0.8X
-1 wide x 100000 rows (read parquet) 59 / 63 1.7
592.8 0.3X
-1 wide x 100000 rows (write parquet) 81 / 87 1.2
814.6 0.2X
-100 wide x 1000 rows (read in-mem) 21 / 25 4.8
208.7 0.7X
-100 wide x 1000 rows (exec in-mem) 72 / 81 1.4
718.5 0.2X
-100 wide x 1000 rows (read parquet) 75 / 85 1.3
752.6 0.2X
-100 wide x 1000 rows (write parquet) 88 / 95 1.1
876.7 0.2X
-2500 wide x 40 rows (read in-mem) 28 / 34 3.5
282.2 0.5X
-2500 wide x 40 rows (exec in-mem) 1269 / 1284 0.1
12688.1 0.0X
-2500 wide x 40 rows (read parquet) 549 / 578 0.2
5493.4 0.0X
-2500 wide x 40 rows (write parquet) 96 / 104 1.0
959.1 0.2X
-
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_92-b14 on Mac OS X 10.11.6
-Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz
+1 wide x 100000 rows (read in-mem) 23 / 42 4.4
226.2 1.0X
+1 wide x 100000 rows (exec in-mem) 29 / 53 3.5
288.5 0.8X
+1 wide x 100000 rows (read parquet) 93 / 102 1.1
928.2 0.2X
+1 wide x 100000 rows (write parquet) 201 / 222 0.5
2009.6 0.1X
+100 wide x 1000 rows (read in-mem) 42 / 55 2.4
421.8 0.5X
+100 wide x 1000 rows (exec in-mem) 55 / 113 1.8
547.0 0.4X
+100 wide x 1000 rows (read parquet) 139 / 263 0.7
1390.6 0.2X
+100 wide x 1000 rows (write parquet) 245 / 338 0.4
2450.9 0.1X
+2500 wide x 40 rows (read in-mem) 51 / 72 2.0
511.7 0.4X
+2500 wide x 40 rows (exec in-mem) 265 / 303 0.4
2654.8 0.1X
+2500 wide x 40 rows (read parquet) 1285 / 1339 0.1
12845.1 0.0X
+2500 wide x 40 rows (write parquet) 238 / 262 0.4
2378.8 0.1X
+
+================================================================================================
+deeply nested struct field read and write
+================================================================================================
+
+Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
+Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz
deeply nested struct field r/w: Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
------------------------------------------------------------------------------------------------
-1 deep x 100000 rows (read in-mem) 14 / 16 7.0
143.8 1.0X
-1 deep x 100000 rows (exec in-mem) 17 / 19 5.9
169.7 0.8X
-1 deep x 100000 rows (read parquet) 33 / 35 3.1
327.0 0.4X
-1 deep x 100000 rows (write parquet) 79 / 84 1.3
786.9 0.2X
-100 deep x 1000 rows (read in-mem) 21 / 24 4.7
211.3 0.7X
-100 deep x 1000 rows (exec in-mem) 221 / 235 0.5
2214.5 0.1X
-100 deep x 1000 rows (read parquet) 1928 / 1952 0.1
19277.1 0.0X
-100 deep x 1000 rows (write parquet) 91 / 96 1.1
909.5 0.2X
-250 deep x 400 rows (read in-mem) 57 / 61 1.8
567.1 0.3X
-250 deep x 400 rows (exec in-mem) 1329 / 1385 0.1
13291.8 0.0X
-250 deep x 400 rows (read parquet) 36563 / 36750 0.0
365630.2 0.0X
-250 deep x 400 rows (write parquet) 126 / 130 0.8
1262.0 0.1X
-
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_92-b14 on Mac OS X 10.11.6
-Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz
+1 deep x 100000 rows (read in-mem) 20 / 24 5.1
197.9 1.0X
+1 deep x 100000 rows (exec in-mem) 23 / 28 4.4
227.8 0.9X
+1 deep x 100000 rows (read parquet) 50 / 58 2.0
500.1 0.4X
+1 deep x 100000 rows (write parquet) 195 / 219 0.5
1945.1 0.1X
+100 deep x 1000 rows (read in-mem) 39 / 57 2.5
393.1 0.5X
+100 deep x 1000 rows (exec in-mem) 480 / 556 0.2
4795.7 0.0X
+100 deep x 1000 rows (read parquet) 7943 / 7950 0.0
79427.5 0.0X
--- End diff --
Ur, @wangyum . Is this 4 times slower than before?
cc @dbtsai .
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
