andygrove commented on PR #5638:
URL:
https://github.com/apache/datafusion-comet/pull/5638#issuecomment-5533915969
_LLM-assisted: posted with Claude Code._
The full `spark/benchmarks/CometIcebergSystemFunctionBenchmark-results.txt`
from the run described in the previous comment, unedited. It is the
`SPARK_GENERATE_BENCHMARK_FILES=1` artifact, so it carries the JVM and CPU
banner per case and would carry any `WARNING` block the benchmark emitted;
there are none.
<details>
<summary>CometIcebergSystemFunctionBenchmark-results.txt</summary>
```
================================================================================================
Iceberg system functions
================================================================================================
================================================================================================
bucket(int)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
bucket(int): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 29 46
15 36.3 27.5 1.0X
Comet 23 26
2 44.9 22.3 1.2X
================================================================================================
bucket(int, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
bucket(int, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 33 73
28 32.2 31.0 1.0X
Comet 24 26
2 44.3 22.6 1.4X
================================================================================================
bucket(long)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
bucket(long): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 27 40
20 38.5 26.0 1.0X
Comet 23 24
2 46.2 21.6 1.2X
================================================================================================
bucket(long, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
bucket(long, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 32 45
24 32.3 30.9 1.0X
Comet 24 25
1 44.6 22.4 1.4X
================================================================================================
bucket(dec)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
bucket(dec): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 109 120
23 9.6 104.1 1.0X
Comet 30 31
1 34.6 28.9 3.6X
================================================================================================
bucket(dec, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
bucket(dec, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 108 117
14 9.7 102.6 1.0X
Comet 33 35
1 31.4 31.8 3.2X
================================================================================================
bucket(str_dict)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
bucket(str_dict): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 32 48
31 33.0 30.3 1.0X
Comet 20 22
1 52.6 19.0 1.6X
================================================================================================
bucket(str_dict, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
bucket(str_dict, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 38 62
28 27.9 35.9 1.0X
Comet 22 23
1 47.7 21.0 1.7X
================================================================================================
bucket(str)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
bucket(str): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 101 110
24 10.4 96.0 1.0X
Comet 76 77
1 13.8 72.3 1.3X
================================================================================================
bucket(str, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
bucket(str, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 93 118
36 11.2 89.0 1.0X
Comet 75 76
1 14.1 71.1 1.3X
================================================================================================
bucket(bin)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
bucket(bin): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 71 116
41 14.9 67.3 1.0X
Comet 51 53
1 20.4 48.9 1.4X
================================================================================================
bucket(bin, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
bucket(bin, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 68 85
32 15.4 64.8 1.0X
Comet 52 54
1 20.2 49.5 1.3X
================================================================================================
bucket(date)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
bucket(date): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 24 39
22 43.0 23.3 1.0X
Comet 21 22
1 50.9 19.6 1.2X
================================================================================================
bucket(date, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
bucket(date, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 30 37
7 34.4 29.1 1.0X
Comet 21 22
1 50.0 20.0 1.5X
================================================================================================
bucket(ts)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
bucket(ts): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 34 53
23 30.6 32.7 1.0X
Comet 21 22
1 49.1 20.4 1.6X
================================================================================================
bucket(ts, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
bucket(ts, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 39 57
28 27.2 36.8 1.0X
Comet 23 25
2 45.9 21.8 1.7X
================================================================================================
truncate(int)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
truncate(int): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 22 30
17 48.2 20.8 1.0X
Comet 19 21
1 54.8 18.2 1.1X
================================================================================================
truncate(int, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
truncate(int, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 29 52
36 36.5 27.4 1.0X
Comet 21 22
1 50.6 19.8 1.4X
================================================================================================
truncate(long)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
truncate(long): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 24 70
44 44.3 22.6 1.0X
Comet 20 21
1 53.6 18.7 1.2X
================================================================================================
truncate(long, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
truncate(long, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 30 51
33 35.0 28.6 1.0X
Comet 22 24
3 48.8 20.5 1.4X
================================================================================================
truncate(str_dict)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
truncate(str_dict): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 41 72
44 25.8 38.8 1.0X
Comet 25 26
1 42.0 23.8 1.6X
================================================================================================
truncate(str_dict, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
truncate(str_dict, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 49 81
40 21.2 47.1 1.0X
Comet 28 30
1 37.6 26.6 1.8X
================================================================================================
truncate(str)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
truncate(str): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 94 140
39 11.1 89.8 1.0X
Comet 77 80
2 13.7 73.2 1.2X
================================================================================================
truncate(str, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
truncate(str, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 92 155
52 11.4 87.4 1.0X
Comet 73 76
2 14.3 69.8 1.3X
================================================================================================
truncate(bin)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
truncate(bin): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 67 142
60 15.6 64.0 1.0X
Comet 48 49
1 22.0 45.4 1.4X
================================================================================================
truncate(bin, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
truncate(bin, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 63 96
47 16.6 60.4 1.0X
Comet 47 49
2 22.5 44.5 1.4X
================================================================================================
years(date)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
years(date): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 37 74
55 28.4 35.3 1.0X
Comet 19 21
2 56.4 17.7 2.0X
================================================================================================
years(date, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
years(date, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 45 111
57 23.4 42.7 1.0X
Comet 20 22
2 51.8 19.3 2.2X
================================================================================================
years(ts)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
years(ts): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 52 106
50 20.3 49.2 1.0X
Comet 20 22
1 51.4 19.4 2.5X
================================================================================================
years(ts, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
years(ts, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 54 123
58 19.5 51.3 1.0X
Comet 21 23
1 49.3 20.3 2.5X
================================================================================================
months(date)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
months(date): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 37 73
52 28.4 35.2 1.0X
Comet 19 20
1 56.3 17.8 2.0X
================================================================================================
months(date, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
months(date, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 49 98
67 21.2 47.1 1.0X
Comet 20 21
1 52.0 19.2 2.4X
================================================================================================
months(ts)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
months(ts): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 52 76
56 20.1 49.6 1.0X
Comet 20 22
1 52.4 19.1 2.6X
================================================================================================
months(ts, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
months(ts, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 53 74
49 19.8 50.6 1.0X
Comet 21 23
1 49.2 20.3 2.5X
================================================================================================
days(date)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
days(date): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 20 45
53 51.9 19.3 1.0X
Comet 19 20
1 55.8 17.9 1.1X
================================================================================================
days(date, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
days(date, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 28 61
63 38.0 26.3 1.0X
Comet 20 22
1 51.4 19.5 1.4X
================================================================================================
days(ts)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
days(ts): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 59 92
65 17.7 56.5 1.0X
Comet 20 22
1 51.2 19.5 2.9X
================================================================================================
days(ts, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
days(ts, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 60 92
65 17.6 56.9 1.0X
Comet 22 23
1 48.6 20.6 2.8X
================================================================================================
hours(ts)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
hours(ts): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 60 66
7 17.4 57.4 1.0X
Comet 21 21
1 51.1 19.6 2.9X
================================================================================================
hours(ts, nulls)
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 26.6.2
Apple M3 Max
hours(ts, nulls): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 61 79
35 17.1 58.5 1.0X
Comet 22 23
1 48.5 20.6 2.8X
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
