andygrove opened a new issue, #2000: URL: https://github.com/apache/datafusion-ballista/issues/2000
Follow-up to #1959. That snapshot compared Ballista and Comet at SF100 with data in **MinIO** (`s3://`/`s3a://`). This one re-runs both engines reading the **same TPC-H SF100 Parquet from node-local disk**, at **matched hash joins**, to remove the object-store / interconnect bottleneck. It's the first genuinely same-source (local disk on both sides) cross-engine number, and it flips the result. > Informational snapshot, not a bug report. ## Setup - **Cluster:** 2-node k3s (kube1 32c/128GiB, kube2 32c/128GiB). **2 executors × 8 cores** each engine. - **Data:** TPC-H **SF100** Parquet on **node-local disk** (`/mnt/bigdata/tpch/sf100`, identical copy on both nodes), single-file-per-table layout. No MinIO in the read path. - **Ballista:** `main` + in-flight PRs, DataFusion 54. **Static planner (AQE off)**, `prefer_hash_join=true`, `target_partitions=32`. Both executors on one node (kube2). - **Comet:** 0.15.0-SNAPSHOT (Spark 3.5.3), `spark.comet.exec.replaceSortMergeJoin=true`, **Spark AQE on** (default). Executors spread across both nodes. - **Method:** 3 iterations; per-query metric = min of the 3; total = sum of per-query mins. All 22 row counts matched between the engines. ## Totals — the headline | Engine | SF100 total (local disk) | vs Comet | |---|---:|---:| | **Ballista** (static, hash join, p=32) | **256.9 s** | **0.86×** | | **Comet** (Spark 3.5 + Comet, AQE on) | 300.2 s | 1.0× | **Ballista is ~1.17× faster overall and wins 19 of 22 queries.** Comet leads only Q7, Q18, Q21. **This reverses the MinIO result.** In #1959 at matched hash joins on MinIO, Comet led (Comet 347 s vs Ballista 398 s, ~1.15× Comet). Moving both engines to local disk flips it to Ballista 257 vs Comet 300 (~1.17× Ballista). The implication: **most of Comet's historical edge in this comparison was I/O-bound** — reading SF100 over MinIO across a 2.5 GbE interconnect — not an engine-execution gap. Remove the object-store bottleneck and Ballista's static planner + hash joins is faster on nearly every query. ## Per-query (seconds, best of 3) | Q | Ballista | Comet | Winner | Ratio | |---|---:|---:|---|---:| | Q1 | 2.88 | 4.97 | Ballista | 1.73× | | Q2 | 3.44 | 6.32 | Ballista | 1.84× | | Q3 | 8.03 | 11.77 | Ballista | 1.47× | | Q4 | 2.90 | 6.66 | Ballista | 2.30× | | Q5 | 16.27 | 27.28 | Ballista | 1.68× | | Q6 | 1.13 | 1.21 | Ballista | 1.07× | | Q7 | 24.41 | 15.18 | **Comet** | 1.61× | | Q8 | 24.27 | 36.62 | Ballista | 1.51× | | Q9 | 41.91 | 51.29 | Ballista | 1.22× | | Q10 | 9.34 | 11.23 | Ballista | 1.20× | | Q11 | 3.87 | 5.86 | Ballista | 1.51× | | Q12 | 4.27 | 6.69 | Ballista | 1.57× | | Q13 | 5.10 | 5.50 | Ballista | 1.08× | | Q14 | 2.15 | 2.25 | Ballista | 1.05× | | Q15 | 3.88 | 4.36 | Ballista | 1.12× | | Q16 | 1.67 | 3.94 | Ballista | 2.36× | | Q17 | 30.00 | 33.73 | Ballista | 1.12× | | Q18 | 29.03 | 22.64 | **Comet** | 1.28× | | Q19 | 2.96 | 3.65 | Ballista | 1.23× | | Q20 | 5.36 | 6.71 | Ballista | 1.25× | | Q21 | 32.45 | 29.01 | **Comet** | 1.12× | | Q22 | 1.62 | 3.32 | Ballista | 2.05× | | **Total** | **256.9** | **300.2** | **Ballista** | **1.17×** | ## Caveats Both of these cut *against* Ballista's win, so the direction is robust: - **Node topology:** Comet's executors spread across both nodes (2 disks, 2 machines' memory bandwidth); Ballista's two executors sat on one node (one disk). Same 16 total cores, but Comet had more aggregate hardware — and still lost. A fully apples-to-apples rerun would pin both engines to the same topology. - **AQE:** Comet ran with Spark AQE (runtime partition coalescing / skew handling); Ballista used the static planner with no adaptive equivalent. - Single homelab cluster, single-shot-per-iteration, ~5–10% run-to-run noise. The 1.17× total and the per-query ordering are well above that; the three Comet-win queries (Q7/Q18/Q21) are close (1.1–1.6×) and warrant multi-run confirmation before drawing per-query conclusions. Related: shuffle-read metrics to attribute the Comet-win queries are tracked in #1958. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
