avantgardnerio commented on PR #2406: URL: https://github.com/apache/datafusion-ballista/pull/2406#issuecomment-5516296214
**Cluster numbers, as requested in the PR description.** **Setup.** 4 × `r6i.24xlarge` in EKS, 32 executor pods at 8 vCPU / 64 GiB each (`--vcores=8`, `--memory-pool-size=48103633715`, 1000 GiB gp3 shuffle volume), TPC-H SF1000 Parquet+ZSTD on S3, `target_partitions=256`, `max_partitions_per_task=1`, AQE on — the shape in `docs/source/contributors-guide/benchmarking.md`. Baseline is `b0dfd259`, this PR's merge base; treatment is `7684d651`. Same client image both arms. **Method.** Counterbalanced ABBA (base → pr → pr → base), because a first sequential base→pr pair showed a gap that partly disappeared once the order was balanced. 5 iterations per query per run, first dropped as cold, so n = 8 warm iterations per arm. | query | metric | base | pr | Δ | Welch t | |---|---|---:|---:|---:|---:| | q4 | wall (s) | 10.32 ± 0.97 | 10.68 ± 0.99 | +3.5% | 0.70 | | q4 | `fetch_time` (s) | 6,773 ± 578 | 8,172 ± 1,571 | +20.7% | 2.36 | | q4 | `permit_wait_time` (s) | 43,219 ± 4,523 | 46,082 ± 12,478 | +6.6% | 0.61 | | q18 | wall (s) | 38.09 ± 0.76 | 38.69 ± 1.09 | +1.6% | 1.19 | | q18 | `fetch_time` (s) | 59,751 ± 3,207 | 63,844 ± 3,248 | +6.9% | 2.54 | | q18 | `permit_wait_time` (s) | 527,109 ± 21,846 | 579,635 ± 29,380 | +10.0% | 4.06 | Metric columns are stage sums over 256 partitions. **Wall clock is indistinguishable in either direction** (t = 0.70, 1.19). `fetch_time` is higher on both queries at t > 2.3. **`permit_wait_time` is not strictly comparable across the arms.** This PR starts the timer at a single `queued_at` taken before the loop, where main starts each block's timer when its spawned task first runs, so blocks now accumulate wait from an earlier instant. Some of the +10% is definitional. `fetch_time` and wall clock keep their definitions. **Tail spread** — per-task `exec_duration` across the 256 tasks of the shuffle-reading stage, which is what the change targets. q4: `max/median` 3.53 → 3.74, `cv` 0.372 → 0.376. q18: `max/median` 1.13 → 1.16, `cv` 0.069 → 0.073. No tightening. q18's baseline is already `cv = 0.069` across 256 tasks, so there is little imbalance available to remove at this shape. **Scope.** One shape only: 32 producers, 256 consumers. The PR asks whether more executors widens or narrows the gap; that is untested here. --- 🤖 Run and written by [Claude Code](https://claude.com/claude-code) on @avantgardnerio's cluster. Raw logs and per-stage metric dumps available on request. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
