wirybeaver commented on PR #2749:
URL:
https://github.com/apache/datafusion-ballista/pull/2749#issuecomment-6783604538
<!-- runtime-filter-four-candidate-evaluation -->
## Benchmark Result
| Local candidate / directed case | P50 query latency (ms) | Preflight
output checks |
|---|---|---|
| C1 — Reuse filters for partitioned fallback (D1) | A/B: 370.44 /
163.23<br>A → B: -55.94% | A/B: matched DataFusion |
| C2 — Freeze on actual task binding (D2) | A/B: 365.23 / 364.45<br>A → B:
-0.21% | A/B: matched DataFusion |
| C3 — Bypass Bloom at ≥99% occupancy (D3) | A/B: 212.36 / 176.43<br>A → B:
-16.92% | A/B: matched DataFusion |
| C4 — Skip sibling-only producers (D4) | A/B: 406.40 / 412.09<br>A → B:
+1.40% | A/B: matched DataFusion |
**The four latency rows above are custom directed SQL scenarios on TPC-DS
SF10 data, not standard TPC-DS queries or suite totals.** Negative change means
lower latency.
Four independent local A/Bs after rebasing Task 2: **A = unchanged zero-wait
Task 2; B = A plus one candidate**, with runtime filtering enabled in both.
Candidates are not yet integrated into this PR.
Environment: one Mac (10 CPUs, 24 GiB), two executors per arm (4 vcores / 4
GiB each), 16 partitions, batch 8192, release without LTO. Each comparison
covers 13 unmodified TPC-DS queries (listed below) plus its directed case.
Process: one DataFusion-verified preflight per query/arm, one complete-suite
warmup, then five full-suite rounds. Pair A → B in rounds 1/3/5 and B → A in
2/4; each arm gets one short warmup immediately followed by one timed request.
Servers stay running between visits and restart between candidate comparisons.
Latency is client SQL execution through final result collection, excluding
registration. No A/A, retries or dropped samples.
All **112 preflight checks** matched and **560 timed requests** completed.
Every standard-query change stayed below +3% (maximum +2.92%); five samples do
not establish zero regression.
C1/C3 show benefits in their directed cases. C1 bound a ready filter in B:
5/5 visits versus A: 0/5. C2 was scheduling-dependent (A: 1/5, B: 2/5 ready),
with no clear median gain. C4 removed the producer without improving latency.
These results are not additive or whole-suite speedups.
<details>
<summary>Per-visit results — all timed standard and directed
queries</summary>
| Candidate | Workload | Round | Arm | Client query latency (ms) |
|---|---|---:|---|---:|
| C1 | Q3 | 1 | A | 139.573 |
| C1 | Q3 | 1 | B | 141.601 |
| C1 | Q7 | 1 | A | 381.603 |
| C1 | Q7 | 1 | B | 376.289 |
| C1 | Q13 | 1 | A | 450.944 |
| C1 | Q13 | 1 | B | 463.980 |
| C1 | Q17 | 1 | A | 276.631 |
| C1 | Q17 | 1 | B | 272.633 |
| C1 | Q19 | 1 | A | 206.235 |
| C1 | Q19 | 1 | B | 210.868 |
| C1 | Q25 | 1 | A | 343.506 |
| C1 | Q25 | 1 | B | 351.103 |
| C1 | Q29 | 1 | A | 281.152 |
| C1 | Q29 | 1 | B | 279.793 |
| C1 | Q42 | 1 | A | 142.039 |
| C1 | Q42 | 1 | B | 140.895 |
| C1 | Q52 | 1 | A | 146.114 |
| C1 | Q52 | 1 | B | 144.760 |
| C1 | Q55 | 1 | A | 142.534 |
| C1 | Q55 | 1 | B | 141.652 |
| C1 | Q72 | 1 | A | 1341.684 |
| C1 | Q72 | 1 | B | 1361.048 |
| C1 | Q82 | 1 | A | 350.855 |
| C1 | Q82 | 1 | B | 343.780 |
| C1 | Q96 | 1 | A | 90.381 |
| C1 | Q96 | 1 | B | 90.704 |
| C1 | D1 | 1 | A | 370.439 |
| C1 | D1 | 1 | B | 165.261 |
| C1 | Q3 | 2 | B | 143.802 |
| C1 | Q3 | 2 | A | 144.062 |
| C1 | Q7 | 2 | B | 388.767 |
| C1 | Q7 | 2 | A | 382.769 |
| C1 | Q13 | 2 | B | 461.861 |
| C1 | Q13 | 2 | A | 469.104 |
| C1 | Q17 | 2 | B | 272.624 |
| C1 | Q17 | 2 | A | 272.777 |
| C1 | Q19 | 2 | B | 214.112 |
| C1 | Q19 | 2 | A | 215.072 |
| C1 | Q25 | 2 | B | 352.044 |
| C1 | Q25 | 2 | A | 355.051 |
| C1 | Q29 | 2 | B | 286.462 |
| C1 | Q29 | 2 | A | 286.535 |
| C1 | Q42 | 2 | B | 143.247 |
| C1 | Q42 | 2 | A | 142.377 |
| C1 | Q52 | 2 | B | 147.225 |
| C1 | Q52 | 2 | A | 148.907 |
| C1 | Q55 | 2 | B | 141.157 |
| C1 | Q55 | 2 | A | 144.260 |
| C1 | Q72 | 2 | B | 1372.871 |
| C1 | Q72 | 2 | A | 1371.554 |
| C1 | Q82 | 2 | B | 346.336 |
| C1 | Q82 | 2 | A | 342.188 |
| C1 | Q96 | 2 | B | 90.486 |
| C1 | Q96 | 2 | A | 92.562 |
| C1 | D1 | 2 | B | 163.231 |
| C1 | D1 | 2 | A | 359.540 |
| C1 | Q3 | 3 | A | 142.779 |
| C1 | Q3 | 3 | B | 142.353 |
| C1 | Q7 | 3 | A | 399.583 |
| C1 | Q7 | 3 | B | 390.156 |
| C1 | Q13 | 3 | A | 467.432 |
| C1 | Q13 | 3 | B | 462.071 |
| C1 | Q17 | 3 | A | 275.397 |
| C1 | Q17 | 3 | B | 277.305 |
| C1 | Q19 | 3 | A | 212.801 |
| C1 | Q19 | 3 | B | 215.422 |
| C1 | Q25 | 3 | A | 350.130 |
| C1 | Q25 | 3 | B | 357.501 |
| C1 | Q29 | 3 | A | 285.706 |
| C1 | Q29 | 3 | B | 289.286 |
| C1 | Q42 | 3 | A | 145.310 |
| C1 | Q42 | 3 | B | 143.834 |
| C1 | Q52 | 3 | A | 147.807 |
| C1 | Q52 | 3 | B | 150.047 |
| C1 | Q55 | 3 | A | 144.370 |
| C1 | Q55 | 3 | B | 142.970 |
| C1 | Q72 | 3 | A | 1361.111 |
| C1 | Q72 | 3 | B | 1407.371 |
| C1 | Q82 | 3 | A | 350.761 |
| C1 | Q82 | 3 | B | 346.947 |
| C1 | Q96 | 3 | A | 93.205 |
| C1 | Q96 | 3 | B | 91.096 |
| C1 | D1 | 3 | A | 358.885 |
| C1 | D1 | 3 | B | 163.331 |
| C1 | Q3 | 4 | B | 143.655 |
| C1 | Q3 | 4 | A | 143.944 |
| C1 | Q7 | 4 | B | 390.426 |
| C1 | Q7 | 4 | A | 402.178 |
| C1 | Q13 | 4 | B | 478.196 |
| C1 | Q13 | 4 | A | 464.199 |
| C1 | Q17 | 4 | B | 275.247 |
| C1 | Q17 | 4 | A | 273.162 |
| C1 | Q19 | 4 | B | 217.395 |
| C1 | Q19 | 4 | A | 215.928 |
| C1 | Q25 | 4 | B | 360.807 |
| C1 | Q25 | 4 | A | 355.689 |
| C1 | Q29 | 4 | B | 288.119 |
| C1 | Q29 | 4 | A | 316.799 |
| C1 | Q42 | 4 | B | 149.521 |
| C1 | Q42 | 4 | A | 145.937 |
| C1 | Q52 | 4 | B | 149.501 |
| C1 | Q52 | 4 | A | 158.581 |
| C1 | Q55 | 4 | B | 146.730 |
| C1 | Q55 | 4 | A | 148.443 |
| C1 | Q72 | 4 | B | 1349.907 |
| C1 | Q72 | 4 | A | 1361.220 |
| C1 | Q82 | 4 | B | 357.459 |
| C1 | Q82 | 4 | A | 356.170 |
| C1 | Q96 | 4 | B | 93.309 |
| C1 | Q96 | 4 | A | 89.533 |
| C1 | D1 | 4 | B | 161.566 |
| C1 | D1 | 4 | A | 373.229 |
| C1 | Q3 | 5 | A | 145.926 |
| C1 | Q3 | 5 | B | 145.528 |
| C1 | Q7 | 5 | A | 388.449 |
| C1 | Q7 | 5 | B | 390.472 |
| C1 | Q13 | 5 | A | 459.964 |
| C1 | Q13 | 5 | B | 462.119 |
| C1 | Q17 | 5 | A | 292.691 |
| C1 | Q17 | 5 | B | 277.124 |
| C1 | Q19 | 5 | A | 210.255 |
| C1 | Q19 | 5 | B | 211.269 |
| C1 | Q25 | 5 | A | 354.752 |
| C1 | Q25 | 5 | B | 354.024 |
| C1 | Q29 | 5 | A | 287.953 |
| C1 | Q29 | 5 | B | 283.471 |
| C1 | Q42 | 5 | A | 144.695 |
| C1 | Q42 | 5 | B | 141.589 |
| C1 | Q52 | 5 | A | 150.491 |
| C1 | Q52 | 5 | B | 146.455 |
| C1 | Q55 | 5 | A | 143.938 |
| C1 | Q55 | 5 | B | 144.481 |
| C1 | Q72 | 5 | A | 1363.989 |
| C1 | Q72 | 5 | B | 1386.571 |
| C1 | Q82 | 5 | A | 349.198 |
| C1 | Q82 | 5 | B | 363.567 |
| C1 | Q96 | 5 | A | 89.659 |
| C1 | Q96 | 5 | B | 91.040 |
| C1 | D1 | 5 | A | 376.422 |
| C1 | D1 | 5 | B | 157.786 |
| C2 | Q3 | 1 | A | 140.449 |
| C2 | Q3 | 1 | B | 144.952 |
| C2 | Q7 | 1 | A | 380.068 |
| C2 | Q7 | 1 | B | 384.166 |
| C2 | Q13 | 1 | A | 453.019 |
| C2 | Q13 | 1 | B | 453.472 |
| C2 | Q17 | 1 | A | 271.375 |
| C2 | Q17 | 1 | B | 269.430 |
| C2 | Q19 | 1 | A | 208.208 |
| C2 | Q19 | 1 | B | 206.711 |
| C2 | Q25 | 1 | A | 354.884 |
| C2 | Q25 | 1 | B | 343.782 |
| C2 | Q29 | 1 | A | 272.152 |
| C2 | Q29 | 1 | B | 280.976 |
| C2 | Q42 | 1 | A | 143.541 |
| C2 | Q42 | 1 | B | 140.206 |
| C2 | Q52 | 1 | A | 144.011 |
| C2 | Q52 | 1 | B | 148.081 |
| C2 | Q55 | 1 | A | 141.222 |
| C2 | Q55 | 1 | B | 145.483 |
| C2 | Q72 | 1 | A | 1349.611 |
| C2 | Q72 | 1 | B | 1367.826 |
| C2 | Q82 | 1 | A | 348.322 |
| C2 | Q82 | 1 | B | 348.998 |
| C2 | Q96 | 1 | A | 92.629 |
| C2 | Q96 | 1 | B | 91.566 |
| C2 | D2 | 1 | A | 369.558 |
| C2 | D2 | 1 | B | 167.770 |
| C2 | Q3 | 2 | B | 142.831 |
| C2 | Q3 | 2 | A | 147.322 |
| C2 | Q7 | 2 | B | 381.347 |
| C2 | Q7 | 2 | A | 391.021 |
| C2 | Q13 | 2 | B | 473.401 |
| C2 | Q13 | 2 | A | 458.773 |
| C2 | Q17 | 2 | B | 278.646 |
| C2 | Q17 | 2 | A | 280.106 |
| C2 | Q19 | 2 | B | 210.382 |
| C2 | Q19 | 2 | A | 219.228 |
| C2 | Q25 | 2 | B | 353.646 |
| C2 | Q25 | 2 | A | 347.326 |
| C2 | Q29 | 2 | B | 283.881 |
| C2 | Q29 | 2 | A | 285.423 |
| C2 | Q42 | 2 | B | 144.261 |
| C2 | Q42 | 2 | A | 144.887 |
| C2 | Q52 | 2 | B | 154.472 |
| C2 | Q52 | 2 | A | 149.364 |
| C2 | Q55 | 2 | B | 145.078 |
| C2 | Q55 | 2 | A | 149.314 |
| C2 | Q72 | 2 | B | 1396.799 |
| C2 | Q72 | 2 | A | 1380.563 |
| C2 | Q82 | 2 | B | 347.037 |
| C2 | Q82 | 2 | A | 348.790 |
| C2 | Q96 | 2 | B | 91.038 |
| C2 | Q96 | 2 | A | 90.441 |
| C2 | D2 | 2 | B | 364.451 |
| C2 | D2 | 2 | A | 378.450 |
| C2 | Q3 | 3 | A | 146.382 |
| C2 | Q3 | 3 | B | 142.836 |
| C2 | Q7 | 3 | A | 393.537 |
| C2 | Q7 | 3 | B | 391.289 |
| C2 | Q13 | 3 | A | 477.184 |
| C2 | Q13 | 3 | B | 470.162 |
| C2 | Q17 | 3 | A | 280.792 |
| C2 | Q17 | 3 | B | 275.195 |
| C2 | Q19 | 3 | A | 216.842 |
| C2 | Q19 | 3 | B | 218.255 |
| C2 | Q25 | 3 | A | 355.617 |
| C2 | Q25 | 3 | B | 355.687 |
| C2 | Q29 | 3 | A | 288.733 |
| C2 | Q29 | 3 | B | 288.661 |
| C2 | Q42 | 3 | A | 149.192 |
| C2 | Q42 | 3 | B | 143.577 |
| C2 | Q52 | 3 | A | 156.906 |
| C2 | Q52 | 3 | B | 154.383 |
| C2 | Q55 | 3 | A | 146.512 |
| C2 | Q55 | 3 | B | 146.376 |
| C2 | Q72 | 3 | A | 1350.820 |
| C2 | Q72 | 3 | B | 1380.919 |
| C2 | Q82 | 3 | A | 346.257 |
| C2 | Q82 | 3 | B | 349.253 |
| C2 | Q96 | 3 | A | 92.227 |
| C2 | Q96 | 3 | B | 90.405 |
| C2 | D2 | 3 | A | 164.580 |
| C2 | D2 | 3 | B | 369.568 |
| C2 | Q3 | 4 | B | 144.769 |
| C2 | Q3 | 4 | A | 143.479 |
| C2 | Q7 | 4 | B | 386.732 |
| C2 | Q7 | 4 | A | 394.957 |
| C2 | Q13 | 4 | B | 474.784 |
| C2 | Q13 | 4 | A | 462.351 |
| C2 | Q17 | 4 | B | 278.212 |
| C2 | Q17 | 4 | A | 279.024 |
| C2 | Q19 | 4 | B | 212.298 |
| C2 | Q19 | 4 | A | 211.063 |
| C2 | Q25 | 4 | B | 354.208 |
| C2 | Q25 | 4 | A | 353.842 |
| C2 | Q29 | 4 | B | 285.890 |
| C2 | Q29 | 4 | A | 289.008 |
| C2 | Q42 | 4 | B | 145.812 |
| C2 | Q42 | 4 | A | 142.695 |
| C2 | Q52 | 4 | B | 153.845 |
| C2 | Q52 | 4 | A | 150.268 |
| C2 | Q55 | 4 | B | 145.638 |
| C2 | Q55 | 4 | A | 147.457 |
| C2 | Q72 | 4 | B | 1364.210 |
| C2 | Q72 | 4 | A | 1398.951 |
| C2 | Q82 | 4 | B | 357.387 |
| C2 | Q82 | 4 | A | 355.815 |
| C2 | Q96 | 4 | B | 90.712 |
| C2 | Q96 | 4 | A | 92.204 |
| C2 | D2 | 4 | B | 371.228 |
| C2 | D2 | 4 | A | 365.232 |
| C2 | Q3 | 5 | A | 159.938 |
| C2 | Q3 | 5 | B | 146.792 |
| C2 | Q7 | 5 | A | 402.791 |
| C2 | Q7 | 5 | B | 401.886 |
| C2 | Q13 | 5 | A | 472.417 |
| C2 | Q13 | 5 | B | 464.697 |
| C2 | Q17 | 5 | A | 276.186 |
| C2 | Q17 | 5 | B | 281.513 |
| C2 | Q19 | 5 | A | 214.593 |
| C2 | Q19 | 5 | B | 210.577 |
| C2 | Q25 | 5 | A | 356.861 |
| C2 | Q25 | 5 | B | 360.634 |
| C2 | Q29 | 5 | A | 287.504 |
| C2 | Q29 | 5 | B | 283.985 |
| C2 | Q42 | 5 | A | 146.353 |
| C2 | Q42 | 5 | B | 144.598 |
| C2 | Q52 | 5 | A | 150.100 |
| C2 | Q52 | 5 | B | 148.459 |
| C2 | Q55 | 5 | A | 145.773 |
| C2 | Q55 | 5 | B | 147.225 |
| C2 | Q72 | 5 | A | 1347.075 |
| C2 | Q72 | 5 | B | 1379.192 |
| C2 | Q82 | 5 | A | 355.741 |
| C2 | Q82 | 5 | B | 347.907 |
| C2 | Q96 | 5 | A | 92.095 |
| C2 | Q96 | 5 | B | 92.762 |
| C2 | D2 | 5 | A | 361.319 |
| C2 | D2 | 5 | B | 162.052 |
| C3 | Q3 | 1 | A | 143.838 |
| C3 | Q3 | 1 | B | 147.828 |
| C3 | Q7 | 1 | A | 385.528 |
| C3 | Q7 | 1 | B | 385.969 |
| C3 | Q13 | 1 | A | 462.876 |
| C3 | Q13 | 1 | B | 469.505 |
| C3 | Q17 | 1 | A | 276.654 |
| C3 | Q17 | 1 | B | 275.708 |
| C3 | Q19 | 1 | A | 213.995 |
| C3 | Q19 | 1 | B | 212.383 |
| C3 | Q25 | 1 | A | 350.395 |
| C3 | Q25 | 1 | B | 352.774 |
| C3 | Q29 | 1 | A | 287.388 |
| C3 | Q29 | 1 | B | 280.856 |
| C3 | Q42 | 1 | A | 147.354 |
| C3 | Q42 | 1 | B | 146.256 |
| C3 | Q52 | 1 | A | 148.012 |
| C3 | Q52 | 1 | B | 147.974 |
| C3 | Q55 | 1 | A | 142.654 |
| C3 | Q55 | 1 | B | 144.428 |
| C3 | Q72 | 1 | A | 1352.516 |
| C3 | Q72 | 1 | B | 1386.678 |
| C3 | Q82 | 1 | A | 355.173 |
| C3 | Q82 | 1 | B | 346.486 |
| C3 | Q96 | 1 | A | 90.528 |
| C3 | Q96 | 1 | B | 92.406 |
| C3 | D3 | 1 | A | 213.775 |
| C3 | D3 | 1 | B | 175.904 |
| C3 | Q3 | 2 | B | 143.649 |
| C3 | Q3 | 2 | A | 148.690 |
| C3 | Q7 | 2 | B | 383.065 |
| C3 | Q7 | 2 | A | 380.904 |
| C3 | Q13 | 2 | B | 470.390 |
| C3 | Q13 | 2 | A | 467.996 |
| C3 | Q17 | 2 | B | 279.166 |
| C3 | Q17 | 2 | A | 277.221 |
| C3 | Q19 | 2 | B | 216.734 |
| C3 | Q19 | 2 | A | 219.234 |
| C3 | Q25 | 2 | B | 357.086 |
| C3 | Q25 | 2 | A | 351.216 |
| C3 | Q29 | 2 | B | 283.767 |
| C3 | Q29 | 2 | A | 285.298 |
| C3 | Q42 | 2 | B | 144.700 |
| C3 | Q42 | 2 | A | 144.893 |
| C3 | Q52 | 2 | B | 152.225 |
| C3 | Q52 | 2 | A | 153.677 |
| C3 | Q55 | 2 | B | 146.715 |
| C3 | Q55 | 2 | A | 146.738 |
| C3 | Q72 | 2 | B | 1359.548 |
| C3 | Q72 | 2 | A | 1438.394 |
| C3 | Q82 | 2 | B | 345.175 |
| C3 | Q82 | 2 | A | 349.696 |
| C3 | Q96 | 2 | B | 91.648 |
| C3 | Q96 | 2 | A | 89.571 |
| C3 | D3 | 2 | B | 176.433 |
| C3 | D3 | 2 | A | 211.525 |
| C3 | Q3 | 3 | A | 145.060 |
| C3 | Q3 | 3 | B | 143.178 |
| C3 | Q7 | 3 | A | 386.213 |
| C3 | Q7 | 3 | B | 402.068 |
| C3 | Q13 | 3 | A | 467.670 |
| C3 | Q13 | 3 | B | 470.949 |
| C3 | Q17 | 3 | A | 278.097 |
| C3 | Q17 | 3 | B | 278.001 |
| C3 | Q19 | 3 | A | 215.962 |
| C3 | Q19 | 3 | B | 215.260 |
| C3 | Q25 | 3 | A | 356.260 |
| C3 | Q25 | 3 | B | 361.252 |
| C3 | Q29 | 3 | A | 286.757 |
| C3 | Q29 | 3 | B | 288.625 |
| C3 | Q42 | 3 | A | 147.212 |
| C3 | Q42 | 3 | B | 144.469 |
| C3 | Q52 | 3 | A | 152.423 |
| C3 | Q52 | 3 | B | 152.584 |
| C3 | Q55 | 3 | A | 148.437 |
| C3 | Q55 | 3 | B | 145.682 |
| C3 | Q72 | 3 | A | 1384.417 |
| C3 | Q72 | 3 | B | 1394.090 |
| C3 | Q82 | 3 | A | 348.340 |
| C3 | Q82 | 3 | B | 350.663 |
| C3 | Q96 | 3 | A | 91.193 |
| C3 | Q96 | 3 | B | 93.827 |
| C3 | D3 | 3 | A | 211.847 |
| C3 | D3 | 3 | B | 186.382 |
| C3 | Q3 | 4 | B | 144.931 |
| C3 | Q3 | 4 | A | 145.613 |
| C3 | Q7 | 4 | B | 398.820 |
| C3 | Q7 | 4 | A | 392.707 |
| C3 | Q13 | 4 | B | 471.512 |
| C3 | Q13 | 4 | A | 491.067 |
| C3 | Q17 | 4 | B | 275.074 |
| C3 | Q17 | 4 | A | 279.370 |
| C3 | Q19 | 4 | B | 213.704 |
| C3 | Q19 | 4 | A | 214.631 |
| C3 | Q25 | 4 | B | 358.677 |
| C3 | Q25 | 4 | A | 359.556 |
| C3 | Q29 | 4 | B | 284.779 |
| C3 | Q29 | 4 | A | 281.839 |
| C3 | Q42 | 4 | B | 144.958 |
| C3 | Q42 | 4 | A | 142.834 |
| C3 | Q52 | 4 | B | 152.798 |
| C3 | Q52 | 4 | A | 154.628 |
| C3 | Q55 | 4 | B | 146.574 |
| C3 | Q55 | 4 | A | 143.393 |
| C3 | Q72 | 4 | B | 1365.393 |
| C3 | Q72 | 4 | A | 1390.942 |
| C3 | Q82 | 4 | B | 347.582 |
| C3 | Q82 | 4 | A | 354.104 |
| C3 | Q96 | 4 | B | 90.916 |
| C3 | Q96 | 4 | A | 91.038 |
| C3 | D3 | 4 | B | 179.405 |
| C3 | D3 | 4 | A | 212.362 |
| C3 | Q3 | 5 | A | 143.722 |
| C3 | Q3 | 5 | B | 145.566 |
| C3 | Q7 | 5 | A | 392.497 |
| C3 | Q7 | 5 | B | 397.473 |
| C3 | Q13 | 5 | A | 487.558 |
| C3 | Q13 | 5 | B | 477.143 |
| C3 | Q17 | 5 | A | 280.617 |
| C3 | Q17 | 5 | B | 281.618 |
| C3 | Q19 | 5 | A | 221.811 |
| C3 | Q19 | 5 | B | 213.506 |
| C3 | Q25 | 5 | A | 350.116 |
| C3 | Q25 | 5 | B | 359.556 |
| C3 | Q29 | 5 | A | 282.055 |
| C3 | Q29 | 5 | B | 287.609 |
| C3 | Q42 | 5 | A | 148.076 |
| C3 | Q42 | 5 | B | 148.504 |
| C3 | Q52 | 5 | A | 152.960 |
| C3 | Q52 | 5 | B | 151.998 |
| C3 | Q55 | 5 | A | 144.455 |
| C3 | Q55 | 5 | B | 146.729 |
| C3 | Q72 | 5 | A | 1375.539 |
| C3 | Q72 | 5 | B | 1367.740 |
| C3 | Q82 | 5 | A | 360.113 |
| C3 | Q82 | 5 | B | 354.839 |
| C3 | Q96 | 5 | A | 90.430 |
| C3 | Q96 | 5 | B | 90.914 |
| C3 | D3 | 5 | A | 215.562 |
| C3 | D3 | 5 | B | 176.021 |
| C4 | Q3 | 1 | A | 142.440 |
| C4 | Q3 | 1 | B | 149.397 |
| C4 | Q7 | 1 | A | 390.001 |
| C4 | Q7 | 1 | B | 392.755 |
| C4 | Q13 | 1 | A | 463.714 |
| C4 | Q13 | 1 | B | 477.979 |
| C4 | Q17 | 1 | A | 276.616 |
| C4 | Q17 | 1 | B | 277.898 |
| C4 | Q19 | 1 | A | 219.950 |
| C4 | Q19 | 1 | B | 218.480 |
| C4 | Q25 | 1 | A | 352.529 |
| C4 | Q25 | 1 | B | 353.686 |
| C4 | Q29 | 1 | A | 289.276 |
| C4 | Q29 | 1 | B | 285.099 |
| C4 | Q42 | 1 | A | 142.568 |
| C4 | Q42 | 1 | B | 145.566 |
| C4 | Q52 | 1 | A | 149.266 |
| C4 | Q52 | 1 | B | 147.445 |
| C4 | Q55 | 1 | A | 146.317 |
| C4 | Q55 | 1 | B | 145.327 |
| C4 | Q72 | 1 | A | 1345.623 |
| C4 | Q72 | 1 | B | 1433.301 |
| C4 | Q82 | 1 | A | 347.653 |
| C4 | Q82 | 1 | B | 352.537 |
| C4 | Q96 | 1 | A | 90.393 |
| C4 | Q96 | 1 | B | 96.324 |
| C4 | D4 | 1 | A | 412.539 |
| C4 | D4 | 1 | B | 417.481 |
| C4 | Q3 | 2 | B | 145.595 |
| C4 | Q3 | 2 | A | 148.267 |
| C4 | Q7 | 2 | B | 387.438 |
| C4 | Q7 | 2 | A | 401.299 |
| C4 | Q13 | 2 | B | 475.409 |
| C4 | Q13 | 2 | A | 475.017 |
| C4 | Q17 | 2 | B | 276.746 |
| C4 | Q17 | 2 | A | 277.535 |
| C4 | Q19 | 2 | B | 213.899 |
| C4 | Q19 | 2 | A | 218.838 |
| C4 | Q25 | 2 | B | 354.369 |
| C4 | Q25 | 2 | A | 358.076 |
| C4 | Q29 | 2 | B | 301.437 |
| C4 | Q29 | 2 | A | 287.606 |
| C4 | Q42 | 2 | B | 149.378 |
| C4 | Q42 | 2 | A | 147.480 |
| C4 | Q52 | 2 | B | 153.272 |
| C4 | Q52 | 2 | A | 151.356 |
| C4 | Q55 | 2 | B | 145.157 |
| C4 | Q55 | 2 | A | 147.559 |
| C4 | Q72 | 2 | B | 1403.837 |
| C4 | Q72 | 2 | A | 1424.927 |
| C4 | Q82 | 2 | B | 358.864 |
| C4 | Q82 | 2 | A | 349.899 |
| C4 | Q96 | 2 | B | 93.489 |
| C4 | Q96 | 2 | A | 94.113 |
| C4 | D4 | 2 | B | 410.699 |
| C4 | D4 | 2 | A | 404.287 |
| C4 | Q3 | 3 | A | 146.191 |
| C4 | Q3 | 3 | B | 145.601 |
| C4 | Q7 | 3 | A | 396.820 |
| C4 | Q7 | 3 | B | 388.376 |
| C4 | Q13 | 3 | A | 464.328 |
| C4 | Q13 | 3 | B | 474.095 |
| C4 | Q17 | 3 | A | 279.279 |
| C4 | Q17 | 3 | B | 275.461 |
| C4 | Q19 | 3 | A | 217.989 |
| C4 | Q19 | 3 | B | 212.997 |
| C4 | Q25 | 3 | A | 355.699 |
| C4 | Q25 | 3 | B | 360.235 |
| C4 | Q29 | 3 | A | 293.584 |
| C4 | Q29 | 3 | B | 287.144 |
| C4 | Q42 | 3 | A | 147.233 |
| C4 | Q42 | 3 | B | 142.296 |
| C4 | Q52 | 3 | A | 153.980 |
| C4 | Q52 | 3 | B | 152.960 |
| C4 | Q55 | 3 | A | 147.240 |
| C4 | Q55 | 3 | B | 146.861 |
| C4 | Q72 | 3 | A | 1369.950 |
| C4 | Q72 | 3 | B | 1381.866 |
| C4 | Q82 | 3 | A | 361.870 |
| C4 | Q82 | 3 | B | 350.817 |
| C4 | Q96 | 3 | A | 92.760 |
| C4 | Q96 | 3 | B | 90.126 |
| C4 | D4 | 3 | A | 406.402 |
| C4 | D4 | 3 | B | 412.274 |
| C4 | Q3 | 4 | B | 147.833 |
| C4 | Q3 | 4 | A | 145.064 |
| C4 | Q7 | 4 | B | 392.941 |
| C4 | Q7 | 4 | A | 398.162 |
| C4 | Q13 | 4 | B | 458.999 |
| C4 | Q13 | 4 | A | 473.364 |
| C4 | Q17 | 4 | B | 279.919 |
| C4 | Q17 | 4 | A | 273.116 |
| C4 | Q19 | 4 | B | 219.165 |
| C4 | Q19 | 4 | A | 216.472 |
| C4 | Q25 | 4 | B | 355.572 |
| C4 | Q25 | 4 | A | 356.746 |
| C4 | Q29 | 4 | B | 282.681 |
| C4 | Q29 | 4 | A | 289.236 |
| C4 | Q42 | 4 | B | 146.930 |
| C4 | Q42 | 4 | A | 147.123 |
| C4 | Q52 | 4 | B | 146.180 |
| C4 | Q52 | 4 | A | 155.828 |
| C4 | Q55 | 4 | B | 148.257 |
| C4 | Q55 | 4 | A | 145.163 |
| C4 | Q72 | 4 | B | 1366.750 |
| C4 | Q72 | 4 | A | 1395.273 |
| C4 | Q82 | 4 | B | 350.561 |
| C4 | Q82 | 4 | A | 350.865 |
| C4 | Q96 | 4 | B | 91.931 |
| C4 | Q96 | 4 | A | 94.543 |
| C4 | D4 | 4 | B | 395.323 |
| C4 | D4 | 4 | A | 416.333 |
| C4 | Q3 | 5 | A | 145.156 |
| C4 | Q3 | 5 | B | 142.605 |
| C4 | Q7 | 5 | A | 397.587 |
| C4 | Q7 | 5 | B | 407.597 |
| C4 | Q13 | 5 | A | 472.586 |
| C4 | Q13 | 5 | B | 459.130 |
| C4 | Q17 | 5 | A | 277.855 |
| C4 | Q17 | 5 | B | 273.623 |
| C4 | Q19 | 5 | A | 215.256 |
| C4 | Q19 | 5 | B | 211.242 |
| C4 | Q25 | 5 | A | 350.923 |
| C4 | Q25 | 5 | B | 354.980 |
| C4 | Q29 | 5 | A | 283.470 |
| C4 | Q29 | 5 | B | 290.061 |
| C4 | Q42 | 5 | A | 147.915 |
| C4 | Q42 | 5 | B | 145.058 |
| C4 | Q52 | 5 | A | 151.396 |
| C4 | Q52 | 5 | B | 155.833 |
| C4 | Q55 | 5 | A | 145.627 |
| C4 | Q55 | 5 | B | 144.945 |
| C4 | Q72 | 5 | A | 1347.422 |
| C4 | Q72 | 5 | B | 1367.505 |
| C4 | Q82 | 5 | A | 358.905 |
| C4 | Q82 | 5 | B | 363.043 |
| C4 | Q96 | 5 | A | 91.419 |
| C4 | Q96 | 5 | B | 89.483 |
| C4 | D4 | 5 | A | 401.409 |
| C4 | D4 | 5 | B | 412.087 |
</details>
**SQL for the two accelerated directed cases**
D1 — selective build followed by partitioned fallback:
```sql
SELECT SUM(s.ss_ext_sales_price * c.c_birth_day) AS revenue
FROM customer c JOIN store_sales s ON c.c_customer_sk = s.ss_customer_sk
WHERE c.c_birth_month = 1 AND c.c_birth_day = 1
AND s.ss_quantity BETWEEN 10 AND 90;
```
D3 — ready, nearly saturated Bloom:
```sql
SELECT SUM(s.ss_ext_sales_price * c.c_birth_day) AS revenue
FROM customer c JOIN store_sales s ON c.c_customer_sk = s.ss_customer_sk
WHERE CAST(c.c_birth_month AS VARCHAR) IN ('1', '2')
AND s.ss_quantity BETWEEN 10 AND 90;
```
These forced-plan settings were identical on A and B:
| Setting | D1 | D3 |
|---|---:|---:|
| `ballista.optimizer.broadcast_join_threshold_bytes` | 4096 | 1500000 |
| `ballista.optimizer.stage_build_side_max_estimate_multiple` | 1024 | 1024 |
| `datafusion.optimizer.default_filter_selectivity` | default | 100 |
| `datafusion.optimizer.hash_join_single_partition_threshold` | default | 0 |
| `datafusion.optimizer.hash_join_single_partition_threshold_rows` | default
| 0 |
D1 forces the measured build above the broadcast limit, exposing the missing
fallback consumer. D3 uses a pessimistic filter estimate to stage the build,
then broadcasts its measured result; both arms receive a complete Bloom, and C3
bypasses the near-full one.
Common settings: staged execution, AQE and runtime filtering enabled;
`prefer_hash_join=true`; coalescing, skew optimization and local dynamic filter
pushdown disabled. `stage_build_side=true` for the standard queries, D1 and D3.
D2 uses D1’s SQL; D4 uses the same SELECT/JOIN and quantity predicate with
`c.c_birth_month <= 11`. D2/D4 disable `stage_build_side` and set both
broadcast byte/row thresholds to zero to create sibling stages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
