andygrove opened a new pull request, #2419:
URL: https://github.com/apache/datafusion-ballista/pull/2419
> **Draft — not intended to merge.** Opening this as a review surface so
> the plans render inline on GitHub with per-file diffs and comments;
> I'll close it once the discussion wraps.
## What this is
A snapshot of the logical, pre-execution physical, and executed stage
plans for all 22 TPC-H queries, captured from a single run against
Ballista 54.0.0 with a SF1000 Parquet/Iceberg-partitioned dataset. Plans
were pulled from the scheduler event log added in #2264.
Each `qNN.md` has:
1. Logical plan (from `JobStart`)
2. Pre-execution physical plan (from `JobStart`)
3. One section per executed shuffle stage: `stage_plan` + input/output
rows, elapsed compute, and task duration / task input percentiles
(from `JobEnd`)
Env, config, and wall-clock summary are in the
[README](docs/source/contributors-guide/plans/tpch-sf1000/README.md).
## Wall-clock (single run, one job at a time, no warm-up)
q1 9.1s · q2 39.5s · q3 25.5s · q4 9.0s · q5 35.5s · q6 5.2s · q7 39.6s
· **q8 164.2s** · **q9 91.7s** · q10 39.6s · q11 15.2s · q12 11.3s ·
q13 12.2s · q14 5.1s · q15 10.8s · q16 16.3s · q17 25.8s · q18 50.6s ·
q19 16.0s · q20 26.2s · q21 59.6s · q22 10.0s
## Things I'd love eyes on
- **q8 / q9 dominate wall-clock.** Do the stage shapes look right for
the classic multi-join queries, or is something off (join order,
partial-agg placement, shuffle sizing)?
- **`DataSourceExec` file-groups sections are huge** in every plan —
they list every parquet file per group inline. Is there a compact
representation the community would prefer for reference material like
this?
- Anything else that jumps out as a planner regression or an obvious
win vs earlier Ballista releases.
## Not asking for
- Code review — no code changes here, just Markdown under `docs/`.
- A merge decision — happy to close once folks have had a look, unless
there's appetite for keeping a versioned plan baseline in-tree.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
