vito1317 opened a new pull request, #23567:
URL: https://github.com/apache/datafusion/pull/23567
## Which issue does this PR close?
- Part of #15511.
(Running literally *all* benchmarks is left for follow-ups once the approach
looks right — see "coverage" below.)
## Rationale for this change
Benchmarks keep silently breaking because nothing exercises them in CI (see
#15511). This adds a smoke-test job to the extended tests workflow so breakage
is caught on merge to `main` instead of by the next person who tries to run
them.
## What changes are included in this PR?
- A new `benchmarks-smoke` job in `.github/workflows/extended.yml` that:
1. `cargo check`s every benchmark bin and bench target — this alone
catches the most common "benchmark silently broken" failures cheaply, then
2. runs a representative subset end-to-end on small data, one query each,
as a "does it still run" check (explicitly **not** a performance measurement):
`tpch`, `tpch_mem`, `sort_tpch`, `external_aggr` on tpchgen-generated TPC-H
SF1, plus `cancellation` (which generates its own data). Runs use the
`ci-optimized` profile so debug assertions stay enabled.
The job only runs on push to `main`/`branch-*` and manual dispatch, not on
pull requests, to conserve CI resources.
- A one-line `bench.sh` fix: the TPC-H answers copy used `docker run -it`,
which fails without a TTY in CI (`the input device is not a TTY`).
**Coverage:** benchmarks requiring multi-GB downloads (clickbench, h2o,
imdb, tpcds) are intentionally left out of this first pass. Happy to extend
coverage, change the subset, or gate the job differently based on feedback.
## Are these changes tested?
Yes — validated by running the modified workflow end-to-end on a fork:
[run](https://github.com/vito1317/datafusion/actions/runs/29319876025/job/87042402974)
(all steps green, including the docker answers-copy path and the tpchgen data
generation). `actionlint` passes on the workflow file.
Timing data from that run (free 4-core `ubuntu-latest`): compile check 2m,
tpchgen install 3m, TPC-H SF1 data generation 11s, smoke runs 72m (dominated by
the `ci-optimized` compile). On the repository's `runs-on` runners (16 cores)
the compile-heavy part should shrink substantially; if it is still too slow I
can drop the run subset to compile-check only, or trim it further.
## Are there any user-facing changes?
No. (`bench.sh data tpch` no longer allocates a TTY for the answers-copy
container; behaviour is otherwise unchanged.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
