andygrove opened a new pull request, #2087:
URL: https://github.com/apache/datafusion-ballista/pull/2087
# Which issue does this PR close?
<!-- No tracking issue; small CI infrastructure change. -->
Closes #.
# Rationale for this change
Expensive CI workflows — TPC-H SF10, TPC-DS SF1, Docker build, and the Web
TUI
WASM build — ran on their own independent triggers, unrelated to the lint
checks. GitHub Actions `needs:` only creates dependencies between jobs in the
*same* workflow file, so a PR containing a `rustfmt` or `clippy` error still
consumed the full benchmark/build budget before the lint failure surfaced.
The goal of this PR is to make the lint suite (rustfmt + `cargo tomlfmt` +
clippy) a hard gate that must pass before any expensive job starts, so lint
errors fail fast and cheaply.
# What changes are included in this PR?
- Consolidate `tpch.yml`, `tpcds.yml`, `docker.yml`, and `web-tui.yml` into
`rust.yml` as jobs (bodies copied verbatim) and delete those four files.
- Add a small `lint-gate` aggregating job that `needs: [lint, clippy,
cargo-toml-formatting-checks]`. Every expensive job — the four merged ones
plus the existing heavy test jobs (`linux-test`, `linux-build-workspace`,
`ballista-test`, `macos-test`, `verify-benchmark-results`) — now
`needs: lint-gate`. A lint failure skips the gate, which skips all gated
jobs, so no expensive work runs.
- Leave fast jobs ungated for quick feedback: `build-lib` (cargo check),
`rustdoc`, and `datafusion-proto-sync-check`.
- Union the merged workflows' path filters onto `rust.yml`'s trigger (adds
`dev/docker/**` and `dev/build-ballista-docker.sh`).
- Restrict the workflow token to `permissions: contents: read`, and grant
`build_docker` a job-level `packages: write` for its existing release-tag
ghcr.io push.
The **Python Release Build** (`build.yml`) is intentionally left separate: it
triggers on `push: tags: [ "*-rc*" ]` (release artifacts) and `python/**`
paths, so gating release builds behind Rust lint would be the wrong coupling.
Tradeoff: for a passing PR, wall-clock increases slightly because the
expensive
jobs now wait for the clippy compile instead of running alongside it. That is
the intended cost of a hard clippy gate.
Notes for reviewers/INFRA:
- Job `name:` strings are preserved, so individual required-check names are
unchanged, but the checks now report under the **Ballista Rust** workflow
rather than their old per-workflow names. Branch-protection required-checks
may need to be confirmed/updated.
- `actionlint` reports only pre-existing shellcheck/expression warnings that
already exist on `main` in these same script blocks; the consolidation adds
no new lint findings.
# Are there any user-facing changes?
No. This only changes CI orchestration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
