andygrove opened a new pull request, #2653: URL: https://github.com/apache/datafusion-ballista/pull/2653
# Which issue does this PR close? N/A # Rationale for this change The TPC-H SF10 and TPC-DS SF1 suites run five jobs on every PR that touches code, and a passing run is 10-20 minutes. A merge queue tests the exact tree that will land, so the suites can move there and an iterating PR can run a much smaller set without losing coverage on `main`. This follows what apache/datafusion and apache/datafusion-comet did. # What changes are included in this PR? | | TPC-H / TPC-DS | | --- | --- | | Pull request | only with the `run-tpch` / `run-tpcds` label | | Merge queue | both, unless the entry changes only docs or other non-build files | | Push to `main` | not run (the queue already tested that tree); only the two build caches are refreshed | - `.asf.yaml`: a `Merge Queue` ruleset on the default branch (squash, `ALLGREEN`, `max_entries_to_build: 2`, `apache/root` bypass), written as a raw Rulesets payload as in Comet, so no INFRA ticket should be needed. `TPC Required Checks` becomes the one required status check on `main`. - `tpc.yml` (new): decides when the suites run, and holds the `TPC Required Checks` aggregator. The suites can't be required directly because a skipped job publishes under a different name than a run one. The aggregator fails if a suite that should have run didn't succeed, so a skip can't silently pass the queue. - `tpc_label.yml` (new): runs a suite as soon as its label is applied. It is a separate workflow so a label run can't hide the commit run's verdict for the required check (the problem Comet hit in apache/datafusion-comet#6159). The first run after labelling is feedback only; later pushes to a labelled PR do block it. - `tpch.yml` / `tpcds.yml`: now `workflow_call` + `workflow_dispatch` only. `concurrency` moves to the caller. - Docs: a short section in the contributors guide. `h2o.yml` and the other workflows are unchanged, so the queue gates on the TPC suites only. # Are there any user-facing changes? For contributors: - TPC suites no longer run on PRs unless labelled; the queue enforces them. - Every PR to `main` now needs `TPC Required Checks`. Ballista had no required status checks before. PRs opened before this merges won't have it until they pick up `tpc.yml` (rebase or push), and show "Expected" until then. Before merging, the `run-tpch` and `run-tpcds` labels need creating in the repo. As with Comet, a dev@ note about the queue is probably worthwhile, since a flaky suite will now block everyone's merges rather than one PR. Verified with actionlint and local runs of the path filter and label logic. The `merge_group` paths can't be exercised until the ruleset is live. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
