andygrove opened a new pull request, #7311: URL: https://github.com/apache/datafusion-comet/pull/7311
## Which issue does this PR close? Closes #4289. Supersedes #4570. ## Rationale for this change Nearly every pull request comes from a fork (194 of the 198 open ones), and the pull request tier plus the `run-*` label runs were about 60% of the roughly 2,900 runner-hours that `ci.yml` and `ci_label.yml` spent in the week to 2026-10-28. The merge queue was about 30% and the nightly about 8%. `apache/datafusion` (apache/datafusion#25591) and `apache/arrow-rs` (apache/arrow-rs#11788) moved this work onto the contributor's fork, using the fork's own `push` trigger rather than a separate fork-only wrapper workflow, and treat the fork's result as informational. This applies the same arrangement to Comet. The merge queue stays on ASF runners and still runs every suite on the merge result, so nothing reaches `main` on the strength of a fork run. ## What changes are included in this PR? - `compute-changes.py`: a push to a fork gets the `pr` tier, and a pull request from a fork into `main` loses it upstream. The `run-*` label suites and the `labeled` run are unchanged. So are a pull request from a branch of this repository, a pull request to a release branch, and one opened inside a fork. - `ci_fork.yml` (new): calls `ci.yml` on a push to any fork branch except `main`. It is a separate workflow, as `ci_label.yml` is, so a push to a branch of this repository cannot publish a skipped `Required Checks` next to a pull request's real one. Upstream its only job is skipped. A fork's push to `main` already goes through `ci.yml`, where it runs the PR tier for what changed and writes the fork's cache entries. - `ci.yml`: a fork push diffs against the fork's default branch, since there is no pull request to diff against. The concurrency group is keyed on the branch for a fork push, so the next push cancels the run in flight. Preflight runs the bridge tests. - `fork_ci_status.yml` (new): mirrors the fork's run onto the pull request as an informational `CI (fork)` check, on `workflow_run` and every 15 minutes. It is adapted from arrow-rs, never checks out pull request code, and needs only `checks: write`. `dev/ci/test-fork-ci-status.cjs` covers it against mocked GitHub APIs (33 tests). - `check-ci-config.py`: policy cases for fork pushes and fork pull requests, the new environment wiring, and a check that the caller, the bridge and `ci.yml`'s push trigger agree. Each guard was mutation-tested. - Docs: a "Pull requests from forks" section in the CI guide, a pointer in the development guide, and a "Fork runs" section in the workflows README. `CI (fork)` is advisory, not required, because a contributor controls the workflow that produced it. A green fork run that never ran the tests costs one queue slot. The earlier attempts mirrored the fork's result into a required check; arrow-rs and DataFusion decided against that, and so does this PR. ## How are these changes tested? - `python3 dev/ci/check-ci-config.py`, `node --test dev/ci/test-fork-ci-status.cjs` and `actionlint --shellcheck=off` pass locally. I removed each new guard in turn and confirmed the check fails. - End to end on a fork (the author's), with this branch's workflows: a push to a branch started only `Comet CI (fork run)`, which ran the full PR tier and passed. With cold caches it took 59 minutes of wall time and 240 runner-minutes, with the native build at 23 minutes. With the fork's `main` synced first it took 50 minutes and 213 runner-minutes, with the native build at under 5. For comparison, successful full PR runs on ASF runners over the previous week had a median wall time of 81 minutes. No job waited for a runner. This is one account and two runs. - Not tested on GitHub: `fork_ci_status.yml` and the upstream skip of the PR tier. `workflow_run` and `schedule` use the copy on the default branch, and the upstream skip needs a pull request from a fork, so neither can run until this merges. The routing is covered by the policy cases and the bridge by its mocked tests, as in the arrow-rs and DataFusion changes. Contributors need to enable Actions on their fork and keep its `main` in sync; the CI guide says so, and `CI (fork)` says "Enable CI on your fork" when it finds no run. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
