andygrove opened a new pull request, #6395:
URL: https://github.com/apache/datafusion-comet/pull/6395
## Which issue does this PR close?
Closes #6390.
## Rationale for this change
`pr_build_macos.yml` saved its cargo cache only when `github.ref ==
'refs/heads/main'`, but the macOS build runs only in the merge queue and on
pull requests carrying `run-macos-tests`, never on main. The entry was never
written, so every macOS build in the queue compiled the native library from
scratch, about 31 minutes on `macos-14`. Today that path (31 minutes plus the
39 minute `[expressions]` suites) is shorter than the Spark SQL path, but it
becomes the longest path in a queue run once the Spark SQL shards are
rebalanced (#6393).
## What changes are included in this PR?
The same push-to-main cache refresh the Linux build already has:
- `compute-changes.py`: `build_macos` gains the `push` tier, and a new
`build_macos_full` output (queue, or the `run-macos-tests` label) selects the
test matrix, the way `build_linux_full` does for Linux. `ci.yml` passes
`cache-refresh-only: ${{ build_macos_full != 'true' }}`.
- `pr_build_macos.yml`: a `cache-refresh-only` input. In that mode the test
matrix is skipped, and `build-native` first looks the entry up with
`lookup-only`. It builds and saves only when the dependency set has no entry
yet, so most pushes cost about a minute of a macOS runner. The key covers
`Cargo.lock` and `Cargo.toml` only, as #6392 does for Linux.
- `check-ci-config.py`: the routing and event-policy cases cover
`build_macos_full`, the push tier expects `build_macos`, and the cache-refresh
scope check now holds `pr_build_macos.yml` to the same rule as
`pr_build_linux.yml`: every job either writes a cache main needs or carries the
`!inputs.cache-refresh-only` guard, and `ci.yml` must pass the input.
- The workflows README and `ci.md` describe the macOS push run.
This should land after #6392. The macOS entry adds about 1.4 GB, and the
cache budget has no room for it until #6392 frees some.
## How are these changes tested?
`python3 dev/ci/check-ci-config.py`, `actionlint` and prettier pass locally.
I also checked that the extended cache-refresh check fails when the guard on
`macos-aarch64-test` or the `cache-refresh-only` input in `ci.yml` is removed.
I'm applying `run-macos-tests` so the normal mode runs on this branch. The
cache-refresh mode only runs on push to main, so the first pushes after the
merge are its test: the first should build and save `macOS-cargo-ci-v2-<hash>`,
and later ones should stop after the lookup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
