andygrove opened a new issue, #6390: URL: https://github.com/apache/datafusion-comet/issues/6390
### What is the problem the feature request solves? Every macOS build in the merge queue compiles the native library from scratch, which takes about 31 minutes on `macos-14`. `pr_build_macos.yml` saves its cargo cache only when `github.ref == 'refs/heads/main'`, but the macOS workflow runs only in the merge queue and on pull requests with the `run-macos-tests` label, never on main. So the entry is never written and every restore misses. Today the macOS path (31 minute native build plus the 39 minute `[expressions]` suites, about 70 minutes) is shorter than the Spark SQL path, but once the Spark SQL shards are rebalanced (#6388) it becomes the longest path in a queue run. ### Describe the potential solution Give the macOS build the same push-to-main cache refresh that the Linux build has: - a `build_macos_full` POLICY output next to `build_macos`, the way `build_linux_full` sits next to `build_linux`, with `build_macos` gaining the `push` tier - a `cache-refresh-only` input on `pr_build_macos.yml` that skips the test matrix on push - in that mode, `build-native` checks for the entry with `lookup-only` and builds and saves only when the key is missing Key the entry on `Cargo.lock` and `Cargo.toml` only, as #6387 does for Linux. The push run then builds once per dependency change and otherwise finishes in about a minute. ### Additional context This should land after #6387. The macOS entry adds about 1.4 GB, and the cache budget has no room for it until #6387 frees some. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
