philo-he commented on issue #12288: URL: https://github.com/apache/gluten/issues/12288#issuecomment-4715031110
> One way is to reduce the meta/velox rebases. Like once per week. After the rebase PR is merged, we build velox libraries and cache them in docker. > > We still open the rebase PR every day but don't need to run all CIs. > > Is it feasible? @FelixYBW, we already share the ccache produced when a PR is merged, which relieves the time-consuming native build on a per-PR basis. Lowering the frequency of Velox rebases may further increase the cache hit rate, though I'm not sure how much it would help. cc @zhouyuan > Given that supporting Spark 3.2–4.1 significantly increases the number of CI jobs, check if we can evaluate dropping older versions (starting with Spark 3.2 and potentially 3.3, I can see 3.3 still in the CI workflow but may only a few customer usage) while focusing on actively used and forward-looking versions (3.4+ and 4.x). > > Perhaps we should do like the other projects — keep support for the LTS (3.5) and the latest 2 versions (4.0, 4.1). And for the other versions, we keep only a single test CI, or just running them once every night instead? @weiting-chen, @jackylee-ch, We've already dropped Spark 3.2. For Spark 3.3, and even 3.4, we could indeed consider reducing their CI jobs, while still keeping some for PR verification to ensure no PR introduces obvious issues on the lower versions. Beyond that, I also feel there's some redundancy in the CI jobs, e.g., testing all JDK versions on both Ubuntu 20.04 and 22.04. If anyone in the community can help create a fix PR, please kindly do so. Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
