On Sun, May 24, 2026 at 12:13 PM Steven Wu <[email protected]> wrote: > > Kevin's PR of removing Spark 3.4 was merged a few days ago. It should reduce > the Spark CI cost by ~25%. > > Some heavy-hitter test classes in Spark tests (core and extension) cause high > load due to parameter combinations. I asked AI to analyze the build log and > recommend changes offering the best ROI. Details are in this doc. > > I can look into dropping some combinations without sacrificing essential > coverage. E.g., we can probably drop the Hadoop catalog usage in test, as it > wasn't recommended for production use anyway.
iceberg-cpp skips Actions for draft PRs [1] to reduce CI resource usage a little bit. Perhaps we should apply the same approach across all iceberg subprojects? [1] https://github.com/apache/iceberg-cpp/pull/680 > > > > On Fri, May 22, 2026 at 8:22 AM Matt Butrovich <[email protected]> wrote: >> >> Apache DataFusion similarly received this notice. For visibility to the >> Iceberg community, we have tracking issues to try to discuss solutions: >> >> https://github.com/apache/datafusion/issues/22455 >> https://github.com/apache/datafusion-comet/issues/4406 >> >> DataFusion Comet is consuming the vast majority of DataFusion resources, and >> like the Iceberg project it's due to Spark tests (and Iceberg's Spark >> tests). We are doing some analysis on what subsets might be appropriate for >> our workflows, features, and goals, and will share anything that we think >> might translate back to the Iceberg CI workflows. >> >> On Fri, May 22, 2026 at 7:43 AM Robert Thomson <[email protected]> wrote: >>> >>> Hello, Iceberg PMC. >>> >>> In 2024, the ASF introduced the policy for GitHub Actions usage >>> across the foundation[1]. The ASF Github shared pool of >>> Github-hosted runners has been at, or very close to the limit of >>> 900 jobs most of the time in the past few weeks and this is the >>> case again today. >>> >>> Your project has been identified as being among the top 5 consumers of >>> build time over the past 7 days and we request that you bring your >>> usage down by stream-lining long-running builds. Contact Infra for >>> a consultation if you are unable to streamline your builds further. >>> >>> You can use the infra reporting tool[2] to monitor your GHA usage as you >>> work on stream-lining, as well as locate any bottlenecks in the workflows. >>> >>> Infra will allow you two weeks time (till the 8th of June, 2026) to >>> progress this, but should you still be above the limits by then, >>> without a viable path forward, we will be limiting your GHA usage. >>> >>> Kind regards, >>> Bob Thomson, on behalf of ASF Infrastructure. >>> >>> >>> [1] https://infra.apache.org/github-actions-policy.html >>> [2] >>> https://infra-reports.apache.org/#ghactions&project=iceberg&hours=24&limit=15&group=name >>> -- Regards Junwang Zhao
