naamamaoz opened a new pull request, #15503: URL: https://github.com/apache/iceberg/pull/15503
## Summary - Update default Spark version from 3.5 to 4.1 and add `scala_version` parameter to construct the correct Gradle project name (e.g. `iceberg-spark-4.1_2.13`), matching the pattern used in `spark-ci.yml` - Pass `-DsparkVersions` and `-DscalaVersion` to Gradle so the correct Spark project is included in the build - Wire `extraJvmArgs` from `build.gradle` into `jmh.gradle` to pass `--add-opens` flags needed on JDK 17+ (without these, Spark benchmarks crash with `InaccessibleObjectException`) - Reduce JMH default heap from 32g to 4g to fit within GitHub Actions runner memory (7GB RAM) - Use unique artifact names per matrix job and merge them, fixing `upload-artifact@v4+` conflicts where parallel jobs uploading to the same artifact name caused failures ## Test plan - [x] Verified all 15 recurring JMH benchmarks pass end-to-end with Spark 4.1: https://github.com/ryft-io/iceberg-private/actions/runs/22628065414 - [x] Verified dispatch JMH benchmarks pass: https://github.com/ryft-io/iceberg-private/actions/runs/22628066773 - [x] Confirmed 32g heap causes OOM kills on standard runners (2/15 fail): https://github.com/ryft-io/iceberg-private/actions/runs/22630375297 - [x] Confirmed 8g heap also causes OOM kills (2/15 fail): https://github.com/ryft-io/iceberg-private/actions/runs/22631287507 - [x] 4g heap passes all 15 benchmarks reliably AI was used to help identify the root causes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
