andygrove opened a new pull request, #4282: URL: https://github.com/apache/datafusion-comet/pull/4282
## Which issue does this PR close? Supersedes #4276 (combines that change with the heap cap needed to actually pass on `ubuntu-latest`). Closes #. ## Rationale for this change The ASF runs-on.com app is currently disabled, so PR jobs that target the runs-on.com runner pool queue indefinitely and we cannot merge anything. #4276 gates the runs-on.com selector behind `vars.USE_RUNS_ON` so jobs fall back to `ubuntu-latest` when the variable is unset. That is enough to dispatch jobs again, but several jobs then OOM on `ubuntu-latest` (16 GB) because the `verify-benchmark-results-tpch` and `verify-benchmark-results-tpcds` data-generation steps were calling `MAVEN_OPTS='-Xmx20g' ../mvnw exec:java ...`. Asking for a 20 GB JVM heap on a 16 GB box guarantees an OOM. ## What changes are included in this PR? On top of #4276's `USE_RUNS_ON` gate: - Add a job-level `MAVEN_OPTS: -Xmx6g -XX:MaxMetaspaceSize=1g` env to `verify-benchmark-results-tpch` and `verify-benchmark-results-tpcds`. This applies to every `mvnw` invocation in those jobs (build, data gen, query test). - Drop the inline `MAVEN_OPTS='-Xmx20g'` prefix from the two TPC-H / TPC-DS data-generation steps so they pick up the job-level value. SF=1 data gen needs nowhere near 20 GB. The `linux-test` matrix already pins `MAVEN_OPTS="-Xmx4G -Xms2G ..."` inside the `java-test` composite action, so it is not changed here. ## How are these changes tested? CI on this PR runs the same affected jobs against `ubuntu-latest` (since `USE_RUNS_ON` is unset on this branch / on forks), exercising the new heap cap end-to-end. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
