felipepessoto opened a new pull request, #13131: URL: https://github.com/apache/gluten/pull/13131
## What changes are proposed in this pull request? <!-- Provide a clear and concise description of the changes introduced in this PR. Ensure the PR description aligns with the code changes, especially after updates. If applicable, include "Fixes #<GitHub_Issue_ID>" to automatically close the corresponding issue when the PR is merged. --> Configure Gluten for every Delta test JVM, not only suites using the patched `DeltaSQLCommandTest`. Spark retains its cache serializer across Spark contexts in the same JVM. Suites using plain `SharedSparkSession`, `TestHive`, or their own session builders could therefore use Gluten's native cache serializer without the executor plugin that initializes task resources, causing `TaskResourceRegistry is not initialized`. - Add the Gluten plugin, columnar shuffle manager, and off-heap memory defaults (enabled, 2 GiB) to the shared `JAVA_TOOL_OPTIONS` in `java-test-args.sh`. Both CI and local runs inherit them before constructing Spark contexts. - Keep Delta extensions and catalog configuration suite-specific, preserving tests that intentionally omit them. - Add three JVM-bootstrap regression tests under `gluten-ut`, run them on shard 0, and document the shared configuration. - Remove the 35 confirmed-fixed entries from `known-failures.txt` without suppressing the registry error or disabling native caching. ## How was this patch tested? <!-- Describe how the changes were tested, if applicable. Include new tests to validate the functionality, if necessary. For UI-related changes, attach screenshots to demonstrate the updates. --> The new regression tests failed before the fix because all four required Spark properties were absent, and pass after the fix: ```sh python3 gluten-ut/test/src/test/python/test_delta_spark_test_args.py -v ``` Ran the four affected suites against Delta `v4.2.0`, Spark `4.1.0`, and JDK 17 using the native Gluten bundle from the [original failing CI run](https://github.com/apache/gluten/actions/runs/35897644781). **All 47 tests pass**, both with `TEST_PARALLELISM_COUNT=1` and `TEST_PARALLELISM_COUNT=4` (`NUM_SHARDS=1` for the focused runs): | Suite | Passing tests | Previously failing | |---|---:|---:| | `HiveConvertToDeltaSuite` | 28 | 20 | | `BitmapAggregatorE2ESuite` | 12 | 12 | | `DeltaExtensionAndCatalogSuite` | 6 | 2 | | `JavaDeltaSparkSessionExtensionSuite` | 1 | 1 | Verified every removed baseline entry against passing XML results, and ran the existing results gate with `--fail-on-fixed true`: no regressions or stale now-passing entries for these suites. Python formatting, shell syntax, workflow configuration checks, and the CI license-header checker also pass. The full eight-shard Delta run was not executed locally; this PR is a draft for CI follow-up. ## Was this patch authored or co-authored using generative AI tooling? <!-- If generative AI tooling has been used in the process of authoring this patch, please include the phrase: 'Generated-by: ' followed by the name of the tool and its version. If no, write 'No'. Please refer to the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html) for details. --> Generated-by: GitHub Copilot 1.0.87-0 (GPT-6 Astra) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
