felipepessoto opened a new pull request, #13058: URL: https://github.com/apache/gluten/pull/13058
## What changes are proposed in this pull request? References #12743. Patch Delta v4.2.0 after cloning so `DeletionVectorsWithPredicatePushdownSuite.beforeAll` scopes `spark.gluten.sql.native.parquet.write.blockRows=500000` around its one-million-row fixture write. The fixture keeps its existing `parquet.block.size=2 MiB`, native write path, one input partition, one Delta data file, and one-million-row cardinality. Velox evaluates its buffered-byte row-group flush threshold only after writing an input Arrow batch, so a single batch can remain one row group despite the byte limit. Arrow honors the native row-count limit while writing the batch, deterministically producing at least two row groups. The embedded patch matches the complete v4.2.0 fixture write and fails setup on source drift. The two existing suite-abort baseline entries remain until CI proves which tests proceed to pass or expose downstream DV correctness failures. The latest successful full Delta run before this change (`35309296010`) contained 27 test failures and one suite abort with `1 was not greater than 1`: - 24 `DeltaParquetFileFormatSuite` / `DeltaParquetFileFormatWithPredicatePushdownSuite` test failures at `DeltaParquetFileFormatSuite.scala:87`, all from the 20,000-row/one-file/50 KiB `generateData` fixture already fixed by #13036. - 3 `AutoCompactExecution*Suite` test failures at `AutoCompactSuite.scala:274`; these assert `numOfFiles > 1` after a 10-row write and are unrelated to Parquet row groups. - 1 `DeletionVectorsWithPredicatePushdownSuite` abort in `beforeAll` at `DeletionVectorsSuite.scala:911/930`, from the one-million-row/one-partition/one-file/2 MiB fixture fixed here. ## How was this patch tested? - Downloaded and parsed all eight JUnit report artifacts from successful scheduled run `35309296010` (1,535 XML reports) rather than inferring failures from baseline names. - `bash -n .github/workflows/util/delta-spark-ut/setup-delta.sh` - Ran `setup-delta.sh` twice against Delta v4.2.0 (`9f600c42dd9bc383c025ee6e43f8199571a2783b`) and verified both fixture patches apply exactly once on each clean setup. - Perturbed the expected `DeletionVectorsWithPredicatePushdownSuite` fixture source and verified `git apply --check` rejects it. - Verified the committed baseline remains at 698 non-comment entries and retains both predicate-pushdown suite-abort entries. - `git diff --check` The Python license-header checker could not run locally because this Windows host has no Python runtime; the touched shell file retains its existing ASF header. The focused/full Delta suite and any justified baseline update will be completed in CI before the draft is marked ready. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: GitHub Copilot App 1.0.84-5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
