felipepessoto commented on PR #12969: URL: https://github.com/apache/gluten/pull/12969#issuecomment-5737390588
Closing this draft as superseded by #13036 and #13058. The investigation and green CI here confirmed that `parquet.block.size` set through Spark’s runtime Hadoop configuration is propagated to Gluten’s native Parquet writer. The Delta fixture failures were therefore not caused by a missing configuration. They occur when the fixture is delivered as one Arrow batch, because Velox evaluates the byte-based flush threshold only after that batch has been written and cannot split it into multiple row groups. #13036 and #13058 address the affected Delta fixtures deterministically with scoped native row-count limits and validate the resulting downstream tests in the full Delta Spark UT pipeline. The additional 1M-row regression suites in this PR are no longer necessary. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
