zhztheplayer opened a new pull request, #8816:
URL: https://github.com/apache/incubator-gluten/pull/8816
This adds supports for the following write options to 3.4 / 3.5 Parquet
write:
- compression code:
- sql conf: `spark.sql.parquet.compression.codec`
- option: `compression.codec`
- block size
- sql conf: `spark.gluten.sql.columnar.parquet.write.blockSize`
- option: `parquet.block.size`
- block rows
- sql conf: `spark.gluten.sql.native.parquet.write.blockRows`
- option: `parquet.block.rows`
The options were supported in 3.2 / 3.3. The patch adds them to 3.4 / 3.5 to
make sure the
[document](https://github.com/apache/incubator-gluten/blob/e9bff4f27f341570bec270e0f327e59484b737bf/docs/velox-backend-limitations.md?plain=1#L132-L140)
applies for all Spark versions.
With some other minor code cleanups.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
