rok opened a new pull request, #16738: URL: https://github.com/apache/datafusion/pull/16738
## Which issue does this PR close? - Closes 16737. ## Rationale for this change #16351 added modular encryption reading and writing. This builds on top of #16351 and uses https://github.com/apache/arrow-rs/pull/7818 to enable multi threaded encrypted writing. ## What changes are included in this PR? This uses a lower level `ArrowRowGroupWriterFactory` API to create encryption-aware column writers that are then run multi threaded. ## Are these changes tested? Yes. ## Are there any user-facing changes? Previously `parquet_opts.global.allow_single_file_parallelism == true` would be ignored and encrypted write would always be single threaded. Now it will run multithreaded. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
