[
https://issues.apache.org/jira/browse/HUDI-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen closed HUDI-9535.
----------------------------
Reviewers: Danny Chen
Resolution: Fixed
Fixed via master branch: 1f561d08d7c59d58781ccc2be100eec5b7c14573
> Prevent Broadcast of full Partitioner during mapPartitionsAsRDD
> ---------------------------------------------------------------
>
> Key: HUDI-9535
> URL: https://issues.apache.org/jira/browse/HUDI-9535
> Project: Apache Hudi
> Issue Type: Improvement
> Components: spark, writer-core
> Reporter: Jonathan Vexler
> Assignee: Jonathan Vexler
> Priority: Blocker
> Fix For: 1.1.0
>
>
> Partitioner is used on the executor which can result in a large amount of
> data being serialized to the executors for large tables. Most of the data is
> unused so we can pull what we need into a lighter weight class.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
