[ https://issues.apache.org/jira/browse/HUDI-7906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated HUDI-7906: --------------------------------- Labels: pull-request-available (was: ) > improve the parallelism deduce in rdd write > ------------------------------------------- > > Key: HUDI-7906 > URL: https://issues.apache.org/jira/browse/HUDI-7906 > Project: Apache Hudi > Issue Type: Improvement > Reporter: KnightChess > Assignee: KnightChess > Priority: Major > Labels: pull-request-available > > as [https://github.com/apache/hudi/issues/11274] and > [https://github.com/apache/hudi/pull/11463] describe, there has two case > question. > # if the rdd is input rdd without shuffle, the partitiion number is too > bigger or too small > # user need can not control it easy > ## in some case user can set `spark.default.parallelism` change it. > ## in some case user can not change because hard-code > ## and in spark, the better way is use `spark.default.parallelism` or > `spark.sql.shuffle.partitions` can control it, other is advanced in hudi. -- This message was sent by Atlassian Jira (v8.20.10#820010)