[
https://issues.apache.org/jira/browse/HUDI-5303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen closed HUDI-5303.
----------------------------
Resolution: Fixed
Fixed via master branch: 8eafe17a6a276b1384d2e4b528fd0abdf190bd84
> Allow users to control the concurrency to submit jobs in clustering
> -------------------------------------------------------------------
>
> Key: HUDI-5303
> URL: https://issues.apache.org/jira/browse/HUDI-5303
> Project: Apache Hudi
> Issue Type: Improvement
> Components: clustering, spark
> Reporter: Hui An
> Assignee: Hui An
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.14.0
>
>
> If there are sufficient resources in the clustering job, some clustering
> groups sometimes could still waits to be triggered, we use forkJoinPool to
> submit these jobs, and it's also difficult for clients to adjust this
> configure(--conf
> spark.driver.extraJavaOptions=-Djava.util.concurrent.ForkJoinPool.common.parallelism),
> and it could also affect other tasks using the forkJoinPool, so instead, we
> introduce a new threadPool to control the submitting job parallelism for the
> clustering.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
