[
https://issues.apache.org/jira/browse/HUDI-9067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-9067:
--------------------------------------
Fix Version/s: 1.0.2
> Clean action sometimes falls back to spark.default.parallelism set in the env
> -----------------------------------------------------------------------------
>
> Key: HUDI-9067
> URL: https://issues.apache.org/jira/browse/HUDI-9067
> Project: Apache Hudi
> Issue Type: Improvement
> Components: cleaning
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Major
> Fix For: 1.0.2
>
>
> Rarely, we notice there are 1024 (spark.default.parallelism) tasks spinning
> up for clean actions.
>
> for eg, if we try to ingest very small no of records say just 1, clean action
> executor spins up 1024 tasks even though the clean parallelism is set to
> small value.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
