[
https://issues.apache.org/jira/browse/SPARK-44345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-44345:
----------------------------------
Summary: Lower unknown shuffle map output log level to WARN if shuffle
migration is enabled (was: Only log unknown shuffle map output as error when
shuffle migration disabled)
> Lower unknown shuffle map output log level to WARN if shuffle migration is
> enabled
> ----------------------------------------------------------------------------------
>
> Key: SPARK-44345
> URL: https://issues.apache.org/jira/browse/SPARK-44345
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.4.1
> Reporter: Zhongwei Zhu
> Assignee: Zhongwei Zhu
> Priority: Minor
> Labels: pull-request-available
> Fix For: 4.0.0
>
>
> When decommission and shuffle migration is enabled, there're lots of error
> message like `Asked to update map output for unknown shuffle` .
> As shuffle clean and unregister is done by ContextCleaner in an async way,
> when target block manager received the shuffle block from decommissioned
> block manager, then update shuffle location to map output tracker, but at
> that time, shuffle might have been unregistered. This should not consider as
> error.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
