[
https://issues.apache.org/jira/browse/SPARK-4994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lianhui Wang updated SPARK-4994:
--------------------------------
Description: when the application is completed, yarn's nodemanager can
remove application's local-dirs.but all executors' metadata of completed
application havenot be removed. now it let yarn ShuffleService to have much
more memory to store Executors' ShuffleInfo. so these metadata need to be
removed. (was: when the application is completed, yarn's nodemanager can
remove application's local-dirs.but all executors' metadata of completed
application havenot be removed. so these metadata need to be removed.)
> Cleanup removed executors' ShuffleInfo in yarn shuffle service
> --------------------------------------------------------------
>
> Key: SPARK-4994
> URL: https://issues.apache.org/jira/browse/SPARK-4994
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.2.0
> Reporter: Lianhui Wang
>
> when the application is completed, yarn's nodemanager can remove
> application's local-dirs.but all executors' metadata of completed application
> havenot be removed. now it let yarn ShuffleService to have much more memory
> to store Executors' ShuffleInfo. so these metadata need to be removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
