[
https://issues.apache.org/jira/browse/SPARK-7065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516162#comment-14516162
]
Susu Xie commented on SPARK-7065:
---------------------------------
We do this because we found is some case (we've not identified the root cause
for now) the cache locations are not correctly updated during RDD coalesce. So
we clear the cache after every stage (rather than at the end of reduce) as a
workaround, and find almost no performance penalty.
> Clear the cached locations mapping after every stage to avoid inconsistent
> status
> ---------------------------------------------------------------------------------
>
> Key: SPARK-7065
> URL: https://issues.apache.org/jira/browse/SPARK-7065
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Julien Pierre
> Priority: Minor
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
