[
https://issues.apache.org/jira/browse/SPARK-12883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107085#comment-15107085
]
Saisai Shao commented on SPARK-12883:
-------------------------------------
I get your point now. But I think these two descriptions are still both valid,
the first paragraph describes the result of data cached executor removing, and
the second paragraph says how to workaround this problem. Maybe just different
understanding from different people.
> 1.6 Dynamic allocation document for removing executors with cached data
> differs in different sections
> -----------------------------------------------------------------------------------------------------
>
> Key: SPARK-12883
> URL: https://issues.apache.org/jira/browse/SPARK-12883
> Project: Spark
> Issue Type: Documentation
> Components: Documentation
> Affects Versions: 1.6.0
> Reporter: Manoj Samel
> Priority: Trivial
>
> Spark 1.6 dynamic allocation documentation still refers to 1.2.
> See text "There is currently not yet a solution for this in Spark 1.2. In
> future releases, the cached data may be preserved through an off-heap storage
> similar in spirit to how shuffle files are preserved through the external
> shuffle service"
> It appears 1.6 has parameter to address cache executor
> spark.dynamicAllocation.cachedExecutorIdleTimeout with default value as
> infinity.
> Pl update 1.6 documentation to refer to latest release and features
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
