[ https://issues.apache.org/jira/browse/SPARK-10295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen reassigned SPARK-10295: --------------------------------- Assignee: Sean Owen > Dynamic allocation in Mesos does not release when RDDs are cached > ----------------------------------------------------------------- > > Key: SPARK-10295 > URL: https://issues.apache.org/jira/browse/SPARK-10295 > Project: Spark > Issue Type: Improvement > Components: Documentation, Spark Core > Affects Versions: 1.5.0 > Environment: Spark 1.5.0 RC1 > Centos 6 > java 7 oracle > Reporter: Hans van den Bogert > Assignee: Sean Owen > Priority: Minor > Fix For: 1.6.0, 1.5.1 > > > When running spark in coarse grained mode with shuffle service and dynamic > allocation, the driver does not release executors if a dataset is cached. > The console output OTOH shows: > > 15/08/26 17:29:58 WARN SparkContext: Dynamic allocation currently does not > > support cached RDDs. Cached data for RDD 9 will be lost when executors are > > removed. > However after the default of 1m, executors are not released. When I perform > the same initial setup, loading data, etc, but without caching, the executors > are released. > Is this intended behaviour? > If this is intended behaviour, the console warning is misleading. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org