You want to look at dynamic resource allocation, here: http://spark.apache.org/docs/latest/job-scheduling.html#dynamic-resource-allocation
On 5/11/15, 11:23 AM, "stanley" <wangshua...@yahoo.com> wrote: >I am building an analytics app with Spark. I plan to use long-lived >SparkContexts to minimize the overhead for creating Spark contexts, which in >turn reduces the analytics query response time. > >The number of queries that are run in the system is relatively small each >day. Would long lived contexts hold on to the executor resources when there >is no queries running? Is there a way to free executor resources in this >type of use cases? > > > >-- >View this message in context: >http://apache-spark-user-list.1001560.n3.nabble.com/Does-long-lived-SparkContext-hold-on-to-executor-resources-tp22848.html >Sent from the Apache Spark User List mailing list archive at Nabble.com. > >--------------------------------------------------------------------- >To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >For additional commands, e-mail: user-h...@spark.apache.org >
