Re: Does long-lived SparkContext hold on to executor resources?

2015-05-12 Thread Josh Rosen
) -Original Message- *From: *Silvio Fiorito [silvio.fior...@granturing.com] *Sent: *Monday, May 11, 2015 01:03 PM Eastern Standard Time *To: *stanley; user@spark.apache.org *Subject: *Re: Does long-lived SparkContext hold on to executor resources? You want to look at dynamic resource

RE: Does long-lived SparkContext hold on to executor resources?

2015-05-11 Thread Ganelin, Ilya
, 2015 01:03 PM Eastern Standard Time To: stanley; user@spark.apache.org Subject: Re: Does long-lived SparkContext hold on to executor resources? You want to look at dynamic resource allocation, here: http://spark.apache.org/docs/latest/job-scheduling.html#dynamic-resource-allocation On 5/11

Re: Does long-lived SparkContext hold on to executor resources?

2015-05-11 Thread Silvio Fiorito
You want to look at dynamic resource allocation, here: http://spark.apache.org/docs/latest/job-scheduling.html#dynamic-resource-allocation On 5/11/15, 11:23 AM, stanley wangshua...@yahoo.com wrote: I am building an analytics app with Spark. I plan to use long-lived SparkContexts to minimize