Re: Dynamic Resource Allocation with Spark Streaming (Standalone Cluster, Spark 1.5.1)

2015-10-27 Thread Deenar Toraskar
off <matthias.nieh...@codecentric.de> >> Sent: ‎10/‎26/‎2015 4:00 PM >> To: user@spark.apache.org >> Subject: Dynamic Resource Allocation with Spark Streaming (Standalone >> Cluster, Spark 1.5.1) >> >> Hello everybody, >> >> I have a few (~15) Spar

RE: Dynamic Resource Allocation with Spark Streaming (Standalone Cluster, Spark 1.5.1)

2015-10-26 Thread Silvio Fiorito
:00 PM To: user@spark.apache.org<mailto:user@spark.apache.org> Subject: Dynamic Resource Allocation with Spark Streaming (Standalone Cluster, Spark 1.5.1) Hello everybody, I have a few (~15) Spark Streaming jobs which have load peaks as well as long times with a low load. So I thought the

Dynamic Resource Allocation with Spark Streaming (Standalone Cluster, Spark 1.5.1)

2015-10-26 Thread Matthias Niehoff
Hello everybody, I have a few (~15) Spark Streaming jobs which have load peaks as well as long times with a low load. So I thought the new Dynamic Resource Allocation for Standalone Clusters might be helpful (SPARK-4751). I have a test "cluster" with 1 worker consisting of 4 executors with 2

Re: Dynamic Resource Allocation with Spark Streaming (Standalone Cluster, Spark 1.5.1)

2015-10-26 Thread Ted Yu
g > Subject: Dynamic Resource Allocation with Spark Streaming (Standalone > Cluster, Spark 1.5.1) > > Hello everybody, > > I have a few (~15) Spark Streaming jobs which have load peaks as well as > long times with a low load. So I thought the new Dynamic Resource > Allocati