Re: Question about MEOMORY_AND_DISK persistence

Hi Vishnu, A partition will either be in memory or in disk. -Ashwin On Feb 28, 2016 15:09, "Vishnu Viswanath" wrote: > Hi All, > > I have a question regarding Persistence (MEMORY_AND_DISK) > > Suppose I am trying to persist an RDD which has 2 partitions and only 1

Re: What happens when you create more DStreams then nodes in the cluster?

. Will ~10 streams get assigned to ~10 executors / nodes then the other ~20 streams will be queued for resources or will the other streams just fail and never run? -- Thanks Regards, Ashwin Giridharan

Re: Has anybody ever tried running Spark Streaming on 500 text streams?

is the best way to parallelize this? Any other ideas on design? -- Thanks Regards, Ashwin Giridharan

Re: How to control Spark Executors from getting Lost when using YARN client mode?

...@spark.apache.org -- Thanks Regards, Ashwin Giridharan

Re: Has anybody ever tried running Spark Streaming on 500 text streams?

? What is the best way to parallelize this? Any other ideas on design? -- Thanks Regards, Ashwin Giridharan