Re: Equal number of RDD Blocks

2015-04-20 Thread Laeeq Ahmed
[mailto:laeeqsp...@yahoo.com] Sent: Monday, April 20, 2015 3:38 PM To: Evo Eftimov; user@spark.apache.org Subject: Re: Equal number of RDD Blocks  Hi,  I have two different topics and two Kafka receivers, one for each topic.  Regards,Laeeq      On Monday, April 20, 2015 4:28 PM, Evo Eftimov

RE: Equal number of RDD Blocks

2015-04-20 Thread Evo Eftimov
What is meant by “streams” here: 1. Two different DSTream Receivers producing two different DSTreams consuming from two different kafka topics, each with different message rate 2. One kafka topic (hence only one message rate to consider) but with two different DStream receivers

Re: Equal number of RDD Blocks

2015-04-20 Thread Laeeq Ahmed
And what is the message rate of each topic mate – that was the other part of the required clarifications  From: Laeeq Ahmed [mailto:laeeqsp...@yahoo.com] Sent: Monday, April 20, 2015 3:38 PM To: Evo Eftimov; user@spark.apache.org Subject: Re: Equal number of RDD Blocks  Hi,  I have two different

Re: Equal number of RDD Blocks

2015-04-20 Thread Laeeq Ahmed
Hi, I have two different topics and two Kafka receivers, one for each topic. Regards,Laeeq On Monday, April 20, 2015 4:28 PM, Evo Eftimov evo.efti...@isecc.com wrote: #yiv4992037734 #yiv4992037734 -- _filtered #yiv4992037734 {panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv4992037734

RE: Equal number of RDD Blocks

2015-04-20 Thread Evo Eftimov
And what is the message rate of each topic mate – that was the other part of the required clarifications From: Laeeq Ahmed [mailto:laeeqsp...@yahoo.com] Sent: Monday, April 20, 2015 3:38 PM To: Evo Eftimov; user@spark.apache.org Subject: Re: Equal number of RDD Blocks Hi, I have two

RE: Equal number of RDD Blocks

2015-04-20 Thread Evo Eftimov
number of RDD Blocks I also see that its creating both receivers on the same executor and that might be the cause of having more RDDs on executor than the other. Can I suggest spark to create each receiver on a each executor Regards, Laeeq On Monday, April 20, 2015 4:51 PM, Evo