Re: unoin streams not working for streams > 3

[Cody Koeninger]

I assume you're using the receiver based stream (createStream) rather than
createDirectStream?

Receivers each get scheduled as if they occupy a core, so you need at least
one more core than number of receivers if you want to get any work done.

Try using the direct stream if you can't combine receivers.

On Mon, Sep 14, 2015 at 11:10 PM, Василец Дмитрий 
wrote:

> I use local[*]. And i have 4 cores on laptop.
> On 14 Sep 2015 23:19, "Gerard Maas"  wrote:
>
>> How many cores are you assigning to your spark streaming job?
>>
>> On Mon, Sep 14, 2015 at 10:33 PM, Василец Дмитрий <
>> pronix.serv...@gmail.com> wrote:
>>
>>> hello
>>> I have 4 streams from kafka and streaming not working.
>>> without any errors or logs
>>> but with 3 streams everything perfect.
>>> make sense only amount of streams , different triple combinations always
>>> working.
>>> any ideas how to debug or fix it ?
>>>
>>>
>>>
>>

Re: unoin streams not working for streams > 3

[Василец Дмитрий]

thanks.I will try.

On Tue, Sep 15, 2015 at 4:19 PM, Cody Koeninger  wrote:

> I assume you're using the receiver based stream (createStream) rather than
> createDirectStream?
>
> Receivers each get scheduled as if they occupy a core, so you need at
> least one more core than number of receivers if you want to get any work
> done.
>
> Try using the direct stream if you can't combine receivers.
>
> On Mon, Sep 14, 2015 at 11:10 PM, Василец Дмитрий <
> pronix.serv...@gmail.com> wrote:
>
>> I use local[*]. And i have 4 cores on laptop.
>> On 14 Sep 2015 23:19, "Gerard Maas"  wrote:
>>
>>> How many cores are you assigning to your spark streaming job?
>>>
>>> On Mon, Sep 14, 2015 at 10:33 PM, Василец Дмитрий <
>>> pronix.serv...@gmail.com> wrote:
>>>
 hello
 I have 4 streams from kafka and streaming not working.
 without any errors or logs
 but with 3 streams everything perfect.
 make sense only amount of streams , different triple combinations
 always working.
 any ideas how to debug or fix it ?



>>>
>

Re: unoin streams not working for streams > 3

[Gerard Maas]

How many cores are you assigning to your spark streaming job?

On Mon, Sep 14, 2015 at 10:33 PM, Василец Дмитрий 
wrote:

> hello
> I have 4 streams from kafka and streaming not working.
> without any errors or logs
> but with 3 streams everything perfect.
> make sense only amount of streams , different triple combinations always
> working.
> any ideas how to debug or fix it ?
>
>
>

Re: unoin streams not working for streams > 3

[Василец Дмитрий]

I use local[*]. And i have 4 cores on laptop.
On 14 Sep 2015 23:19, "Gerard Maas"  wrote:

> How many cores are you assigning to your spark streaming job?
>
> On Mon, Sep 14, 2015 at 10:33 PM, Василец Дмитрий <
> pronix.serv...@gmail.com> wrote:
>
>> hello
>> I have 4 streams from kafka and streaming not working.
>> without any errors or logs
>> but with 3 streams everything perfect.
>> make sense only amount of streams , different triple combinations always
>> working.
>> any ideas how to debug or fix it ?
>>
>>
>>
>