Re: DStream Union vs. StreamingContext Union

2015-05-14 Thread Vadim Bichutskiy
>> >>>> *From:* Vadim Bichutskiy [mailto:vadim.bichuts...@gmail.com] >>>> *Sent:* Tuesday, May 12, 2015 5:53 PM >>>> *To:* Evo Eftimov >>>> *Cc:* Saisai Shao; user@spark.apache.org >>>> >>>> *Subject:* Re: DStream Union vs. S

Re: DStream Union vs. StreamingContext Union

2015-05-12 Thread Tathagata Das
, Evo Eftimov >> wrote: >> >>> I can confirm it does work in Java >>> >>> >>> >>> *From:* Vadim Bichutskiy [mailto:vadim.bichuts...@gmail.com] >>> *Sent:* Tuesday, May 12, 2015 5:53 PM >>> *To:* Evo Eftimov >>> *Cc:* Saisai Shao;

Re: DStream Union vs. StreamingContext Union

2015-05-12 Thread Vadim Bichutskiy
I can confirm it does work in Java >> >> >> >> *From:* Vadim Bichutskiy [mailto:vadim.bichuts...@gmail.com] >> *Sent:* Tuesday, May 12, 2015 5:53 PM >> *To:* Evo Eftimov >> *Cc:* Saisai Shao; user@spark.apache.org >> >> *Subject:* Re: DStream

Re: DStream Union vs. StreamingContext Union

2015-05-12 Thread Tathagata Das
:53 PM > *To:* Evo Eftimov > *Cc:* Saisai Shao; user@spark.apache.org > > *Subject:* Re: DStream Union vs. StreamingContext Union > > > > Thanks Evo. I tried chaining Dstream unions like what you have and it > didn't work for me. But passing > > multiple arguments to

RE: DStream Union vs. StreamingContext Union

2015-05-12 Thread Evo Eftimov
I can confirm it does work in Java From: Vadim Bichutskiy [mailto:vadim.bichuts...@gmail.com] Sent: Tuesday, May 12, 2015 5:53 PM To: Evo Eftimov Cc: Saisai Shao; user@spark.apache.org Subject: Re: DStream Union vs. StreamingContext Union Thanks Evo. I tried chaining Dstream unions like

Re: DStream Union vs. StreamingContext Union

2015-05-12 Thread Vadim Bichutskiy
ichuts...@gmail.com] > *Sent:* Tuesday, May 12, 2015 5:37 PM > *To:* Saisai Shao > *Cc:* user@spark.apache.org > *Subject:* Re: DStream Union vs. StreamingContext Union > > > > Thanks Saisai. That makes sense. Just seems redundant to have both. > > ᐧ > > > &g

RE: DStream Union vs. StreamingContext Union

2015-05-12 Thread Evo Eftimov
...@gmail.com] Sent: Tuesday, May 12, 2015 5:37 PM To: Saisai Shao Cc: user@spark.apache.org Subject: Re: DStream Union vs. StreamingContext Union Thanks Saisai. That makes sense. Just seems redundant to have both. <https://mailfoogae.appspot.com/t?sender=admFkaW0uYmljaHV0c2tpeUBnbWFpbC5j

Re: DStream Union vs. StreamingContext Union

2015-05-12 Thread Vadim Bichutskiy
Thanks Saisai. That makes sense. Just seems redundant to have both. ᐧ On Mon, May 11, 2015 at 10:36 PM, Saisai Shao wrote: > DStream.union can only union two DStream, one is itself. While > StreamingContext.union can union an array of DStreams, internally > DStream.union is a special case of Str

Re: DStream Union vs. StreamingContext Union

2015-05-11 Thread Saisai Shao
DStream.union can only union two DStream, one is itself. While StreamingContext.union can union an array of DStreams, internally DStream.union is a special case of StreamingContext.union: def union(that: DStream[T]): DStream[T] = new UnionDStream[T](Array(this, that)) So there's no difference, if

DStream Union vs. StreamingContext Union

2015-05-11 Thread Vadim Bichutskiy
Can someone explain to me the difference between DStream union and StreamingContext union? When do you use one vs the other? Thanks, Vadim ᐧ