concat two Dstreams

2014-11-11 Thread Josh J 
Hi,

Is it possible to concatenate or append two Dstreams together? I have an
incoming stream that I wish to combine with data that's generated by a
utility. I then need to process the combined Dstream.

Thanks,
Josh

Re: concat two Dstreams

2014-11-11 Thread Josh J 
I think it's just called union

On Tue, Nov 11, 2014 at 2:41 PM, Josh J joshjd...@gmail.com wrote:

 Hi,

 Is it possible to concatenate or append two Dstreams together? I have an
 incoming stream that I wish to combine with data that's generated by a
 utility. I then need to process the combined Dstream.

 Thanks,
 Josh

Re: concat two Dstreams

2014-11-11 Thread Sean Owen 
Concatenate? no. It doesn't make sense in this context to think about one
potentially infinite stream coming after another one. Do you just want the
union of batches from two streams? yes, just union(). You can union() with
non-streaming RDDs too.

On Tue, Nov 11, 2014 at 10:41 PM, Josh J joshjd...@gmail.com wrote:

 Hi,

 Is it possible to concatenate or append two Dstreams together? I have an
 incoming stream that I wish to combine with data that's generated by a
 utility. I then need to process the combined Dstream.

 Thanks,
 Josh