Re: Parallel stream partitions

2018-07-19 Thread Fabian Hueske
Hi Nick, What Ken said is correct, but let me add two more things. 1) State Usually, you only need to partition (keyBy()) the data if you want to process tuples with the same same key together. Therefore, it is necessary to hold some tuples or intermediate results (like partial or running

Re: Parallel stream partitions

2018-07-17 Thread Ken Krugler
Hi Nick, > On Jul 17, 2018, at 9:09 AM, Nicholas Walton > wrote: > > Suppose I have a data stream of tuples > with the sequence of ticks being 1,2,3,…. for each separate k. > > I understand and keyBy(2) I think you want keyBy(1), since it’s 0-based. > will partition

Parallel stream partitions

2018-07-17 Thread Nicholas Walton
Suppose I have a data stream of tuples with the sequence of ticks being 1,2,3,…. for each separate k. I understand and keyBy(2) will partition the stream so each partition has the same key in each tuple. I now have a sequence of functions to apply to the streams say f(),g() and h() in that