Re: Maintaining Stream Partitioning after Mapping?

2017-04-17 Thread Ryan Conway
Thank you, Chesnay. My hope is to keep things computationally inexpensive, and if I understand you correctly, that is satisfied even with this rekeying. Ryan On Sat, Apr 15, 2017 at 4:22 AM, Chesnay Schepler wrote: > Hello, > > I think if you have multiple keyBy() transformations with identical

Re: Maintaining Stream Partitioning after Mapping?

2017-04-15 Thread Chesnay Schepler
Hello, I think if you have multiple keyBy() transformations with identical parallelism the partitioning should be "preserved". The second keyBy() will still go through the partitioning process, but since both the key and parallelism are identical the resulting partition should be identical as

Maintaining Stream Partitioning after Mapping?

2017-04-12 Thread Ryan Conway
Greetings, Is there a means of maintaining a stream's partitioning after running it through an operation such as map or filter? I have a pipeline stage S that operates on a stream partitioned by an ID field. S flat maps objects of type A to type B, which both have an "ID" field, and where each in