Re: Guide/design doc for streaming operator states

2015-07-30 Thread Aljoscha Krettek
Ah ok. I think that keyBy() can normally not be chained because we don't know how the fields in the emitted object change. On Thu, 30 Jul 2015 at 13:40 Gyula Fóra gyula.f...@gmail.com wrote: Thanks for the feedback :) My idea when I wrote that was that you can chain keyBy statements to

Re: Guide/design doc for streaming operator states

2015-07-30 Thread Gyula Fóra
Thanks for the feedback :) My idea when I wrote that was that you can chain keyBy statements to maintain order if your key does not change. Otherwise you are right, we need a sorting operator. Gyula Aljoscha Krettek aljos...@apache.org ezt írta (időpont: 2015. júl. 30., Cs, 13:18): Hi, sorry

Re: Guide/design doc for streaming operator states

2015-07-30 Thread Aljoscha Krettek
Hi, sorry for the long wait but I finally found the time to read it. It looks good but the later parts of course still need to be fleshed out. I have one comments/questions: In the description of partitioned state you have this sentence: Operations using partitioned state can also benefit from

Guide/design doc for streaming operator states

2015-07-23 Thread Gyula Fóra
Hey! I started putting together a guide/design document for the streaming operator state interfaces and implementations. The idea would be to create a doc that contains all the details about the implementations so anyone can use it as a reference later.