Re: Streams, Kafka windows

2020-05-28 Thread John Roesler
Setting a new record for procrastination, I've just created this ticket: https://issues.apache.org/jira/browse/KAFKA-10058 On Sat, Jan 18, 2020, at 22:03, John Roesler wrote: > Good idea! I’ll make a note to do it when I’m at a computer. > > On Sat, Jan 18, 2020, at 21:51, Guozhang Wang wrote:

Re: Streams, Kafka windows

2020-01-18 Thread John Roesler
Good idea! I’ll make a note to do it when I’m at a computer. On Sat, Jan 18, 2020, at 21:51, Guozhang Wang wrote: > Hey John, > > Since this is a common question and I've seen many users asking about > window semantics like this, could you file a JIRA ticket for creating a > wiki page like Join

Re: Streams, Kafka windows

2020-01-18 Thread Guozhang Wang
Hey John, Since this is a common question and I've seen many users asking about window semantics like this, could you file a JIRA ticket for creating a wiki page like Join Semantics ( https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Streams+Join+Semantics) to summarize the windowing

Re: Streams, Kafka windows

2020-01-18 Thread John Roesler
Glad it helped! -John On Sat, Jan 18, 2020, at 12:27, Viktor Markvardt wrote: > Hi John, > > Thank you for your assistance! > Your example very help me and I understood kafka-streams more clearly now. > Have a nice weekend :) > > Best regards, > Viktor Markvardt > > чт, 16 янв. 2020 г. в

Re: Streams, Kafka windows

2020-01-18 Thread Viktor Markvardt
Hi John, Thank you for your assistance! Your example very help me and I understood kafka-streams more clearly now. Have a nice weekend :) Best regards, Viktor Markvardt чт, 16 янв. 2020 г. в 19:29, John Roesler : > Hi Viktor, > > I’m starting to wonder what exactly “duplicate” means in this

Re: Streams, Kafka windows

2020-01-16 Thread John Roesler
Hi Viktor, I’m starting to wonder what exactly “duplicate” means in this context. Can you elaborate? In case it helps, with your window definition, if I send a record with timestamp 20, it would actually belong to three different windows: [0,30) [10,40) [20,50) Because of this, you would

Re: Streams, Kafka windows

2020-01-16 Thread Viktor Markvardt
Hi John, Thanks for answering my questions! I observe behavior which I can not understand. The code is working, but when delay between records larger then window duration I receive duplicated records. With the code below I received duplicated records in the output kstream. Count of duplicate

Re: Streams, Kafka windows

2020-01-15 Thread John Roesler
Hi Viktor, I’m not sure why you get two identical outputs in response to a single record. Regardless, since you say that you want to get a single, final result for the window and you expect multiple inputs to the windows, you need Suppression. My guess is that you just sent one record to try

Re: Streams, Kafka windows

2020-01-14 Thread Sachin Mittal
You can try to convert the final resultant stream to table. Check this page for more info: https://docs.confluent.io/current/streams/faq.html#how-can-i-convert-a-kstream-to-a-ktable-without-an-aggregation-step This way table would always contain the latest (single) record for a given key. Sachin

Streams, Kafka windows

2020-01-14 Thread Viktor Markvardt
Hi, My name is Viktor. I'm currently working with Kafka streams and have several questions about Kafka and I can not find answers in the official docs. 1. Why suppress functionality does not work with Hopping windows? How to make it work? Example of the code: KStream finalStream = source