Re: [Kafka Streams], Processor API for KTable and KGroupedStream

You cannot add a `Processor`. You can only use `aggregate() / reduce() / count()` (which of course will add a pre-defined processor). `groupByKey()` is really just a "meta operation" that checks if the key was changes upstream, and to insert a repartition/shuffle step if necessary. Thus, if

Re: [Kafka Streams], Processor API for KTable and KGroupedStream

Thanks a lot for explanation but could you provide a bit more details about KGroupedStream? It is just interface and not extends KStream so how I can add processor in the case below? /   KStream someStream = / /  someStream / /     .groupByKey() / */how to add processor for resulted grouped

Re: [Kafka Streams], Processor API for KTable and KGroupedStream

`KGroupedStream` is just an "intermediate representation" to get a better flow in the DSL. It's not a "top level" abstraction like KStream/KTable. For `KTable` there is `transformValue()` -- there is no `transform()` because keying must be preserved -- if you want to change the keying you

[Kafka Streams], Processor API for KTable and KGroupedStream

Hello Is there any way in Kafka Streams API to define processors for KTable and KGroupedStream like KStream#transform? How to provide a custom processor for KTable or KGroupedStream which could for instance provide way to not downstream selected events? -- Igor Maznitsa email:

Re: Kafka Streams Processor API state stores not restored via changelog topics

Desai​ | Senior Software Developer | *ude...@itrsgroup.com* > > *www.itrsgroup.com* <https://www.itrsgroup.com/> > <https://www.itrsgroup.com/> > > *From: *Guozhang Wang > *Date: *Tuesday, March 30, 2021 at 2:10 PM > *To: *Users > *Cc: *Bart Lilje > *Subject: *Re: Ka

Re: Kafka Streams Processor API state stores not restored via changelog topics

From: *Guozhang Wang > *Date: *Tuesday, March 30, 2021 at 1:00 PM > *To: *Users > *Cc: *Bart Lilje > *Subject: *Re: Kafka Streams Processor API state stores not restored via > changelog topics > > Hello Upesh, > > These are super helpful logs, and I think I'm very close

Re: Kafka Streams Processor API state stores not restored via changelog topics

; > > Best, > > Upesh > > > Upesh Desai​ | Senior Software Developer | *ude...@itrsgroup.com* > > *www.itrsgroup.com* <https://www.itrsgroup.com/> > <https://www.itrsgroup.com/> > > *From: *Guozhang Wang > *Date: *Tuesday, March 30, 2021 at 1:0

Re: Kafka Streams Processor API state stores not restored via changelog topics

Desai | Senior Software Developer | ude...@itrsgroup.com www.itrsgroup.com From: Guozhang Wang Date: Tuesday, March 30, 2021 at 1:00 PM To: Users Cc: Bart Lilje Subject: Re: Kafka Streams Processor API state stores not restored via changelog topics Hello Upesh, These are super helpful logs

Re: Kafka Streams Processor API state stores not restored via changelog topics

> insight into where this issue could originate from? > > > > Thanks, > Upesh > > > Upesh Desai​ | Senior Software Developer | *ude...@itrsgroup.com* > > *www.itrsgroup.com* <https://www.itrsgroup.com/> > <https://www.itrsgroup.com/> > > *From

Re: Kafka Streams Processor API state stores not restored via changelog topics

Date: Thursday, March 25, 2021 at 6:46 PM To: users@kafka.apache.org Cc: Bart Lilje Subject: Re: Kafka Streams Processor API state stores not restored via changelog topics We have not tried running 2.6 brokers and 2.7 client, I will try and get back to you. We are not enabling EOS on the streams

Re: Kafka Streams Processor API state stores not restored via changelog topics

instantly. Best, Upesh Upesh Desai | Senior Software Developer | ude...@itrsgroup.com www.itrsgroup.com From: Guozhang Wang Date: Thursday, March 25, 2021 at 6:31 PM To: Users Subject: Re: Kafka Streams Processor API state stores not restored via changelog topics BTW, yes that indicates

Re: Kafka Streams Processor API state stores not restored via changelog topics

when we run everything on Kafka version 2.6. >> >> >> >> Thanks, >> >> Upesh >> >> >> Upesh Desai​ | Senior Software Developer | *ude...@itrsgroup.com* >> >> *www.itrsgroup.com* <https://www.itrsgroup.com/> >> <https://w

Re: Kafka Streams Processor API state stores not restored via changelog topics

rsgroup.com/> > <https://www.itrsgroup.com/> > > *From: *Guozhang Wang > *Date: *Thursday, March 25, 2021 at 4:01 PM > *To: *Users > *Cc: *Bart Lilje > *Subject: *Re: Kafka Streams Processor API state stores not restored via > changelog topics > > Hell

Re: Kafka Streams Processor API state stores not restored via changelog topics

Subject: Re: Kafka Streams Processor API state stores not restored via changelog topics Hello Upesh, Could you confirm a few more things for me: 1. After you stopped the application, and wiped out the state dir; check if the corresponding changelog topic has one record indeed at offset 0

Re: Kafka Streams Processor API state stores not restored via changelog topics

gt; Upesh Desai​ | Senior Software Developer | *ude...@itrsgroup.com* > > *www.itrsgroup.com* <https://www.itrsgroup.com/> > <https://www.itrsgroup.com/> > > *From: *Guozhang Wang > *Date: *Wednesday, March 24, 2021 at 1:37 PM > *To: *Users > *Cc: *Bart Lilje > *S

Re: Kafka Streams Processor API state stores not restored via changelog topics

...@itrsgroup.com www.itrsgroup.com From: Guozhang Wang Date: Wednesday, March 24, 2021 at 1:37 PM To: Users Cc: Bart Lilje Subject: Re: Kafka Streams Processor API state stores not restored via changelog topics Hello Upesh, Thanks for the detailed report. I looked through the code and tried

Re: Kafka Streams Processor API state stores not restored via changelog topics

all, > > > > Our team think we discovered a bug over the weekend withing the Kafka > Streams / Processor API. We are running 2.7.0. > > > > When configuring a state store backed by a changelog topic with the > cleanup policy configuration set to “compact,delete”: > >

Kafka Streams Processor API state stores not restored via changelog topics

Hi all, Our team think we discovered a bug over the weekend withing the Kafka Streams / Processor API. We are running 2.7.0. When configuring a state store backed by a changelog topic with the cleanup policy configuration set to “compact,delete”: final StoreBuilder> store = Sto

Re: Kafka Streams & Processor API

Hey Navneeth, thank you for your interest in trying out Kafka Streams! Normally we will redirect new folks to the stream FAQ first in case you haven't checked it out. For details to your question: 1. Joining 2 topics using processor API (we

Kafka Streams & Processor API

Hi All, I'm new to kafka streams and I'm trying to model my use case that is currently written on flink to kafka streams. I have couple of questions. - How can I join two kafka topics using the processor API? I have data stream and change stream which needs to be joined based on a key. - I read