Re: AW: Table updates are not consistent when doing a join with a Stream

2023-09-04 Thread Matthias J. Sax
Your update to the KTable is async when you send data back to the KTable input topic. So your program is subject to race-conditions. So switching to the PAPI was the right move: it make the update to the state store sync and thus fixes the issue. -Matthias On 9/4/23 5:53 AM, Mauricio Lopez

Re: kafka streams consumer group reporting lag even on source topics removed from topology

2023-09-04 Thread Matthias J. Sax
As long as the consumer group is active, nothing will be deleted. That is the reason why you get those incorrect alerts -- Kafka cannot know that you stopped consuming from those topics. (That is what I tried to explain -- seems I did a bad job...) Changing the group.id is tricky because

RE: AW: Table updates are not consistent when doing a join with a Stream

2023-09-04 Thread Mauricio Lopez
Hello, They were getting processed by the same consumer as we only had a single machine running this. What we ended up doing is basically drawing the same topology but interacting directly with the stateStore using the Processor API instead of DSL. Seems that fixed everything up (and made it