Fix slow processing rate in Kafka streams

Hi All, My streams application is not processing more than 350 records/sec on a high load of 3milliom records produced every 2-3 minutes. My scenarios are as below - I am on Kafka and streams version of 3.5.1 . My key-value pair is in protobuf format . I do a groupbykey followed by TimeWindow of

Re: production ready for zookeeper to kraft migration

Matthieu, There are a few things to look out for during the migration. 1) Migration not starting. This is pretty much always going to be due to mis-configuration. Controller KRaftMigrationDriver logs should reveal what its waiting for 2) Migration of metadata fails. If some poison record in ZK

Re: outerJoin confusion

Yeah, that is some quirk of KS runtime... There is some internal config (for perf reasons) that delays emitting results... An alternative to advancing wall-clock time would be to set this internal config to zero, to disable the delay. Maybe we should disable this config when topology test

Re: production ready for zookeeper to kraft migration

Hey Luke, Thank you for the update. Out of curiosity if the migration is not working what are the symptoms ? is it just that the controller won't show that the migration is complete ? or could the controller claim (wrongfully) that the migration is complete when it's not ? Best. Matthieu On

Re: [EXTERNAL] Re: Kafka Streams 3.5.1 based app seems to get stalled

Glad to hear that the config change helped. For continuous rebalancing, it might be expected for KS, as KS uses the protocol in advanced ways. If you see log lines saying "follow up rebalance requested" than there is nothing to worry about, and the group is stable. If you see "no follow up

Re: [VOTE] 3.6.2 RC2

Thanks all for voting. I'm now closing the vote. The vote passes with - 3 +1 bindings votes from Divij Vaidya, Justine, and Manikumar - 3 +1 non-binding votes from Andrew, Jakub, and Lianet - 0 -1 votes I'll go ahead and finish the release process. Thanks, On Thu, Apr 4, 2024 at 8:53 PM

Re: [VOTE] 3.6.2 RC2

+1 (binding) from my side also. - verified the signatures, artifacts - ran the tests on the source archive - verified the quickstarts Thanks On Thu, Apr 4, 2024 at 7:46 PM Justine Olshan wrote: > Thanks for clarifying Manikumar! > > +1 (binding) from me > > Justine > > On Thu, Apr 4, 2024 at

Re: Kafka capabilities

Dear Kafka experts , Could anyone having this data share the details please On Wed, Apr 3, 2024 at 3:42 PM Kafka Life wrote: > Hi Kafka users > > Does any one have a document or ppt that showcases the capabilities of > Kafka along with any cost management capability? > i have a customer who is