Re: Kafka Streams leave group behaviour

2021-08-12 Thread Lerh Chuan Low
I think you may have stumbled upon this: https://issues.apache.org/jira/browse/KAFKA-4881. 1 thing that you could try is using static membership - we have yet to try that though so can't comment yet on how that might work out. On Thu, Aug 12, 2021 at 11:29 PM c...@uweeisele.eu wrote: > Hello all

Mass transaction abort on topic retention configuration change

2021-08-04 Thread Lerh Chuan Low
Hey folks, I'm wondering if anyone has any insights into what I'm observing in Kafka. We have a 6 node cluster with our topics consuming a lot of disk space. All our apps use transactions. We recently ran kafka-topics.sh --zookeeper --alter --topic --config to alter retention for a single topic, fr

Re: Is exactly-once possible with kafka consumer-producer ?

2021-07-18 Thread Lerh Chuan Low
Yep that's the one. You can see here for a great example on the typical flow https://www.confluent.io/blog/transactions-apache-kafka/. On Sat, Jul 17, 2021 at 3:24 AM Pushkar Deole wrote: > Hi Lerh Chuan Low, > > MAny thanks for your response. I get it now, that it provide

Re: Is exactly-once possible with kafka consumer-producer ?

2021-07-16 Thread Lerh Chuan Low
Pushkar, My understanding is you can easily turn it on by using Kafka streams as Chris mentioned. Otherwise you'd have to do it yourself - I don't think you can get exactly once processing, but what you can do (which is also what Kafka streams does) is exactly once schematics (You won't be able to