Weird behavior of topic retention - some are cleaned up too often, some are not at all

2021-05-20 Thread Marina Popova
Hi, I have posted this question on SO: https://stackoverflow.com/questions/67625641/kafka-segments-are-deleted-too-often-or-not-at-all but wanted to re-post here as well in case someone spots the issue right away Thank you for your help! > We have two topics on our Kafka cluster that ex

[ANNOUNCE] Apache Kafka 2.7.1

2021-05-20 Thread Mickael Maison
The Apache Kafka community is pleased to announce the release for Apache Kafka 2.7.1 This is a bug fix release and it includes fixes and improvements from 45 JIRAs, including a few critical bugs. All of the changes in this release can be found in the release notes: https://www.apache.org/dist/ka

Re: Application state persistence in Kafka

2021-05-20 Thread Urko Lekuona
Hello Mangat, I am using Kstreams, does this make a difference? Anyways, I've read about stateful stream processing here: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Streams+Internal+Data+Management I think I can manage to solve my issue using changelog topics for my state store. I'll