Matthias, you read my mind—having examined Kafka Streams intermediate topic configs and then Googled my way to KIP-71 <https://cwiki.apache.org/confluence/display/KAFKA/KIP-71%3A+Enable+log+compaction+and+deletion+to+co-exist>, I was confused about this dual policy. Thank you.
Still wondering about my second question though: does deletion/compaction affect the currently opened log segment? Seems like it cannot. On Mon, Dec 4, 2017 at 2:54 PM, Matthias J. Sax <matth...@confluent.io> wrote: > Topic can be configured in "dual" mode too via > > >> cleanup.policy="delete,compact" > > For this case, `retention.ms` is basically a TTL for a key that is not > updated for this amount of time. > > > -Matthias > > > > On 12/3/17 11:54 AM, Jan Filipiak wrote: > > Hi > > > > the only retention time that applies for compacted topics is the > > delete.retention.ms > > The duration that tombstones for deletes will be kept in the topic > > during compaction. > > > > A very detail explaination on what is going on can be found here: > > > > https://kafka.apache.org/documentation/#compaction > > > > Hope this helps > > > > Best Jan > > > > > > On 03.12.2017 20:27, Dmitry Minkovsky wrote: > >> This is a pretty stupid question. Mostly likely I should verify these by > >> observation, but really I want to verify that my understanding of the > >> documentation is correct: > >> > >> Suppose I have topic configurations like: > >> > >> retention.ms=$time > >> cleanup.policy=compact > >> > >> > >> My questions are: > >> > >> 1. After $time, any offsets older than $time will be eligible for > >> compaction? > >> 2. Regardless of $time, any offsets in the current segment will > >> not be > >> compacted? > >> > >> > >> Thank you, > >> Dmitry > >> > > > >
