I ran some tests, and also looked through the code. log.roll.ms isn't sufficient.
Segments appear to get rolled only if log.roll.ms or segment.ms has passed log.segment.bytes has passed But those rules are only evaluated when a new record is appended to the log. So it appears that for in-active topics, segments will never roll. See https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/log/Log.scala#L383 and https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/log/Log.scala#L617-L634 Tom, does that sound right? -James > On May 20, 2016, at 6:05 AM, Tom Crayford <tcrayf...@heroku.com> wrote: > > Hi there, > > The missing piece is the config `log.roll.hours` or it's alternative ` > log.roll.ms`. Log segments are by default rolled once a week, regardless of > activity, but you can tune that down as you like. > > Thanks > > Tom Crayford > Heroku Kafka > > On Fri, May 20, 2016 at 12:49 AM, James Cheng <wushuja...@gmail.com> wrote: > >> Time-based log retention only happens on old log segments. And log >> compaction only happens on old segments as well. >> >> Currently, I believe segments only roll whenever a new record is written >> to the log. That is, during the write of the new record is when the current >> segment is evaluated to see if it should be rolled. Is that true? >> >> That means that if there is *no* traffic on a topic, that the messages on >> disk may persist past the log retention time, or past the log compaction >> time. Is that the case? If so, is there any way to trigger rolling of a >> segment without active traffic on the topic? >> >> Thanks! >> -James >> >>
