I would actually suggest only using the ms versions of the retention config. Be sure to check/set all the configs below and look at the documented explanations here http://kafka.apache.org/documentation.html#brokerconfigs.
I am guessing the default log.retention.check.interval.ms of 5 minutes is too long for your case or you may have changed from the default log.cleanup.policy and it is no longer set to delete. log.retention.ms log.retention.check.interval.ms log.cleanup.policy On Fri, Jul 24, 2015 at 9:14 PM, Mayuresh Gharat <gharatmayures...@gmail.com > wrote: > To add on, the main thing here is you should be using only one of these > properties. > > Thanks, > > Mayuresh > > On Fri, Jul 24, 2015 at 6:47 PM, Mayuresh Gharat < > gharatmayures...@gmail.com > > wrote: > > > Yes. It should. Do not set other retention settings. Just use the "hours" > > settings. > > Let me know about this :) > > > > Thanks, > > > > Mayuresh > > > > On Fri, Jul 24, 2015 at 6:43 PM, JIEFU GONG <jg...@berkeley.edu> wrote: > > > >> Mayuresh, thanks for your comment. I won't be able to change these > >> settings > >> until next Monday, but just so confirm you are saying that if I restart > >> the > >> brokers my logs should delete themselves with respect to the newest > >> settings, correct? > >> ᐧ > >> > >> On Fri, Jul 24, 2015 at 6:29 PM, Mayuresh Gharat < > >> gharatmayures...@gmail.com > >> > wrote: > >> > >> > No. This should not happen. At Linkedin we just use the log retention > >> > hours. Try using that. Chang e it and bounce the broker. It should > work. > >> > Also looking back at the config's I am not sure why we had 3 different > >> > configs for the same property : > >> > > >> > "log.retention.ms" > >> > "log.retention.minutes" > >> > "log.retention.hours" > >> > > >> > We should probably be having just the milliseconds. > >> > > >> > Thanks, > >> > > >> > Mayuresh > >> > > >> > On Fri, Jul 24, 2015 at 4:12 PM, JIEFU GONG <jg...@berkeley.edu> > wrote: > >> > > >> > > Hi all, > >> > > > >> > > I have a few broad questions on how log deletion works, specifically > >> in > >> > > conjunction with the log.retention.time setting. Say I published > some > >> > > messages to some topics when the configuration was originally set to > >> > > something like log.retention.hours=168 (default). If I publish these > >> > > messages successfully, then later set the configuration to something > >> like > >> > > log.retention.minutes=1, are those logs supposed to persist for the > >> > newest > >> > > settings or the old settings? Right now my logs are refusing to > delete > >> > > themselves unless I specifically mark them for deletion -- is this > the > >> > > correct/anticipated/wanted behavior? > >> > > > >> > > Thanks for the help! > >> > > > >> > > -- > >> > > > >> > > Jiefu Gong > >> > > University of California, Berkeley | Class of 2017 > >> > > B.A Computer Science | College of Letters and Sciences > >> > > > >> > > jg...@berkeley.edu <elise...@berkeley.edu> | (925) 400-3427 > >> > > > >> > > >> > > >> > > >> > -- > >> > -Regards, > >> > Mayuresh R. Gharat > >> > (862) 250-7125 > >> > > >> > >> > >> > >> -- > >> > >> Jiefu Gong > >> University of California, Berkeley | Class of 2017 > >> B.A Computer Science | College of Letters and Sciences > >> > >> jg...@berkeley.edu <elise...@berkeley.edu> | (925) 400-3427 > >> > > > > > > > > -- > > -Regards, > > Mayuresh R. Gharat > > (862) 250-7125 > > > > > > -- > -Regards, > Mayuresh R. Gharat > (862) 250-7125 > -- Grant Henke Solutions Consultant | Cloudera ghe...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
