I have seen this issue as well with 0.9. I also thought that it was because of the upgrade, but that doesn't seem to be it. But there were also a couple of instances when it didn't change the timestamps, so I was unable to pinpoint the exact root cause or steps and hence had not yet posted it here.
On Wed, May 25, 2016 at 2:15 PM, tao xiao <xiaotao...@gmail.com> wrote: > I noticed the same issue too with 0.9. > > On Wed, 25 May 2016 at 09:49 Andrew Otto <o...@wikimedia.org> wrote: > > > “We use the default log retention of 7 *days*" :)* > > > > On Wed, May 25, 2016 at 12:34 PM, Andrew Otto <o...@wikimedia.org> > wrote: > > > > > Hiya, > > > > > > We’ve recently upgraded to 0.9. In 0.8, when we restarted a broker, > data > > > log file mtimes were not changed. In 0.9, any data log file that was > on > > > disk before the broker has it’s mtime modified to the time of the > broker > > > restart. > > > > > > This causes problems with log retention, as all the files then look > like > > > they contain recent data to kafka. We use the default log retention > of 7 > > > weeks, but if all the files are touched at the same time, this can > cause > > us > > > to retain up to 2 weeks of log data, which can fill up our disks. > > > > > > We saw this during our initial upgrade, but I had just thought it had > > > something to do with the change of inter.broker.protocol.version, and > > > assumed it wouldn’t happen again. We just did our first broker restart > > > after the upgrade, and we are seeing this again. We worked around this > > > during our upgrade by temporarily setting a high volume topic’s > retention > > > very low, causing brokers to purge more recent data. This allowed us > to > > > avoid filling up our disks, but we shouldn’t have to do this every time > > we > > > bounce brokers. > > > > > > Has anyone else noticed this? > > > > > > -Ao > > > > > >
