Hi, Are you enabling log compaction on a topic with compressed messages? If yes, then that might be the reason for the exception. 0.8.2.2 Log Compaction does not support compressed messages. This got fixed in 0.9.0.0 (KAFKA-1641, KAFKA-1374)
Check below mail thread for some corrective actions http://grokbase.com/t/kafka/users/159jbe18en/log-cleaner-thread-stops On Thu, Apr 28, 2016 at 1:44 AM, Rakesh Vidyadharan < [email protected]> wrote: > Hello, > > We enabled log compaction on a few topics, as we want to preserve > permanently the latest versions of messages published to specific topics. > After enabling compaction, the log cleaner thread dies with the same error > for the topics we tried it on. It looks like kafka has starting offset > that does not exist in the topic (at least that is how I am reading the > error). Any ideas on how we can work around this error? > > Thanks > Rakesh > > [2016-04-27 15:52:11,306] INFO [kafka-log-cleaner-thread-0], Starting > (kafka.log.LogCleaner) > [2016-04-27 15:52:11,322] INFO Cleaner 0: Beginning cleaning of log > metamorphosis.lineup-0. (kafka.log.LogCleaner) > [2016-04-27 15:52:11,323] INFO Cleaner 0: Building offset map for > metamorphosis.lineup-0... (kafka.log.LogCleaner) > [2016-04-27 15:52:11,415] INFO Cleaner 0: Building offset map for log > metamorphosis.lineup-0 for 1 segments in offset range [1553258, 2138466). > (kafka.log.LogCleaner) > [2016-04-27 15:52:11,435] ERROR [kafka-log-cleaner-thread-0], Error due > to (kafka.log.LogCleaner) > java.lang.IllegalArgumentException: requirement failed: Last clean offset > is 1553258 but segment base offset is 2125968 for log > metamorphosis.lineup-0. > at scala.Predef$.require(Predef.scala:233) > at kafka.log.Cleaner.buildOffsetMap(LogCleaner.scala:509) > at kafka.log.Cleaner.clean(LogCleaner.scala:307) > at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:221) > at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:199) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) > [2016-04-27 15:52:11,436] INFO [kafka-log-cleaner-thread-0], Stopped > (kafka.log.LogCleaner) > >
