Hi Manikumar,

No, we are not using compression on our topics.  I will try out Todd Palino’s 
suggestion regarding the offsets topic.

Thanks
Rakesh




On 27/04/2016 23:34, "Manikumar Reddy" <[email protected]> wrote:

>Hi,
>
> Are you enabling log compaction on a topic with compressed messages?
> If yes, then that might be the reason for the exception.  0.8.2.2 Log
>Compaction does
> not support compressed  messages. This got fixed in 0.9.0.0 (KAFKA-1641,
>KAFKA-1374)
>
>Check below mail thread for some corrective actions
>http://grokbase.com/t/kafka/users/159jbe18en/log-cleaner-thread-stops
>
>
>On Thu, Apr 28, 2016 at 1:44 AM, Rakesh Vidyadharan <
>[email protected]> wrote:
>
>> Hello,
>>
>> We enabled log compaction on a few topics, as we want to preserve
>> permanently the latest versions of messages published to specific topics.
>> After enabling compaction, the log cleaner thread dies with the same error
>> for the topics we tried it on.  It looks like kafka has starting offset
>> that does not exist in the topic (at least that is how I am reading the
>> error).  Any ideas on how we can work around this error?
>>
>> Thanks
>> Rakesh
>>
>> [2016-04-27 15:52:11,306] INFO [kafka-log-cleaner-thread-0], Starting
>> (kafka.log.LogCleaner)
>> [2016-04-27 15:52:11,322] INFO Cleaner 0: Beginning cleaning of log
>> metamorphosis.lineup-0. (kafka.log.LogCleaner)
>> [2016-04-27 15:52:11,323] INFO Cleaner 0: Building offset map for
>> metamorphosis.lineup-0... (kafka.log.LogCleaner)
>> [2016-04-27 15:52:11,415] INFO Cleaner 0: Building offset map for log
>> metamorphosis.lineup-0 for 1 segments in offset range [1553258, 2138466).
>> (kafka.log.LogCleaner)
>> [2016-04-27 15:52:11,435] ERROR [kafka-log-cleaner-thread-0], Error due
>> to  (kafka.log.LogCleaner)
>> java.lang.IllegalArgumentException: requirement failed: Last clean offset
>> is 1553258 but segment base offset is 2125968 for log
>> metamorphosis.lineup-0.
>> at scala.Predef$.require(Predef.scala:233)
>> at kafka.log.Cleaner.buildOffsetMap(LogCleaner.scala:509)
>> at kafka.log.Cleaner.clean(LogCleaner.scala:307)
>> at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:221)
>> at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:199)
>> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
>> [2016-04-27 15:52:11,436] INFO [kafka-log-cleaner-thread-0], Stopped
>> (kafka.log.LogCleaner)
>>
>>

Reply via email to