[jira] [Commented] (KAFKA-3330) Truncate log cleaner offset checkpoint if the log is truncated

2016-05-05 Thread Jun Rao (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-3330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272621#comment-15272621
 ] 

Jun Rao commented on KAFKA-3330:


If you see the issue now, the simplest thing is to remove the affected 
partition from the cleaner-offset-checkpoint file when the broker is down. This 
will let the cleaner to start cleaning from the beginning of the log.

> Truncate log cleaner offset checkpoint if the log is truncated
> --
>
> Key: KAFKA-3330
> URL: https://issues.apache.org/jira/browse/KAFKA-3330
> Project: Kafka
>  Issue Type: Bug
>Reporter: Dong Lin
>Assignee: Dong Lin
>Priority: Critical
> Fix For: 0.10.0.0
>
>
> Were getting a number of failures of the log compaction thread with the
> following error:
> 2016/02/02 00:13:58.832 [LogCleaner] Cleaner 0: Beginning cleaning of log
> __consumer_offsets-93.
> 2016/02/02 00:13:58.832 [LogCleaner] Cleaner 0: Building offset map for
> __consumer_offsets-93...
> 2016/02/02 00:13:59.048 [LogCleaner] Cleaner 0: Building offset map for log
> __consumer_offsets-93 for 2 segments in offset range [11951210572,
> 11952632314).
> 2016/02/02 00:13:59.066 [LogCleaner] [kafka-log-cleaner-thread-0], Error
> due to
> java.lang.IllegalArgumentException: requirement failed: Last clean offset
> is 11951210572 but segment base offset is 11950300163 for log
> __consumer_offsets-93.
> at scala.Predef$.require(Predef.scala:233) ~[scala-library-2.10.4.jar:?]
> at kafka.log.Cleaner.buildOffsetMap(LogCleaner.scala:561)
> ~[kafka_2.10-0.8.2.56.jar:?]
> at kafka.log.Cleaner.clean(LogCleaner.scala:306)
> ~[kafka_2.10-0.8.2.56.jar:?]
> at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:217)
> ~[kafka_2.10-0.8.2.56.jar:?]
> at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:195)
> ~[kafka_2.10-0.8.2.56.jar:?]
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
> ~[kafka_2.10-0.8.2.56.jar:?]
> 2016/02/02 00:13:59.066 [LogCleaner] [kafka-log-cleaner-thread-0], Stopped
> We found that this may be caused in the following scenario:
> - we have three log segments with offset range [100, 200), [200, 300), and 
> [300, 400) respectively. 300 is the base offset of the active log segment. 
> Log cleaner offset checkpoint is also 300.
> - After log is truncated to offset 220, the log segments become [100, 200), 
> [200, 220). The Log cleaner offset checkpoint is still 300.
> - After new messages are appended to the log, the log segments become [100, 
> 200), [200, 320), [320, 420). The Log cleaner offset checkpoint is still 300.
> - Log cleaner cleans the log starting at offset 300. The require(offset == 
> start) in Cleaner.buildOffsetMap() fails because the the offset 300 is not 
> the base offset of any segments.
> To fix the problem, when the log is truncated to an offset smaller than 
> cleaner offset checkpoint, we should reset cleaner offset checkpoint to the 
> base offset of the active segment if this value is larger than the 
> checkpointed offset.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3330) Truncate log cleaner offset checkpoint if the log is truncated

2016-03-19 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-3330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200509#comment-15200509
 ] 

ASF GitHub Bot commented on KAFKA-3330:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/1009


> Truncate log cleaner offset checkpoint if the log is truncated
> --
>
> Key: KAFKA-3330
> URL: https://issues.apache.org/jira/browse/KAFKA-3330
> Project: Kafka
>  Issue Type: Bug
>Reporter: Dong Lin
>Assignee: Dong Lin
>Priority: Critical
> Fix For: 0.10.0.0
>
>
> Were getting a number of failures of the log compaction thread with the
> following error:
> 2016/02/02 00:13:58.832 [LogCleaner] Cleaner 0: Beginning cleaning of log
> __consumer_offsets-93.
> 2016/02/02 00:13:58.832 [LogCleaner] Cleaner 0: Building offset map for
> __consumer_offsets-93...
> 2016/02/02 00:13:59.048 [LogCleaner] Cleaner 0: Building offset map for log
> __consumer_offsets-93 for 2 segments in offset range [11951210572,
> 11952632314).
> 2016/02/02 00:13:59.066 [LogCleaner] [kafka-log-cleaner-thread-0], Error
> due to
> java.lang.IllegalArgumentException: requirement failed: Last clean offset
> is 11951210572 but segment base offset is 11950300163 for log
> __consumer_offsets-93.
> at scala.Predef$.require(Predef.scala:233) ~[scala-library-2.10.4.jar:?]
> at kafka.log.Cleaner.buildOffsetMap(LogCleaner.scala:561)
> ~[kafka_2.10-0.8.2.56.jar:?]
> at kafka.log.Cleaner.clean(LogCleaner.scala:306)
> ~[kafka_2.10-0.8.2.56.jar:?]
> at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:217)
> ~[kafka_2.10-0.8.2.56.jar:?]
> at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:195)
> ~[kafka_2.10-0.8.2.56.jar:?]
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
> ~[kafka_2.10-0.8.2.56.jar:?]
> 2016/02/02 00:13:59.066 [LogCleaner] [kafka-log-cleaner-thread-0], Stopped
> We found that this may be caused in the following scenario:
> - we have three log segments with offset range [100, 200), [200, 300), and 
> [300, 400) respectively. 300 is the base offset of the active log segment. 
> Log cleaner offset checkpoint is also 300.
> - After log is truncated to offset 220, the log segments become [100, 200), 
> [200, 220). The Log cleaner offset checkpoint is still 300.
> - After new messages are appended to the log, the log segments become [100, 
> 200), [200, 320), [320, 420). The Log cleaner offset checkpoint is still 300.
> - Log cleaner cleans the log starting at offset 300. The require(offset == 
> start) in Cleaner.buildOffsetMap() fails because the the offset 300 is not 
> the base offset of any segments.
> To fix the problem, when the log is truncated to an offset smaller than 
> cleaner offset checkpoint, we should reset cleaner offset checkpoint to the 
> base offset of the active segment if this value is larger than the 
> checkpointed offset.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3330) Truncate log cleaner offset checkpoint if the log is truncated

2016-03-03 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-3330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15178784#comment-15178784
 ] 

ASF GitHub Bot commented on KAFKA-3330:
---

GitHub user lindong28 opened a pull request:

https://github.com/apache/kafka/pull/1009

KAFKA-3330; Truncate log cleaner offset checkpoint if the log is truncated

@becketqin Can you take a look?

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lindong28/kafka KAFKA-3330

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1009.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1009


commit 5fc32b4020198dfeabc6bba80d4657f7de20e169
Author: Dong Lin 
Date:   2016-03-03T23:02:23Z

KAFKA-3330; Truncate log cleaner offset checkpoint if the log is truncated




> Truncate log cleaner offset checkpoint if the log is truncated
> --
>
> Key: KAFKA-3330
> URL: https://issues.apache.org/jira/browse/KAFKA-3330
> Project: Kafka
>  Issue Type: Bug
>Reporter: Dong Lin
>Assignee: Dong Lin
>
> Were getting a number of failures of the log compaction thread with the
> following error:
> 2016/02/02 00:13:58.832 [LogCleaner] Cleaner 0: Beginning cleaning of log
> __consumer_offsets-93.
> 2016/02/02 00:13:58.832 [LogCleaner] Cleaner 0: Building offset map for
> __consumer_offsets-93...
> 2016/02/02 00:13:59.048 [LogCleaner] Cleaner 0: Building offset map for log
> __consumer_offsets-93 for 2 segments in offset range [11951210572,
> 11952632314).
> 2016/02/02 00:13:59.066 [LogCleaner] [kafka-log-cleaner-thread-0], Error
> due to
> java.lang.IllegalArgumentException: requirement failed: Last clean offset
> is 11951210572 but segment base offset is 11950300163 for log
> __consumer_offsets-93.
> at scala.Predef$.require(Predef.scala:233) ~[scala-library-2.10.4.jar:?]
> at kafka.log.Cleaner.buildOffsetMap(LogCleaner.scala:561)
> ~[kafka_2.10-0.8.2.56.jar:?]
> at kafka.log.Cleaner.clean(LogCleaner.scala:306)
> ~[kafka_2.10-0.8.2.56.jar:?]
> at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:217)
> ~[kafka_2.10-0.8.2.56.jar:?]
> at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:195)
> ~[kafka_2.10-0.8.2.56.jar:?]
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
> ~[kafka_2.10-0.8.2.56.jar:?]
> 2016/02/02 00:13:59.066 [LogCleaner] [kafka-log-cleaner-thread-0], Stopped
> We found that this may be caused in the following scenario:
> - we have three log segments with offset range [100, 200), [200, 300), and 
> [300, 400) respectively. 300 is the base offset of the active log segment. 
> Log cleaner offset checkpoint is also 300.
> - After log is truncated to offset 220, the log segments become [100, 200), 
> [200, 220). The Log cleaner offset checkpoint is still 300.
> - After new messages are appended to the log, the log segments become [100, 
> 200), [200, 320), [320, 420). The Log cleaner offset checkpoint is still 300.
> - Log cleaner cleans the log starting at offset 300. The require(offset == 
> start) in Cleaner.buildOffsetMap() fails because the the offset 300 is not 
> the base offset of any segments.
> To fix the problem, when the log is truncated to an offset smaller than 
> cleaner offset checkpoint, we should reset cleaner offset checkpoint to the 
> base offset of the active segment if this value is larger than the 
> checkpointed offset.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)