[
https://issues.apache.org/jira/browse/KAFKA-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276497#comment-14276497
]
Alex Tian commented on KAFKA-1112:
----------------------------------
I also met similar issues in Kafka-2.9.2-0.8.1 ( after an uncleanShutDown ---
kill -9. ).
I am afraid that I cannot provide more logs since it was months ago:
******************************************************************************************************************************
FATAL Fatal error during KafkaServerStable startup. Prepare to shutdown
(kafka.server.KafkaServerStartable)
java.lang.IllegalArgumentException: requirement failed: Corrupt index found,
index file
(/home/storm/kafka_2.9.2-0.8.1/logs/tracker-1/00000000000006911940.index) has
non-zero size but the last offset is 6911940 and the base offset is 6911940
******************************************************************************************************************************
Here is my analysis:
Once unclean shutdown, Kafka needs to rebuild index files for her logsegments
when restarted.
If InvalidOffsetException is thrown when appending entries to an index file,
recoverLog() function will handle it by truncating the log segment to
baseOffset (==> lastIndexEntry == baseOffset ).
However, since the index files are written by mmap, OS will flush the update to
disks in time. It means the index file already has some entries now.
I am a beginner for scala, just guessing the logSegment with
InvalidOffsetException thrown has been passed by the iterator, so it was not
deleted in fact? (I mean all log segments after it will be deleted in
recoverLog() in log.scala. )
This log segment missed to delete could not pass sanityCheck(), since it has
index file with non-zero size but lastIndexEntry == baseOffset ( it was
truncated to baseOffset when handling the InvalidOffsetException. ).
Quite sorry if any mistake above.
> broker can not start itself after kafka is killed with -9
> ---------------------------------------------------------
>
> Key: KAFKA-1112
> URL: https://issues.apache.org/jira/browse/KAFKA-1112
> Project: Kafka
> Issue Type: Bug
> Components: log
> Affects Versions: 0.8.0, 0.8.1
> Reporter: Kane Kim
> Assignee: Jay Kreps
> Priority: Critical
> Fix For: 0.8.1
>
> Attachments: KAFKA-1112-v1.patch, KAFKA-1112-v2.patch,
> KAFKA-1112-v3.patch, KAFKA-1112-v4.patch, KAFKA-1112.out
>
>
> When I kill kafka with -9, broker cannot start itself because of corrupted
> index logs. I think kafka should try to delete/rebuild indexes itself without
> manual intervention.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
