[
https://issues.apache.org/jira/browse/KAFKA-4790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904333#comment-15904333
]
Pengwei commented on KAFKA-4790:
--------------------------------
[~junrao] If KIP-98 can fix this issue, we can wait for the KIP-98. There is
a method to avoid this issue, we should make sure the
log.segment.bytes / log.index.interval.bytes >= log.index.size.max.bytes / 8
8 bytes is every index item size
In our case, we should set log.segment.bytes to 512MB instead of 1G, after set
log.index.size.max.bytes to 1MB
> Kafka cannot recover after a disk full
> --------------------------------------
>
> Key: KAFKA-4790
> URL: https://issues.apache.org/jira/browse/KAFKA-4790
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.9.0.1, 0.10.1.1
> Reporter: Pengwei
> Labels: reliability
>
> [2017-02-23 18:43:57,736] INFO zookeeper state changed (SyncConnected)
> (org.I0Itec.zkclient.ZkClient)
> [2017-02-23 18:43:57,887] INFO Loading logs. (kafka.log.LogManager)
> [2017-02-23 18:43:57,935] INFO Recovering unflushed segment 0 in log test1-0.
> (kafka.log.Log)
> [2017-02-23 18:43:59,297] ERROR There was an error in one of the threads
> during logs loading: java.lang.IllegalArgumentException: requirement failed:
> Attempt to append to a full index (size = 128000). (kafka.log.LogManager)
> [2017-02-23 18:43:59,299] FATAL Fatal error during KafkaServer startup.
> Prepare to shutdown (kafka.server.KafkaServer)
> java.lang.IllegalArgumentException: requirement failed: Attempt to append to
> a full index (size = 128000).
> at scala.Predef$.require(Predef.scala:219)
> at
> kafka.log.OffsetIndex$$anonfun$append$1.apply$mcV$sp(OffsetIndex.scala:200)
> at kafka.log.OffsetIndex$$anonfun$append$1.apply(OffsetIndex.scala:199)
> at kafka.log.OffsetIndex$$anonfun$append$1.apply(OffsetIndex.scala:199)
> at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262)
> at kafka.log.OffsetIndex.append(OffsetIndex.scala:199)
> at kafka.log.LogSegment.recover(LogSegment.scala:191)
> at kafka.log.Log.recoverLog(Log.scala:259)
> at kafka.log.Log.loadSegments(Log.scala:234)
> at kafka.log.Log.<init>(Log.scala:92)
> at
> kafka.log.LogManager$$anonfun$loadLogs$2$$anonfun$4$$anonfun$apply$10$$anonfun$apply$1.apply$mcV$sp(LogManager.scala:201)
> at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:60)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
