iamgd67 opened a new pull request #10818: URL: https://github.com/apache/kafka/pull/10818
more detail see jira [KAFKA-12889](https://issues.apache.org/jira/browse/KAFKA-12889) to avoid log index 4 byte relative offset overflow, log cleaner group check log segments offset to make sure group offset range not exceed Int.MaxValue. this offset check currentlly not cosider next is next log segment is empty, so there will left empty log files every about 2^31 messages. the left empty logs will be reprocessed every clean cycle, which will rewrite it with same empty content, witch cause little no need io. for __consumer_offsets topic, normally we can set cleanup.policy to compact,delete to get rid of this. my cluster is 0.10.1.1, but after aylize trunk code, it should has same problem too. some of my left empty logs,(run ls -l) rw-r---- 1 u g 0 Dec 16 2017 00000000000000000000.index rw-r---- 1 u g 0 Dec 16 2017 00000000000000000000.log rw-r---- 1 u g 0 Dec 16 2017 00000000000000000000.timeindex rw-r---- 1 u g 0 Jan 15 2018 00000000002148249632.index rw-r---- 1 u g 0 Jan 15 2018 00000000002148249632.log rw-r---- 1 u g 0 Jan 15 2018 00000000002148249632.timeindex rw-r---- 1 u g 0 Jan 27 2018 00000000004295766494.index rw-r---- 1 u g 0 Jan 27 2018 00000000004295766494.log rw-r---- 1 u g 0 Jan 27 2018 00000000004295766494.timeindex -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
