Hi, I’m shutting down kafka properly see down the bottom. But on reboot, I sometimes get the following:
[2016-12-23 07:45:26,544] INFO Loading logs. (kafka.log.LogManager) [2016-12-23 07:45:26,609] WARN Found a corrupted index file due to requirement failed: Corrupt index found, index file (/mnt/kafka-logs/__consumer_offsets-36/00000000000000000000.index) has non-zero size but the last offset is 0 which is no larger than the base offset 0.}. deleting /mnt/kafka-logs/__consumer_offsets-36/00000000000000000000.timeindex, /mnt/kafka-logs/__consumer_offsets-36/00000000000000000000.index and rebuilding index... (kafka.log.Log) [2016-12-23 07:45:26,617] INFO Recovering unflushed segment 0 in log __consumer_offsets-36. (kafka.log.Log) [2016-12-23 07:45:26,621] INFO Completed load of log __consumer_offsets-36 with 1 log segments and log end offset 0 in 28 ms (kafka.log.Log) [2016-12-23 07:45:26,645] WARN Found a corrupted index file due to requirement failed: Corrupt index found, index file (/mnt/kafka-logs/dsresolvedbet-5/00000000000000590991.index) has non-zero size but the last offset is 590991 which is no larger than the base offset 590991.}. deleting /mnt/kafka-logs/dsresolvedbet-5/00000000000000590991.timeindex, /mnt/kafka-logs/dsresolvedbet-5/00000000000000590991.index and rebuilding index... (kafka.log.Log) [2016-12-23 07:45:31,683] INFO Recovering unflushed segment 590991 in log dsresolvedbet-5. (kafka.log.Log) [2016-12-23 07:45:31,930] INFO Completed load of log dsresolvedbet-5 with 6 log segments and log end offset 650751 in 5301 ms (kafka.log.Log) I’m a little bit puzzled by those, do you have any insights into what may be the cause? Thanks! Stephane Shutdown log: [2016-12-23 07:44:31,731] INFO [Kafka Server 7], Controlled shutdown succeeded (kafka.server.KafkaServer) [2016-12-23 07:44:31,737] INFO [Socket Server on Broker 7], Shutting down (kafka.network.SocketServer) [2016-12-23 07:44:31,744] INFO [Socket Server on Broker 7], Shutdown completed (kafka.network.SocketServer) [2016-12-23 07:44:31,749] INFO [Kafka Request Handler on Broker 7], shutting down (kafka.server.KafkaRequestHandlerPool) [2016-12-23 07:44:31,883] INFO [Kafka Request Handler on Broker 7], shut down completely (kafka.server.KafkaRequestHandlerPool) [2016-12-23 07:44:31,885] INFO [ThrottledRequestReaper-Fetch], Shutting down (kafka.server.ClientQuotaManager$ThrottledRequestReaper) [2016-12-23 07:44:32,429] INFO [ThrottledRequestReaper-Fetch], Stopped (kafka.server.ClientQuotaManager$ThrottledRequestReaper) [2016-12-23 07:44:32,430] INFO [ThrottledRequestReaper-Fetch], Shutdown completed (kafka.server.ClientQuotaManager$ThrottledRequestReaper) [2016-12-23 07:44:32,430] INFO [ThrottledRequestReaper-Produce], Shutting down (kafka.server.ClientQuotaManager$ThrottledRequestReaper) [2016-12-23 07:44:33,429] INFO [ThrottledRequestReaper-Produce], Stopped (kafka.server.ClientQuotaManager$ThrottledRequestReaper) [2016-12-23 07:44:33,429] INFO [ThrottledRequestReaper-Produce], Shutdown completed (kafka.server.ClientQuotaManager$ThrottledRequestReaper) [2016-12-23 07:44:33,430] INFO [KafkaApi-7] Shutdown complete. (kafka.server.KafkaApis) [2016-12-23 07:44:33,461] INFO [Replica Manager on Broker 7]: Shutting down (kafka.server.ReplicaManager) [2016-12-23 07:44:33,461] INFO [ReplicaFetcherManager on broker 7] shutting down (kafka.server.ReplicaFetcherManager) [2016-12-23 07:44:33,462] INFO [ReplicaFetcherThread-0-8], Shutting down (kafka.server.ReplicaFetcherThread) [2016-12-23 07:44:33,868] INFO [ReplicaFetcherThread-0-8], Stopped (kafka.server.ReplicaFetcherThread) [2016-12-23 07:44:33,869] INFO [ReplicaFetcherThread-0-8], Shutdown completed (kafka.server.ReplicaFetcherThread) [2016-12-23 07:44:33,872] INFO [ReplicaFetcherThread-0-9], Shutting down (kafka.server.ReplicaFetcherThread) [2016-12-23 07:44:34,305] INFO [ReplicaFetcherThread-0-9], Stopped (kafka.server.ReplicaFetcherThread) [2016-12-23 07:44:34,305] INFO [ReplicaFetcherThread-0-9], Shutdown completed (kafka.server.ReplicaFetcherThread) [2016-12-23 07:44:34,309] INFO [ReplicaFetcherManager on broker 7] shutdown completed (kafka.server.ReplicaFetcherManager) [2016-12-23 07:44:34,309] INFO [ExpirationReaper-7], Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2016-12-23 07:44:34,498] INFO [ExpirationReaper-7], Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2016-12-23 07:44:34,498] INFO [ExpirationReaper-7], Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2016-12-23 07:44:34,498] INFO [ExpirationReaper-7], Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2016-12-23 07:44:34,508] INFO [ExpirationReaper-7], Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2016-12-23 07:44:34,508] INFO [ExpirationReaper-7], Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2016-12-23 07:44:34,512] INFO [Replica Manager on Broker 7]: Shut down completely (kafka.server.ReplicaManager) [2016-12-23 07:44:34,513] INFO [ExpirationReaper-7], Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2016-12-23 07:44:34,659] INFO [ExpirationReaper-7], Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2016-12-23 07:44:34,660] INFO [ExpirationReaper-7], Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2016-12-23 07:44:34,661] INFO Shutting down. (kafka.log.LogManager) [2016-12-23 07:44:34,662] INFO Shutting down the log cleaner. (kafka.log.LogCleaner) [2016-12-23 07:44:34,662] INFO [kafka-log-cleaner-thread-0], Shutting down (kafka.log.LogCleaner) [2016-12-23 07:44:34,663] INFO [kafka-log-cleaner-thread-0], Stopped (kafka.log.LogCleaner) [2016-12-23 07:44:34,663] INFO [kafka-log-cleaner-thread-0], Shutdown completed (kafka.log.LogCleaner)
