Hi Brijendra, I suspect that the path you are using for log.dir is being updated/modified/deleted etc. Can you ensure that this path -> D:\kafka\kafka_logs is not being cleaned/modified in anyway (manually/automatically/by some application)? Are you going to use windows server or linux in production? If you going to use linux do not configure brokers to use /tmp path. log.dir path should exclusively be for Kafka broker process only.
Let me know if you have any findings. Thanks, Akash On Thu, Sep 5, 2024 at 8:26 PM Brijendra Suman <brijendra.su...@nic.in.invalid> wrote: > Dear Kafka Team, > > > > We are developing one application in which we are utilising Kafka for some > use cases. > > > > We have created a Topic and retention period we have set is 1 hour for a > topic. It has been observed that when Kafka starts deleting the messages > from the topics then below error pops up and servers shuts down > automatically. > > > > [2024-09-05 10:31:06,964] INFO [ReplicaFetcherManager on broker 1] Removed > fetcher for partitions Set(candidates_identity-0) > (kafka.server.ReplicaFetcherManager) > > [2024-09-05 10:31:06,965] INFO [ReplicaAlterLogDirsManager on broker 1] > Removed fetcher for partitions Set(candidates_identity-0) > (kafka.server.ReplicaAlterLogDirsManager) > > [2024-09-05 10:31:06,969] WARN [ReplicaManager broker=1] Broker 1 stopped > fetcher for partitions candidates_identity-0 and stopped moving logs for > partitions because they are in the failed log directory > D:\kafka\kafka_logs. (kafka.server.ReplicaManager) > > [2024-09-05 10:31:06,970] WARN Stopping serving logs in dir > D:\kafka\kafka_logs (kafka.log.LogManager) > > [2024-09-05 10:31:06,972] ERROR Shutdown broker because all log dirs in > D:\kafka\kafka_logs have failed (kafka.log.LogManager) > > > > > > Please help as we are planning to utilize Kafka in production but with > this issue we are stuck. > > > > > > > Thanks & Regards, > > Brijendra Kumar Suman > > Scientist-B > > NRA Division, NIC > > Mobile - 9754661383, IP Phone number - 5399
