Seeing this exception a LOT (3-4 times per second, same log topic). I'm using external code to feed data to about 50 different log topics over a cluster of 3 Kafka 0.8 brokers. There are 3 ZooKeeper instances as well, all of this is running on EC2. My application creates a high-level consumer (1 per topic) to consumer data from each and do further processing.
The problem is this exception is in the high-level consumer, so my code has no way of knowing that it's become stuck. This exception does not always appear, but as far as I can tell, once this happens, the only cure is to restart my application's process. I saw this in 0.8 built from source about 1 week ago, and also am seeing it today after pulling the latest 0.8 sources and rebuilding Kafka. Thoughts? Failed to find leader for Set([topic6,0]): java.lang.NullPointerException at org.I0Itec.zkclient.ZkClient$2.call(ZkClient.java:416) at org.I0Itec.zkclient.ZkClient$2.call(ZkClient.java:413) at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675) at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:413) at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:409) at kafka.utils.ZkUtils$.getChildrenParentMayNotExist(ZkUtils.scala:438) at kafka.utils.ZkUtils$.getAllBrokersInCluster(ZkUtils.scala:75) at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:63) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)