On Wed, Feb 4, 2015 at 9:23 PM, Jun Rao <j...@confluent.io> wrote: > Could you try the 0.8.2.0 release? It fixed one issue related to topic > creation. >
Jun, Tried with 0.8.2.0 and I still see the same error. I see the error given below almost incessantly on the client side for topic LAX1-GRIFFIN-r47-1423165897627. Things look fine one the broker side: Topic:LAX1-GRIFFIN-r47-1423165897627 PartitionCount:8 ReplicationFactor:1 Configs: Topic: LAX1-GRIFFIN-r47-1423165897627 Partition: 0 Leader: 49817 Replicas: 49817 Isr: 49817 Topic: LAX1-GRIFFIN-r47-1423165897627 Partition: 1 Leader: 49818 Replicas: 49818 Isr: 49818 Topic: LAX1-GRIFFIN-r47-1423165897627 Partition: 2 Leader: 49814 Replicas: 49814 Isr: 49814 Topic: LAX1-GRIFFIN-r47-1423165897627 Partition: 3 Leader: 49817 Replicas: 49817 Isr: 49817 Topic: LAX1-GRIFFIN-r47-1423165897627 Partition: 4 Leader: 49818 Replicas: 49818 Isr: 49818 Topic: LAX1-GRIFFIN-r47-1423165897627 Partition: 5 Leader: 49814 Replicas: 49814 Isr: 49814 Topic: LAX1-GRIFFIN-r47-1423165897627 Partition: 6 Leader: 49817 Replicas: 49817 Isr: 49817 Topic: LAX1-GRIFFIN-r47-1423165897627 Partition: 7 Leader: 49818 Replicas: 49818 Isr: 49818 Broker logs at http://d.pr/f/1jLOf/5H8uWPqu in case you need them ERROR: 2015-02-05 12:02:03,764 (LAX1-GRIFFIN-r47-1423165897627-pf6898-lax1-GriffinDownloader-1423166442395_f27c6501f3d5-1423166444213-d572da8f-leader-finder-thread) ClientUtils$ WARN: Fetching topic metadata with correlation id 152 for topics [Set(LAX1-GRIFFIN-r47-1423165897627)] from broker [id:49814,host:172.16.204.44,port:49814] failed java.lang.ArrayIndexOutOfBoundsException: 7 at kafka.api.TopicMetadata$$anonfun$readFrom$1.apply$mcVI$sp(TopicMetadata.scala:38) at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:78) at kafka.api.TopicMetadata$.readFrom(TopicMetadata.scala:36) at kafka.api.TopicMetadataResponse$$anonfun$3.apply(TopicMetadataResponse.scala:31) at kafka.api.TopicMetadataResponse$$anonfun$3.apply(TopicMetadataResponse.scala:31) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233) at scala.collection.immutable.Range.foreach(Range.scala:81) at scala.collection.TraversableLike$class.map(TraversableLike.scala:233) at scala.collection.immutable.Range.map(Range.scala:46) at kafka.api.TopicMetadataResponse$.readFrom(TopicMetadataResponse.scala:31) at kafka.producer.SyncProducer.send(SyncProducer.scala:114) at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58) at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93) at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) > > Thanks, > > Jun > > On Wed, Feb 4, 2015 at 12:58 AM, Sumit Rangwala <sumitrangw...@gmail.com> > wrote: > > > I am observing the following exception with kafka client: > > > > 2015-02-04 00:17:27,345 > > > > > (LAX1-GRIFFIN-r8-1423037468055-pf13797-lax1-GriffinDownloader-1423037818264_c7b1e843ff51-1423037822122-eb7afca7-leader-finder-thread) > > ClientUtils$ WARN: Fetching topic metadata with correlation id 112 for > > topics [Set(LAX1-GRIFFIN-r8-1423037468055)] from broker > > [id:49649,host:172.16.204.44,port:49649] failed > > java.lang.ArrayIndexOutOfBoundsException: 7 > > at > > > > > kafka.api.TopicMetadata$$anonfun$readFrom$1.apply$mcVI$sp(TopicMetadata.scala:38) > > at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:78) > > at kafka.api.TopicMetadata$.readFrom(TopicMetadata.scala:36) > > at > > > > > kafka.api.TopicMetadataResponse$$anonfun$3.apply(TopicMetadataResponse.scala:31) > > at > > > > > kafka.api.TopicMetadataResponse$$anonfun$3.apply(TopicMetadataResponse.scala:31) > > at > > > > > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233) > > at > > > > > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233) > > at scala.collection.immutable.Range.foreach(Range.scala:81) > > at scala.collection.TraversableLike$class.map(TraversableLike.scala:233) > > at scala.collection.immutable.Range.map(Range.scala:46) > > at > > kafka.api.TopicMetadataResponse$.readFrom(TopicMetadataResponse.scala:31) > > at kafka.producer.SyncProducer.send(SyncProducer.scala:115) > > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58) > > at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93) > > at > > > > > kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66) > > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) > > > > > > > > # Background on what my code is doing: > > > > In my setup kafka brokers are set for auto topic creation. In the > scenario > > above a node informs other nodes (currently 5 in total) about ~50 new > > (non-existent) topics, and all the nodes almost simultaneously open a > > consumer for each of these topics. This triggers topic creation for all > the > > topics on kafka brokers. Most of the topics are created fine but there is > > almost always few topics that throw the above exception and a kafka > > producer is unable to send any data to such a topic > > (LAX1-GRIFFIN-r8-1423037468055 > > in the above case) > > > > > > # Logs > > All kafka broker logs (3 brokers) available at > > http://d.pr/f/1eOGM/5UPMPfg5 > > For these logs only LAX1-GRIFFIN-r8-1423037468055 had an issue. All other > > topics were fine. > > > > > > Setup > > -------- > > Zookeeper: 3.4.6 > > Kafka broker: 0.8.2-beta > > Kafka clients: 0.8.2-beta > > > > # Kafka boker settings (all other settings are default 0.8.2-beta > settings) > > kafka.controlled.shutdown.enable: 'FALSE' > > kafka.auto.create.topics.enable: 'TRUE' > > kafka.num.partitions: 8 > > kafka.default.replication.factor: 1 > > kafka.rebalance.backoff.ms: 3000 > > kafka.rebalance.max.retries: 10 > > kafka.log.retention.minutes: 1200 > > kafka.delete.topic.enable: 'TRUE' > > > > > > > > Sumit > > >
