Not bothering. This is good info we should document.... On May 27, 2016 11:47 AM, "Igor Kravzov" <igork.ine...@gmail.com> wrote:
> Guys, I found the problem. It was my fault. > Kafka broker configured as fully qualified domain name. > What happens is when you call zookeper it returns FQDN but my hosts file > was not updated with IPs and node names. > > Sorry guys for bothering you. > > On Thu, May 26, 2016 at 8:08 PM Andrew Psaltis <psaltis.and...@gmail.com> > wrote: > >> Hi Igor, >> It looks from the stack trace that the consumer is having trouble finding >> the leader for topic "Test1". Can you execute on one of your brokers, >> perhaps the one with ip address 10.1.132.113 the following command: >> >> $KAFKA_HOME/bin/kafka-topics.sh --describe --zookeeper <urls> --topic >> Test1 >> >> and let me know what the output is. >> >> Also another test that you can do to take NiFi out of the picture is to >> use the console consumer. You can execute that with the following command, >> again from one of your Kafka brokers. >> >> $KAFKA_HOME/bin/kafka-console-consumer.sh --zookeeper <urls> --topic Test1 >> >> If you do not have a process writing any data into kafka, then you can >> add the following parameter to the above command "--from-beginning" >> >> >> Looking forward to hearing back from you. >> >> Thanks, >> Andrew >> >> >> On Thu, May 26, 2016 at 5:44 PM, Igor Kravzov <igork.ine...@gmail.com> >> wrote: >> >>> Thanks Oleg. I'll try. If you find more data please let me know.. >>> >>> On Thu, May 26, 2016 at 5:39 PM, Oleg Zhurakousky < >>> ozhurakou...@hortonworks.com> wrote: >>> >>>> Igor, you may want to go they Kafka mailing list. I've seen this >>>> several times, and all I know it has to do with broker >>>> I'll look in more details once back on line >>>> >>>> Oleg >>>> >>>> Sent from my iPhone >>>> >>>> On May 26, 2016, at 17:20, Igor Kravzov <igork.ine...@gmail.com> wrote: >>>> >>>> Why I am getting exception bellow >>>> >>>> >>>> 2016-05-26 17:13:36,776 INFO >>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >>>> kafka.consumer.ConsumerFetcherManager >>>> [ConsumerFetcherManager-1464297163222] Added fetcher for partitions >>>> ArrayBuffer() >>>> 2016-05-26 17:13:36,981 INFO >>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >>>> kafka.utils.VerifiableProperties Verifying properties >>>> 2016-05-26 17:13:36,981 INFO >>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >>>> kafka.utils.VerifiableProperties Property client.id is overridden to >>>> NiFi-d217bceb-d58b-4031-9143-26d8ddd15e40 >>>> 2016-05-26 17:13:36,981 INFO >>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >>>> kafka.utils.VerifiableProperties Property metadata.broker.list is >>>> overridden to 10.1.132.113:6667 >>>> 2016-05-26 17:13:36,981 INFO >>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >>>> kafka.utils.VerifiableProperties Property request.timeout.ms is >>>> overridden to 30000 >>>> 2016-05-26 17:13:36,981 INFO >>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >>>> kafka.client.ClientUtils$ Fetching metadata from broker >>>> id:1001,host:10.1.132.113,port:6667 with correlation id 256 for 1 topic(s) >>>> Set(Test1) >>>> 2016-05-26 17:13:36,981 INFO >>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >>>> kafka.producer.SyncProducer Connected to 10.1.132.113:6667 for >>>> producing >>>> 2016-05-26 17:13:36,981 INFO >>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >>>> kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667 >>>> 2016-05-26 17:13:36,981 WARN >>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >>>> kafka.client.ClientUtils$ Fetching topic metadata with correlation id 256 >>>> for topics [Set(Test1)] from broker [id:1001,host:10.1.132.113,port:6667] >>>> failed >>>> java.nio.channels.ClosedChannelException: null >>>> at kafka.network.BlockingChannel.send(BlockingChannel.scala:100) >>>> ~[kafka_2.10-0.8.2.2.jar:na] >>>> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73) >>>> ~[kafka_2.10-0.8.2.2.jar:na] >>>> at >>>> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72) >>>> ~[kafka_2.10-0.8.2.2.jar:na] >>>> at kafka.producer.SyncProducer.send(SyncProducer.scala:113) >>>> ~[kafka_2.10-0.8.2.2.jar:na] >>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58) >>>> [kafka_2.10-0.8.2.2.jar:na] >>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93) >>>> [kafka_2.10-0.8.2.2.jar:na] >>>> at >>>> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66) >>>> [kafka_2.10-0.8.2.2.jar:na] >>>> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) >>>> [kafka_2.10-0.8.2.2.jar:na] >>>> 2016-05-26 17:13:36,981 INFO >>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >>>> kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667 >>>> 2016-05-26 17:13:36,981 WARN >>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >>>> k.c.ConsumerFetcherManager$LeaderFinderThread >>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread], >>>> Failed to find leader for Set([Test1,0]) >>>> kafka.common.KafkaException: fetching topic metadata for topics >>>> [Set(Test1)] from broker [ArrayBuffer(id:1001,host:10.1.132.113,port:6667)] >>>> failed >>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72) >>>> ~[kafka_2.10-0.8.2.2.jar:na] >>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93) >>>> ~[kafka_2.10-0.8.2.2.jar:na] >>>> at >>>> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66) >>>> ~[kafka_2.10-0.8.2.2.jar:na] >>>> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) >>>> [kafka_2.10-0.8.2.2.jar:na] >>>> Caused by: java.nio.channels.ClosedChannelException: null >>>> at kafka.network.BlockingChannel.send(BlockingChannel.scala:100) >>>> ~[kafka_2.10-0.8.2.2.jar:na] >>>> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73) >>>> ~[kafka_2.10-0.8.2.2.jar:na] >>>> at >>>> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72) >>>> ~[kafka_2.10-0.8.2.2.jar:na] >>>> at kafka.producer.SyncProducer.send(SyncProducer.scala:113) >>>> ~[kafka_2.10-0.8.2.2.jar:na] >>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58) >>>> ~[kafka_2.10-0.8.2.2.jar:na] >>>> ... 3 common frames omitted >>>> >>>> >>> >> >> >> -- >> Thanks, >> Andrew >> >> Subscribe to my book: Streaming Data <http://manning.com/psaltis> >> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306> >> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata> >> >
