You may need to also check your Metron configs for the readers/writers via Ambari. Since it's failing in the bulk writer, I suspect there may still be a problem with the config there.
On Thu, Sep 5, 2019 at 7:24 AM Hema malini <[email protected]> wrote: > That issue got resolved after changing the property listeners to > PLAINTEXT://:9092. > > I pushed the logsto bro and syslog.i could see messages only in bro > topology but with the error as below.can someone pls help > > 2019-09-05 18:43:31.622 o.a.s.d.executor Thread-12-parserBolt-executor[5 5] > [ERROR] > java.util.concurrent.ExecutionException: > org.apache.kafka.common.errors.TimeoutException: Failed to update metadata > after 60000 ms. > at > org.apache.kafka.clients.producer.KafkaProducer$FutureFailure.<init>(KafkaProducer.java:730) > ~[stormjar.jar:?] > at > org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:483) > ~[stormjar.jar:?] > at > org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:430) > ~[stormjar.jar:?] > at > org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:353) > ~[stormjar.jar:?] > at > org.apache.metron.writer.kafka.KafkaWriter.write(KafkaWriter.java:257) > ~[stormjar.jar:?] > at > org.apache.metron.writer.BulkWriterComponent.flush(BulkWriterComponent.java:123) > [stormjar.jar:?] > at > org.apache.metron.writer.BulkWriterComponent.applyShouldFlush(BulkWriterComponent.java:179) > [stormjar.jar:?] > at > org.apache.metron.writer.BulkWriterComponent.write(BulkWriterComponent.java:99) > [stormjar.jar:?] > at > org.apache.metron.parsers.bolt.WriterHandler.write(WriterHandler.java:90) > [stormjar.jar:?] > at > org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:269) > [stormjar.jar:?] > at > org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735) > [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] > at > org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466) > [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] > at > org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40) > [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] > at > org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472) > [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] > at > org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451) > [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] > at > org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) > [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] > at > org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855) > [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] > at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484) > [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] > at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] > Caused by: org.apache.kafka.common.errors.TimeoutException: Failed to update > metadata after 60000 ms. > > > > On Wed, 4 Sep, 2019, 3:15 PM Hema malini, <[email protected]> wrote: > >> 9092 for bootstrap server . Node1 is the example hostname. Kafka port is >> 6667. Can you please let me know what shouldbe configured for listeners for >> three node. >> Thanks and regards, >> Hema >> On Wed, 4 Sep, 2019, 1:45 PM Simon Elliston Ball, < >> [email protected]> wrote: >> >>> The default port for Kafka in an HDP install is 6667, not 9092. Also >>> node1 is the full dev Kafka. You will need to provide a correct >>> bootstrap-server setting for your brokers. >>> >>> Simon >>> >>> On Wed, 4 Sep 2019 at 09:12, Hema malini <[email protected]> >>> wrote: >>> >>>> Hi, >>>> I installed using Hdp and managing Kafka using ambari. >>>> >>>> I gave the command from the node as >>>> bin/kafka-console-consumer.sh --bootstrap-server node1:9092 --topic bro >>>> --from-beginning >>>> >>>> I am getting warning as >>>> Connection to node -1 could not be established. Broker may not be >>>> available. (Org.apache.kafka.clients.NetworkClient) >>>> >>>> Error as below >>>> kafka.common.NoReplicaOnlineException: No replica in ISR for partition >>>> __consumer_offsets-2 is alive. Live brokers are: [Set(1002)], ISR brokers >>>> are: [1001] >>>> at >>>> kafka.controller.OfflinePartitionLeaderSelector.selectLeader(PartitionLeaderSelector.scala:65) >>>> at >>>> kafka.controller.PartitionStateMachine.electLeaderForPartition(PartitionStateMachine.scala:303) >>>> at >>>> kafka.controller.PartitionStateMachine.kafka$controller$PartitionStateMachine$$handleStateChange(PartitionStateMachine.scala:163) >>>> at >>>> kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:84) >>>> at >>>> kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:81) >>>> at >>>> scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733) >>>> at scala.collection.mutable.HashMap$$anonfun$foreach$1 >>>> .apply(HashMap.scala:130) >>>> at scala.collection.mutable.HashMap$$anonfun$foreach$1 >>>> .apply(HashMap.scala:130) >>>> at >>>> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:236) >>>> at >>>> scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40) >>>> at scala.collection.mutable.HashMap.foreach(HashMap.scala:130) >>>> at >>>> scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732) >>>> at >>>> kafka.controller.PartitionStateMachine.triggerOnlinePartitionStateChange(PartitionStateMachine.scala:81) >>>> at >>>> kafka.controller.KafkaController.onBrokerStartup(KafkaController.scala:402) >>>> at >>>> kafka.controller.KafkaController$BrokerChange.process(KafkaController.scala:1226) >>>> at >>>> kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply$mcV$sp(ControllerEventManager.scala:53) >>>> at >>>> kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply(ControllerEventManager.scala:53) >>>> at >>>> kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply(ControllerEventManager.scala:53) >>>> at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:31) >>>> at >>>> kafka.controller.ControllerEventManager$ControllerEventThread.doWork(ControllerEventManager.scala:52) >>>> at >>>> kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64) >>>> For multinode,(3 node) , what should I configure in ambari console >>>> Kafka config for >>>> 1.listeners ( do I need to give ip of three nodes) >>>> >>>> Asofnow for three nodes, individually changed the property in each node >>>> and started Kafka from console using >>>> bin/kafka-server-start.sh >>>> How can I manage from ambari console. >>>> >>>> Thanks and Regards, >>>> Hema >>>> >>>> >>>> On Tue, 3 Sep, 2019, 11:05 PM James Sirota, <[email protected]> wrote: >>>> >>>>> + 1 to what Mike said. Also, if you could attach any kafka logs that >>>>> contain any error messages that would be helpful >>>>> >>>>> >>>>> 03.09.2019, 08:42, "Michael Miklavcic" <[email protected]>: >>>>> >>>>> Hi Hema, >>>>> >>>>> A couple Q's for you to help narrow this down: >>>>> >>>>> 1. How did you got about installing Kafka and the rest of your >>>>> Hadoop cluster? Is it an HDP installation managed by Ambari? >>>>> 2. Please copy/paste the exact commands you're running to >>>>> produce/consume messages to Kafka >>>>> 3. Full stack trace of any errors you encounter. >>>>> >>>>> If you're using Ambari, this should be fully managed for you. It looks >>>>> like you may have installed Kafka manually? e.g. >>>>> https://kafka.apache.org/quickstart#quickstart_multibroker >>>>> >>>>> Best, >>>>> Mike >>>>> >>>>> >>>>> On Tue, Sep 3, 2019 at 8:39 AM Hema malini <[email protected]> >>>>> wrote: >>>>> >>>>> I am able to send messages when I configure listeners properties to >>>>> single node in Kafka ( for each node,changed the listener property to that >>>>> host name) and then restarted Kafka from command prompt. How can I manage >>>>> the same using ambari. >>>>> >>>>> Thanks, >>>>> Hema >>>>> >>>>> On Tue, 3 Sep, 2019, 7:04 PM Hema malini, <[email protected]> >>>>> wrote: >>>>> >>>>> Also, I am able to create topic and see the topics being created. >>>>> Facing issues while consuming the messages. >>>>> >>>>> On Tue, 3 Sep, 2019, 7:00 PM Hema malini, <[email protected]> >>>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> I have installed Metron 0.7.2 in three node cluster set up. When >>>>> running Kafka consumer in the command prompt getting error as "connection >>>>> to node -1 could not be established.broker may not be available". What >>>>> should I configure listeners properties in server.properties file.what are >>>>> all the other properties needs to be changed. please help to fix the Kafka >>>>> issue. >>>>> >>>>> Thanks and regards, >>>>> Hema >>>>> >>>>> >>>>> >>>>> ------------------- >>>>> Thank you, >>>>> >>>>> James Sirota >>>>> PMC- Apache Metron >>>>> jsirota AT apache DOT org >>>>> >>>>> -- >>> -- >>> simon elliston ball >>> @sireb >>> >>
