The default port for Kafka in an HDP install is 6667, not 9092. Also node1 is the full dev Kafka. You will need to provide a correct bootstrap-server setting for your brokers.
Simon On Wed, 4 Sep 2019 at 09:12, Hema malini <[email protected]> wrote: > Hi, > I installed using Hdp and managing Kafka using ambari. > > I gave the command from the node as > bin/kafka-console-consumer.sh --bootstrap-server node1:9092 --topic bro > --from-beginning > > I am getting warning as > Connection to node -1 could not be established. Broker may not be > available. (Org.apache.kafka.clients.NetworkClient) > > Error as below > kafka.common.NoReplicaOnlineException: No replica in ISR for partition > __consumer_offsets-2 is alive. Live brokers are: [Set(1002)], ISR brokers > are: [1001] > at > kafka.controller.OfflinePartitionLeaderSelector.selectLeader(PartitionLeaderSelector.scala:65) > at > kafka.controller.PartitionStateMachine.electLeaderForPartition(PartitionStateMachine.scala:303) > at > kafka.controller.PartitionStateMachine.kafka$controller$PartitionStateMachine$$handleStateChange(PartitionStateMachine.scala:163) > at > kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:84) > at > kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:81) > at > scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733) > at scala.collection.mutable.HashMap$$anonfun$foreach$1 > .apply(HashMap.scala:130) > at scala.collection.mutable.HashMap$$anonfun$foreach$1 > .apply(HashMap.scala:130) > at > scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:236) > at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40) > at scala.collection.mutable.HashMap.foreach(HashMap.scala:130) > at > scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732) > at > kafka.controller.PartitionStateMachine.triggerOnlinePartitionStateChange(PartitionStateMachine.scala:81) > at > kafka.controller.KafkaController.onBrokerStartup(KafkaController.scala:402) > at > kafka.controller.KafkaController$BrokerChange.process(KafkaController.scala:1226) > at > kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply$mcV$sp(ControllerEventManager.scala:53) > at > kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply(ControllerEventManager.scala:53) > at > kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply(ControllerEventManager.scala:53) > at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:31) > at > kafka.controller.ControllerEventManager$ControllerEventThread.doWork(ControllerEventManager.scala:52) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64) > For multinode,(3 node) , what should I configure in ambari console Kafka > config for > 1.listeners ( do I need to give ip of three nodes) > > Asofnow for three nodes, individually changed the property in each node > and started Kafka from console using > bin/kafka-server-start.sh > How can I manage from ambari console. > > Thanks and Regards, > Hema > > > On Tue, 3 Sep, 2019, 11:05 PM James Sirota, <[email protected]> wrote: > >> + 1 to what Mike said. Also, if you could attach any kafka logs that >> contain any error messages that would be helpful >> >> >> 03.09.2019, 08:42, "Michael Miklavcic" <[email protected]>: >> >> Hi Hema, >> >> A couple Q's for you to help narrow this down: >> >> 1. How did you got about installing Kafka and the rest of your Hadoop >> cluster? Is it an HDP installation managed by Ambari? >> 2. Please copy/paste the exact commands you're running to >> produce/consume messages to Kafka >> 3. Full stack trace of any errors you encounter. >> >> If you're using Ambari, this should be fully managed for you. It looks >> like you may have installed Kafka manually? e.g. >> https://kafka.apache.org/quickstart#quickstart_multibroker >> >> Best, >> Mike >> >> >> On Tue, Sep 3, 2019 at 8:39 AM Hema malini <[email protected]> >> wrote: >> >> I am able to send messages when I configure listeners properties to >> single node in Kafka ( for each node,changed the listener property to that >> host name) and then restarted Kafka from command prompt. How can I manage >> the same using ambari. >> >> Thanks, >> Hema >> >> On Tue, 3 Sep, 2019, 7:04 PM Hema malini, <[email protected]> >> wrote: >> >> Also, I am able to create topic and see the topics being created. Facing >> issues while consuming the messages. >> >> On Tue, 3 Sep, 2019, 7:00 PM Hema malini, <[email protected]> >> wrote: >> >> Hi, >> >> I have installed Metron 0.7.2 in three node cluster set up. When running >> Kafka consumer in the command prompt getting error as "connection to node >> -1 could not be established.broker may not be available". What should I >> configure listeners properties in server.properties file.what are all the >> other properties needs to be changed. please help to fix the Kafka issue. >> >> Thanks and regards, >> Hema >> >> >> >> ------------------- >> Thank you, >> >> James Sirota >> PMC- Apache Metron >> jsirota AT apache DOT org >> >> -- -- simon elliston ball @sireb
