Daniel, Are you sure about that (it's not what I would have understood).
Generally, the way to do it is use a round-robin dns entry, which returns successive nodes for successive requests. Kafka will retry a zookeeper request on failure (and in the process get re-routed to a new zk node). If a zk node fails, and a new one comes online, then it is necessary to update the dns config of course. If you don't have dns, then you can list your nodes explicitly, but you won't be able to discover new nodes in the cluster if they are added (but kafka will retry if it tries the bad one until it finds a good one). Jason On Mon, May 25, 2015 at 1:51 AM, Achanta Vamsi Subhash < achanta.va...@flipkart.com> wrote: > Oh ok. Got it. Thanks Gwen and Daniel. > > On Mon, May 25, 2015 at 5:15 AM, Daniel Compton < > daniel.compton.li...@gmail.com> wrote: > > > Correct me if I'm wrong, but I thought that the zk connect config was > only > > needed explicitly at startup, and the zk cluster would update the active > > zookeepers as they joined and exited. You only need to specify one zk to > > join the cluster, and it will bootstrap the rest. > > > > So zk changes won't require a restart, but you'll want to make sure your > > config is up to date when you do eventually come to do a restart. > > On Mon, 25 May 2015 at 1:44 am Achanta Vamsi Subhash < > > achanta.va...@flipkart.com> wrote: > > > > > Ok. I was talking about a scenario where there is no DNS/hostNames for > > the > > > zk nodes. > > > > > > If the connection string is given with all ip addresess of zk hosts > and a > > > new host is brought up in the zk cluster replacing a old node with a > > > different ip address, we still need to re-load the zk connection > string. > > > > > > If hostnames are used instead, the DNS mapping can point to the new IP > > but > > > in a scenario where there is no DNS, we need to again hard-code the IP > > > address in the zk connection string and restart the broker. > > > > > > One way is to give the local mapping of the zk local hostname to ip in > > > /etc/hosts file and change it to the new ip when the node changes. But > > > would reload of the Kafka config with new zk nodes be a better option? > > But > > > as you said, if we cannot reload the server.properties file, what is > the > > > best way in case of no service discovery? > > > > > > > > > On Sun, May 24, 2015 at 6:52 PM, Gwen Shapira <gshap...@cloudera.com> > > > wrote: > > > > > > > You can't dynamically re-load server properties. > > > > > > > > However, the norm in zookeeper is to configure the connection string > > with > > > > all the nodes in the zk cluster, so there will be no need to modify > > > > properties when you replace zk nodes. > > > > > > > > On Sun, May 24, 2015 at 4:13 PM, Achanta Vamsi Subhash < > > > > achanta.va...@flipkart.com> wrote: > > > > > > > > > Thanks Gwen. > > > > > > > > > > One more question: > > > > > Is there a way we can dynamically re-load the contents of the Kafka > > > > > server.properties contents without restarting the broker? Example > > > > use-case > > > > > is when a zk-node goes down and a new one is brought up, we need to > > > > update > > > > > the server.properties file to reflect this. Currently there is no > way > > > to > > > > do > > > > > this other than broker restart. Broker restart requires a lot of > > things > > > > to > > > > > do before triggering it. > > > > > > > > > > This JIRA is already filed but un-resolved. We don't require all > the > > > > > configs to be reloaded but only variable external config changes > > should > > > > be > > > > > allowed. > > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-1229 > > > > > > > > > > On Sun, May 24, 2015 at 1:14 PM, Gwen Shapira < > gshap...@cloudera.com > > > > > > > > wrote: > > > > > > > > > > > If you set advertised.hostname in server.properties to the ip > > > address, > > > > > the > > > > > > IP will be registered in ZooKeeper. > > > > > > > > > > > > > > > > > > On Fri, May 22, 2015 at 2:20 PM, Achanta Vamsi Subhash < > > > > > > achanta.va...@flipkart.com> wrote: > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > Currently Kakfa brokers register the hostname in zookeeper. > > > > > > > > > > > > > > [zk: localhost:2181(CONNECTED) 5] get > > /varadhi/kafka/brokers/ids/0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > {"jmx_port":9999,"timestamp":"1427704934158","host":"currHostName","version":1,"port":9092} > > > > > > > > > > > > > > Is there any config to make it use ip-address instead so that > we > > > > don't > > > > > > > make a DNS lookup for the hostname? > > > > > > > > > > > > > > -- > > > > > > > Regards > > > > > > > Vamsi Subhash > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Regards > > > > > Vamsi Subhash > > > > > > > > > > > > > > > > > > > > > -- > > > Regards > > > Vamsi Subhash > > > > > > > > > -- > Regards > Vamsi Subhash >
