I agree that KAFKA-1070 would be great to get in. I especially felt the need for something like this while using a few other systems that automated the port, id etc to give a good OOTB experience. Sorry, I lost track of the review. Will do so in the next few days.
Thanks, Neha On Mon, Nov 3, 2014 at 3:56 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > I agree it would be really nice to get KAFKA-1070 figured out. > > FWIW, the reason for having a name or id other than ip was to make it > possible to move the identity to another physical server (e.g. scp the data > directory) and have it perform the same role on that new piece of hardware. > Systems that tie the data to a particular host tend to be sort of hated on > since you can't do anything simple/stupid to back them up or replace them. > > -Jay > > On Mon, Nov 3, 2014 at 2:23 PM, Gwen Shapira <gshap...@cloudera.com> > wrote: > > > +1 > > Thats what we use to generate broker id in automatic deployments. > > This method makes troubleshooting easier (you know where each broker is > > running), and doesn't require keeping extra files around. > > > > On Mon, Nov 3, 2014 at 2:17 PM, Joe Stein <joe.st...@stealth.ly> wrote: > > > > > Most folks strip the IP and use that as the broker.id. KAFKA-1070 does > > not > > > yet accommodate for that very widely used method. I think it would be > bad > > > if KAFKA-1070 only worked for new installations because that is how > > people > > > use Kafka today (per > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-1070?focusedCommentId=14085808&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14085808 > > > ) > > > > > > On Mon, Nov 3, 2014 at 2:12 PM, Joel Koshy <jjkosh...@gmail.com> > wrote: > > > > > > > KAFKA-1070 will help with this and is pending a review. > > > > > > > > On Mon, Nov 03, 2014 at 05:03:20PM -0500, Otis Gospodnetic wrote: > > > > > Hi, > > > > > > > > > > How do people handle situations, and specifically the broker.id > > > > property, > > > > > where the Kafka (broker) cluster is not fully defined right away? > > > > > > > > > > Here's the use case we have at Sematext: > > > > > * Our software ships as a VM > > > > > * All components run in this single VM, including 1 Kafka broker > > > > > * Of course, this is just for a nice OOTB experience, but to scale > > one > > > > > needs to have more instances of this VM, including more Kafka > brokers > > > > > * *One can clone our VM and launch N instances of it, but because > we > > > > have a > > > > > single Kafka broker config with a single broker.id < > http://broker.id > > > > > > in > > > > > it, one can't just launch more of these VMs and expect to see more > > > Kafka > > > > > brokers join the cluster. One would have to change the broker.id > > > > > <http://broker.id> on each new VM instance.* > > > > > > > > > > How do others handle this in a software that is packages and ships > to > > > > user > > > > > and is not in your direct control to allow you to edit configs? > > > > > > > > > > Would it be best to have a script that connect to ZooKeeper to get > > the > > > > list > > > > > of all existing brokers and their IDs and then generate a new > > distinct > > > > ID + > > > > > config for the new Kafka broker? > > > > > > > > > > Or are there slicker ways to do this that people use? > > > > > > > > > > Thanks, > > > > > Otis > > > > > -- > > > > > Monitoring * Alerting * Anomaly Detection * Centralized Log > > Management > > > > > Solr & Elasticsearch Support * http://sematext.com/ > > > > > > > > > > > > > >
