[
https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085808#comment-14085808
]
Joe Stein edited comment on KAFKA-1070 at 8/5/14 5:08 AM:
----------------------------------------------------------
This sounds like a very cool and VERY useful feature. Excited to use it myself
often.
I know of a few (>10) different clusters that not only use varying sized
numbers for their broker.id but do so in what is a seemingly random (but not
really when you think about it) way.
so in a cluster there may be broker.id that is 1721632 and another 172164875
and another 172162240 . Making your brokers by replacing "." in
chef/puppet/salt/ansemble/etc type scripts and sometimes folks get more fancy
just doing 2288, 2388, 17, 177 (where just the last two octets get used and "."
is replaced).
I am not saying I agree with this approach and I actively advocate away from
doing this but in some/many scenarios it is the best/only way to automate their
deploys for how things are setup. It is also what seems to make sense when
folks are building their automation scripts since they have no other option
without doing more than they should be expected to-do (and the IP replace "."
is so obvious to them, and it is).
So, for folks in these cases they would just pick the upper bound to be, lets
say 17216255256 and then it would auto assign from there?
Is there some better way to go about this where you might have a start
increment and and some exclusion list? or a way to see broker.id already had
that and not use it? I think a lot of folks would like to get having broker id
be more continious and be easier to communicate but the desire to automate
everything will outweigh that. We could give them some sanity back with
brokers 1,2,3,4,5,6,7,8,9,10,11,12 for a 12 node cluster.
not crucial and you may have already accounted for the scenario I brought up,
but wanted to bring it up as a real use case for how people automate things.
it might be better for folks to manually migrate their scripts but not sure
they would do it and if they did would have to decommission brokers which in a
prod environment could take a few weeks/months. If we let them start at 1 and
exclude what they have then they can do it one at a time. After taking down
the first broker and bring it back up (empty) it is broker.id=1, and so on (and
if they have a 5 they don't have to take it down), etc.
For new clusters this is a slam dunk and wouldn't want to hold up the feature
for existing users that have already decided a work around as I don't know what
the intent of this was or not. Some folks might not change knowing
broker.id=17216520 sometimes is nice you just login to that box but talking
about broker 17216520 over and over again is a pita.
was (Author: joestein):
This sounds like a very cool and VERY useful feature. Excited to use it myself
often.
I know of a few (>10) different clusters that not only use varying sized
numbers for their broker.id but do so in what is a seemingly random (but not
really when you think about it) way.
so in a cluster there may be broker.id that is 1721632 and another 172164875
and another 172162240 . Making your brokers by replacing "." in
chef/puppet/salt/ansemble/etc type scripts and sometimes folks get more fancy
just doing 2288, 2388, 17, 177 (where just the last two octets get used and "."
is replaced).
I am not saying I agree with this approach and I actively advocate away from
doing this but in some/many scenarios it is the best/only way to automate their
deploys for how things are setup. It is also what seems to make sense when
folks are building their automation scripts since they have no other option
without doing more than they should be expected to-do (and the IP replace "."
is so obvious to them, and it is).
So, for folks in these cases they would just pick the upper bound to be, lets
say 17216255256 and then it would auto assign from there?
Is there some better way to go about this where you might have a start
increment and and some exclusion list? or a way to see broker.id already had
that and not use it? I think a lot of folks would like to get having broker id
be more continious and be easier to communicate but the desire to automate
everything will outweigh that. We could give them some sanity back with
brokers 1,2,3,4,5,6,7,8,9,10,11,12 for a 12 node cluster.
not crucial and you may have already accounted for the scenario I brought up,
but wanted to bring it up as a real use case for how people automate things.
it might be better for folks to manually migrate their scripts but not sure
they would do it and if they did would have to decommission brokers which in a
prod environment could take a few weeks/months. If we let them start at 1 and
exclude what they have then they can do it one at a time. After taking down
the first broker and bring it back up (empty) it is broker.id=1, and so on (and
if they have a 5 they don't have to take it down), etc.
> Auto-assign node id
> -------------------
>
> Key: KAFKA-1070
> URL: https://issues.apache.org/jira/browse/KAFKA-1070
> Project: Kafka
> Issue Type: Bug
> Reporter: Jay Kreps
> Assignee: Sriharsha Chintalapani
> Labels: usability
> Attachments: KAFKA-1070.patch, KAFKA-1070_2014-07-19_16:06:13.patch,
> KAFKA-1070_2014-07-22_11:34:18.patch, KAFKA-1070_2014-07-24_20:58:17.patch,
> KAFKA-1070_2014-07-24_21:05:33.patch
>
>
> It would be nice to have Kafka brokers auto-assign node ids rather than
> having that be a configuration. Having a configuration is irritating because
> (1) you have to generate a custom config for each broker and (2) even though
> it is in configuration, changing the node id can cause all kinds of bad
> things to happen.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
