[
https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702030#comment-14702030
]
Alex Etling commented on KAFKA-1070:
------------------------------------
Hello Kafka Geniuses,
I have a question regarding this PR. Some context:
We have started playing around with kafka 0.8.2.1 to build a data pipeline
at the company I work at. Initially, to get autoscaling with AWS working, we
were mapping the IP of our boxes to our broker id. For a while everything was
good. Today I realized (please correct me if I am wrong) that kafka assigns
the replicas to a topic at topic creation time. These replicas are not modified
later unless you specifically rebalance the cluster(this is different than ISR
which can go from 0 servers to the set of replicas). This leads to an
interesting question on how to cycle in new boxes. The easiest way seems to be
to copy all data from one box to another, kill the old box and start the new
box with the same broker.id. This is not really easy when you do a direct
mapping of IP -> broker.id.
So now we come to this Jira ticket. I was wondering if you could
enumerate for me how this auto-assign node id would deal with the cycling of a
box. If a bring down a box that was auto-assigned a broker.id of X and bring
back up a new box, what will happen. Will that new box have broker.id X as
well? What if I bring down two boxes with broker.id X and broker.id Y, what is
the broker.id of the new box i spin up.
Thanks for the help,
Alex
> Auto-assign node id
> -------------------
>
> Key: KAFKA-1070
> URL: https://issues.apache.org/jira/browse/KAFKA-1070
> Project: Kafka
> Issue Type: Bug
> Reporter: Jay Kreps
> Assignee: Sriharsha Chintalapani
> Labels: usability
> Fix For: 0.8.3
>
> Attachments: KAFKA-1070.patch, KAFKA-1070_2014-07-19_16:06:13.patch,
> KAFKA-1070_2014-07-22_11:34:18.patch, KAFKA-1070_2014-07-24_20:58:17.patch,
> KAFKA-1070_2014-07-24_21:05:33.patch, KAFKA-1070_2014-08-21_10:26:20.patch,
> KAFKA-1070_2014-11-20_10:50:04.patch, KAFKA-1070_2014-11-25_20:29:37.patch,
> KAFKA-1070_2015-01-01_17:39:30.patch, KAFKA-1070_2015-01-12_10:46:54.patch,
> KAFKA-1070_2015-01-12_18:30:17.patch
>
>
> It would be nice to have Kafka brokers auto-assign node ids rather than
> having that be a configuration. Having a configuration is irritating because
> (1) you have to generate a custom config for each broker and (2) even though
> it is in configuration, changing the node id can cause all kinds of bad
> things to happen.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
