Working on the above. Since trunk currently allows Topic configs, and altering topics, does it make sense to you to make this a topic config so that it is kept in sync between topic alterations and partition adds?
For compatibility: I can make the zookeeper state parsing backwards / forwards compatible. I don't think I can make the readFrom / writeTo ByteBuffer compatible since chained calls of these during api parsing are size dependent? Can you clarify exactly what you expect to be compatible? I'm also not very familiar with the consequences of changing the broker version in zookeeper registration. Can you please provide a reference to documentation for this? Thanks, Joris On Fri, Jan 24, 2014 at 9:43 AM, Jun Rao (JIRA) <j...@apache.org> wrote: > > [ > https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881193#comment-13881193] > > Jun Rao commented on KAFKA-1215: > -------------------------------- > > Thanks for the patch. Looks good over all. Some comments. > > 1. KafkaConfig: > 1.1 We need a config for default max-rack-replication for auto topic > creation. > 1.2 rackId: We probably don't want to make this a required property. So, > perhaps we can default it to 0? > > 2. AdminUtils.assignReplicasToBrokers(): > 2.1 Could you add some comments on the rack-aware assignment algorithm? > 2.2 It's a bit weird for this method to take zkclient in the input. We > probably can pass in a list of Broker objects instead. > > 3. Unit tests: I suggest that we leave most existing tests intact by > keeping the rackId default and add a new test for rack-aware assignment. > > 4. Compatibility test: It seems that the changes for the broker format in > ZK is backward compatible. Could you double check? For example, an old > reader (controller, consumer, etc) should be able to parse the broker > registered in new format and a new reader should be able to parse the > broker registered in the old format. Also, we probably should increase the > version in the ZK registration for the broker. > > 5. Could you rebase to trunk? > > > Rack-Aware replica assignment option > > ------------------------------------ > > > > Key: KAFKA-1215 > > URL: https://issues.apache.org/jira/browse/KAFKA-1215 > > Project: Kafka > > Issue Type: Improvement > > Components: replication > > Affects Versions: 0.8.0, 0.8.1 > > Reporter: Joris Van Remoortere > > Assignee: Neha Narkhede > > Fix For: 0.8.0, 0.8.1 > > > > Attachments: rack_aware_replica_assignment_v1.patch > > > > > > Adding a rack-id to kafka config. This rack-id can be used during > replica assignment by using the max-rack-replication argument in the admin > scripts (create topic, etc.). By default the original replication > assignment algorithm is used because max-rack-replication defaults to -1. > max-rack-replication > -1 is not honored if you are doing manual replica > assignment (preffered). > > If this looks good I can add some test cases specific to the rack-aware > assignment. > > I can also port this to trunk. We are currently running 0.8.0 in > production and need this, so i wrote the patch against that. > > > > -- > This message was sent by Atlassian JIRA > (v6.1.5#6160) >
