Re: Add partitions with replica assignment in same command
If you add partitions, existing partitions' assignment won't change. Changing the replica assignment subsequently should be fine. Thanks, Jun On Wed, Nov 12, 2014 at 2:24 PM, Allen Wang wrote: > I found this JIRA > > https://issues.apache.org/jira/browse/KAFKA-1656 > > Now, we have to use two commands to accomplish the goal - first add > partitions using TopicCommand and then reassign replicas using > ReassignPartitionsCommand. When we first add partitions, will it change the > assignment of replicas for existing partitions? This is what we would like > to avoid. Also, will there be any issues executing the second reassignment > command which will change the assignment again for the new partitions > added? > > > > > On Sun, Nov 9, 2014 at 9:01 PM, Jun Rao wrote: > > > Yes, it seems that we need to fix the tool to support that. It's probably > > more intuitive to have TopicCommand just take the replica-assignment (for > > the new partitions) when altering a topic. Could you file a jira? > > > > Thanks, > > > > Jun > > > > On Fri, Nov 7, 2014 at 4:17 PM, Allen Wang > > wrote: > > > > > I am trying to figure out how to add partitions and assign replicas > using > > > one admin command. I tried kafka.admin.TopicCommand to increase the > > > partition number from 9 to 12 with the following options: > > > > > > /apps/kafka/bin/kafka-run-class.sh kafka.admin.TopicCommand > --zookeeper > > > ${ZOOKEEPER} --alter --topic test_topic_4 --partitions 12 > > > --replica-assignment 2:1,0:2,1:0,1:2,2:0,0:1,1:0,2:1,0:2,2:1,0:2,1:0 > > > > > > This gives me an error > > > > > > Option "[replica-assignment]" can't be used with option"[partitions]" > > > > > > Looking into the TopicCommand, alterTopic function seems to be able to > > > handle that but the command exits with the above error before this > > function > > > is invoked. > > > > > > Is there any workaround or other recommended way to achieve this? > > > > > > Thanks, > > > Allen > > > > > >
Re: Add partitions with replica assignment in same command
When we first add partitions, will it change the assignment of replicas for existing partitions? Nope. It should not touch the existing partitions. Also, will there be any issues executing the second reassignment command which will change the assignment again for the new partitions added? No. 2nd reassignment should work as expected. On Wed, Nov 12, 2014 at 2:24 PM, Allen Wang wrote: > I found this JIRA > > https://issues.apache.org/jira/browse/KAFKA-1656 > > Now, we have to use two commands to accomplish the goal - first add > partitions using TopicCommand and then reassign replicas using > ReassignPartitionsCommand. When we first add partitions, will it change the > assignment of replicas for existing partitions? This is what we would like > to avoid. Also, will there be any issues executing the second reassignment > command which will change the assignment again for the new partitions > added? > > > > > On Sun, Nov 9, 2014 at 9:01 PM, Jun Rao wrote: > > > Yes, it seems that we need to fix the tool to support that. It's probably > > more intuitive to have TopicCommand just take the replica-assignment (for > > the new partitions) when altering a topic. Could you file a jira? > > > > Thanks, > > > > Jun > > > > On Fri, Nov 7, 2014 at 4:17 PM, Allen Wang > > wrote: > > > > > I am trying to figure out how to add partitions and assign replicas > using > > > one admin command. I tried kafka.admin.TopicCommand to increase the > > > partition number from 9 to 12 with the following options: > > > > > > /apps/kafka/bin/kafka-run-class.sh kafka.admin.TopicCommand > --zookeeper > > > ${ZOOKEEPER} --alter --topic test_topic_4 --partitions 12 > > > --replica-assignment 2:1,0:2,1:0,1:2,2:0,0:1,1:0,2:1,0:2,2:1,0:2,1:0 > > > > > > This gives me an error > > > > > > Option "[replica-assignment]" can't be used with option"[partitions]" > > > > > > Looking into the TopicCommand, alterTopic function seems to be able to > > > handle that but the command exits with the above error before this > > function > > > is invoked. > > > > > > Is there any workaround or other recommended way to achieve this? > > > > > > Thanks, > > > Allen > > > > > >
Re: Add partitions with replica assignment in same command
I found this JIRA https://issues.apache.org/jira/browse/KAFKA-1656 Now, we have to use two commands to accomplish the goal - first add partitions using TopicCommand and then reassign replicas using ReassignPartitionsCommand. When we first add partitions, will it change the assignment of replicas for existing partitions? This is what we would like to avoid. Also, will there be any issues executing the second reassignment command which will change the assignment again for the new partitions added? On Sun, Nov 9, 2014 at 9:01 PM, Jun Rao wrote: > Yes, it seems that we need to fix the tool to support that. It's probably > more intuitive to have TopicCommand just take the replica-assignment (for > the new partitions) when altering a topic. Could you file a jira? > > Thanks, > > Jun > > On Fri, Nov 7, 2014 at 4:17 PM, Allen Wang > wrote: > > > I am trying to figure out how to add partitions and assign replicas using > > one admin command. I tried kafka.admin.TopicCommand to increase the > > partition number from 9 to 12 with the following options: > > > > /apps/kafka/bin/kafka-run-class.sh kafka.admin.TopicCommand --zookeeper > > ${ZOOKEEPER} --alter --topic test_topic_4 --partitions 12 > > --replica-assignment 2:1,0:2,1:0,1:2,2:0,0:1,1:0,2:1,0:2,2:1,0:2,1:0 > > > > This gives me an error > > > > Option "[replica-assignment]" can't be used with option"[partitions]" > > > > Looking into the TopicCommand, alterTopic function seems to be able to > > handle that but the command exits with the above error before this > function > > is invoked. > > > > Is there any workaround or other recommended way to achieve this? > > > > Thanks, > > Allen > > >
Re: Add partitions with replica assignment in same command
Yes, it seems that we need to fix the tool to support that. It's probably more intuitive to have TopicCommand just take the replica-assignment (for the new partitions) when altering a topic. Could you file a jira? Thanks, Jun On Fri, Nov 7, 2014 at 4:17 PM, Allen Wang wrote: > I am trying to figure out how to add partitions and assign replicas using > one admin command. I tried kafka.admin.TopicCommand to increase the > partition number from 9 to 12 with the following options: > > /apps/kafka/bin/kafka-run-class.sh kafka.admin.TopicCommand --zookeeper > ${ZOOKEEPER} --alter --topic test_topic_4 --partitions 12 > --replica-assignment 2:1,0:2,1:0,1:2,2:0,0:1,1:0,2:1,0:2,2:1,0:2,1:0 > > This gives me an error > > Option "[replica-assignment]" can't be used with option"[partitions]" > > Looking into the TopicCommand, alterTopic function seems to be able to > handle that but the command exits with the above error before this function > is invoked. > > Is there any workaround or other recommended way to achieve this? > > Thanks, > Allen >
Add partitions with replica assignment in same command
I am trying to figure out how to add partitions and assign replicas using one admin command. I tried kafka.admin.TopicCommand to increase the partition number from 9 to 12 with the following options: /apps/kafka/bin/kafka-run-class.sh kafka.admin.TopicCommand --zookeeper ${ZOOKEEPER} --alter --topic test_topic_4 --partitions 12 --replica-assignment 2:1,0:2,1:0,1:2,2:0,0:1,1:0,2:1,0:2,2:1,0:2,1:0 This gives me an error Option "[replica-assignment]" can't be used with option"[partitions]" Looking into the TopicCommand, alterTopic function seems to be able to handle that but the command exits with the above error before this function is invoked. Is there any workaround or other recommended way to achieve this? Thanks, Allen