Hi Tulio, Thanks a bunch for responding to my question! We basically need to migrate data from brokers hosted on racks being decommisioned, but the new and old brokers are actually within the same cluster. The new brokers are configured to use the new Zookeeper ensemble, but, again, they are intended to be brokers within the same cluster. Once the data is moved to the new brokers, the old brokers will be deleted, or at least that's what I am intending to do.
Please confirm if my approach makes sense or if there is a problem and/or a better way to do it. Thanks --John On Mon, Nov 2, 2015 at 2:20 PM, Tulio Ballari <tulioball...@gmail.com> wrote: > I'm not really sure but I think kafka-reassing-partitions.sh only works to > move partitions between instanses of the same cluster > > If you have two separete clusters, it is not going to work. Are you using > two different zookeeper clusters, or simply adding instance to the existing > one? > > > > On Sun, Nov 1, 2015 at 1:18 PM, John Yost <hokiege...@gmail.com> wrote: > > > Hi Everyone, > > > > I need to migrate my organization's Kafka cluster along with the > underlying > > Zookeeper ensemble (cluster) from one set of racks to another within our > > data center. I am pretty sure I have the steps correct, but I need to > > confirm just to ensure I am not missing anything. > > > > Here's what I think I need to do in the following order: > > > > 1. Configure new Zookeeper ensemble > > 2. Configure brokers for new Kafka cluster, with zookeeper.connect list > of > > new Zookeeper ensemble nodes > > 3. Startup new Zookeeper ensemble > > 4. Startup new Kafka cluster > > 5. Shutdown data feeds to old Kafka cluster and reconfigure for new > cluster > > 6. Migrate topics from old Kafka brokers to new brokers via > > kafka-reassign-partitions.sh script and the instructions here: > > > http://kafka.apache.org/082/documentation.html#basic_ops_cluster_expansion > > 7. Export consumer offsets from old Zookeeper ensemble and import into > new > > Zookeeper ensemble > > 8. Shutdown old Kafka cluster > > 9. Restart data feeds into new Kafka cluster > > > > The Kafka documentation is great and I've tested out the topic > reassignment > > and consumer offset import and export, but, again, just want to ensure I > am > > not missing anything. > > > > Thanks > > > > --John > > >
