Re: Fixing Wrecked Kafka Topic Partitions after a cluster reconfigure

2018-02-01 Thread Traiano Welcome
Hi Michal Many thanks for the detailed advice. I've tested this on a dev cluster and indeed the data is recoverable by reverting. On Thu, Feb 1, 2018 at 10:11 PM, Michal Michalski < michal.michal...@zalando.ie> wrote: > Hey, > > Changing broker IDs made existing data in ZK irrelevant - since

Re: Fixing Wrecked Kafka Topic Partitions after a cluster reconfigure

2018-02-01 Thread Michal Michalski
Hey, Changing broker IDs made existing data in ZK irrelevant - since ZK uses broker ID to identify brokers and "associate" them with the partition placement (where the data is for each partition), it expects your data to be on the "old" brokers, while the ones that are connecting now are

Fixing Wrecked Kafka Topic Partitions after a cluster reconfigure

2018-02-01 Thread Traiano Welcome
Hi all, I reconfigured my kafka cluster, changing: - default replication factor from 1 to 3 and also - changing the location of the kafka data dir on disk So after restarting all nodes, the cluster seemed ok but then I noticed all the topics are failing to come online. In the logs there are