Hi,

I have done some experiments and gone through kafka documentation, which makes 
me conclude that there is a small chance of data loss or availability in a rack 
scenario. Can someone please validate my understanding ?

The minimum configuration for a single rack system against single machine 
failure is Replication Factor = 3, min.insync.replicas=2, ack=all. This will 
ensure that leader + at least one replica receives the data written by a 
producer and there will be no data loss as well as the system continues to be 
available for further writes by the producer when a broker goes down.

With rack awareness enabled, Kafka will distribute replicas of a partition 
across racks, giving reliability in case of rack failure. However rack 
awareness is only concerned with distribution of replicas, not prioritising the 
order of replication when followers catch up with the leader.

Moving to a rack aware setup which has 2 racks, the above configuration would 
create a problem because one of the racks might get 2 replicas and if that rack 
goes down, data will be lost.

Extending the minimum configuration for a 2 rack setup, Replication Factor = 4, 
min.insync.replicas=2, ack=all. This will ensure that when a rack goes down, 
one of the replicas will be available as it would be on a different rack than 
the leader. This was my understanding and I cannot find any documentation to 
back this. I studied the mechanism by which producer writes to leader - all IN 
SYNC REPLICAS (ISR) pull the newest data, and if the leader confirms that at 
least min.insync.replicas have got the newest data, it sends an ack back to the 
producer. In a rack aware system, I think Kafka will send an ack even if the 2 
replicas which are in sync are on the same rack. And at this instant if that 
rack goes down, data is lost.

If we make min.insync.replicas=3, we can guarantee that one of the replicas 
will be on a different rack and data will not be lost. However if any rack goes 
down, producer’s writes will start failing as it won’t have the requisite 
replicas available.

Is my understanding correct ? Is there a way to configure Kafka in a rack 
scenario to make it tolerant to data loss as well as make it available for 
further writes even when a single node or an entire rack goes down ?

Regards,
Sanjay

Reply via email to