Hi, We are using the functionality provided by KIP-392 (a consumer can fetch the data from a ISR replica instead of the partition leader) in a Kafka cluster stretched between two very close DCs (average round-trip latency about 2 milliseconds).
What we have seen is that, on average, when the consumer is in the same DC (configured by rack.id) as the partition leader (i.e. the consumer will consume from the leader), the time that takes the message to get to the consumer is close to 20 milliseconds. However, when the consumer is in a different DC than the partition leader (the consumer will consume from a replica that is in the same DC as the consumer) that latency goes to around 400 milliseconds. We have also checked that if we dont configure the rack.id in a consumer to force it to consume from the leader although the partition leader is a different DC (i.e. the consumer is in DC1 and the partition leader is in DC2 so the consumer goes from a DC to the other DC) , the latency is reduced to the 20 milliseconds. >From those tests, we have concluded that consuming from a ISR replica implies to have higher latencies. Please does anybody share any thoughts on this? Thanks in advance
