Hi, Thanks very much for the information. In https://github.com/apache/kafka/pull/11942, it is said the fetch request goes to the purgatory and will wait for wait timeout. Does it refer to fetch.max.wait.ms? In our test, we have configured that parameter to 100 ms. If that is the case, do you think that part of 400 ms is explained by that PR?
Regards On Wed, May 11, 2022 at 9:45 AM Luke Chen <show...@gmail.com> wrote: > Hi, > > We have some improvement for the preferred read replica configured case. > Ex: > https://github.com/apache/kafka/pull/11942 > https://github.com/apache/kafka/pull/11965 > > I know one improvement will be included in the v3.2.0 release, which will > be released soon. > Maybe you can give it a try to see if it improves the throughput. > > Thank you. > Luke > > On Wed, May 11, 2022 at 2:56 PM benitocm <benit...@gmail.com> wrote: > > > Hi, > > > > We are using the functionality provided by KIP-392 (a consumer can fetch > > the data from a ISR replica instead of the partition leader) in a Kafka > > cluster stretched between two very close DCs (average round-trip latency > > about 2 milliseconds). > > > > What we have seen is that, on average, when the consumer is in the same > DC > > (configured by rack.id) as the partition leader (i.e. the consumer will > > consume from the leader), the time that takes the message to get to the > > consumer is close to 20 milliseconds. However, when the consumer is in a > > different DC than the partition leader (the consumer will consume from a > > replica that is in the same DC as the consumer) that latency goes to > around > > 400 milliseconds. > > > > We have also checked that if we dont configure the rack.id in a > consumer > > to force it to consume from the leader although the partition leader is > a > > different DC (i.e. the consumer is in DC1 and the partition leader is in > > DC2 so the consumer goes from a DC to the other DC) , the latency is > > reduced to the 20 milliseconds. > > > > From those tests, we have concluded that consuming from a ISR replica > > implies to have higher latencies. > > > > Please does anybody share any thoughts on this? > > > > Thanks in advance > > >
