Jason Gustafson created KAFKA-9952:
--------------------------------------
Summary: Reconsider immediate propagation of hw when replica
selector is used
Key: KAFKA-9952
URL: https://issues.apache.org/jira/browse/KAFKA-9952
Project: Kafka
Issue Type: Improvement
Reporter: Jason Gustafson
For KIP-392, we added logic to make sure that high watermark changes are
propagated to followers without delay in order to improve end to end latency
when fetching from followers. The downside of this change is that it increases
the rate of fetch requests from followers which can have a noticeable impact on
performance (see KAFKA-9731). To fix that problem, we modified the code so that
we only propagate high watermark changes immediately when a replica selector is
used (which is not the default). However, we should consider whether we want to
go even further and disable immediate propagation of the high watermark more
generally since it makes this feature risky to use. Instead, users can use the
max wait time in order to control the worst-case latency. Note that this is
typically only a problem for low-throughput topics since otherwise we will have
a steady rate of high watermark updates.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
