Kamal Chandraprakash created KAFKA-15776:
--------------------------------------------
Summary: Configurable delay timeout for DelayedRemoteFetch request
Key: KAFKA-15776
URL: https://issues.apache.org/jira/browse/KAFKA-15776
Project: Kafka
Issue Type: Task
Reporter: Kamal Chandraprakash
Assignee: Kamal Chandraprakash
We are reusing the {{fetch.max.wait.ms}} config as a delay timeout for
DelayedRemoteFetchPurgatory. {{fetch.max.wait.ms}} purpose is to wait for the
given amount of time when there is no data available to serve the FETCH request.
{code:java}
The maximum amount of time the server will block before answering the fetch
request if there isn't sufficient data to immediately satisfy the requirement
given by fetch.min.bytes.
{code}
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/server/DelayedRemoteFetch.scala#L41
Using the same timeout in the DelayedRemoteFetchPurgatory can confuse the user
on how to configure optimal value for each purpose. Moreover, the config is of
*LOW* importance and most of the users won't configure it and use the default
value of 500 ms.
Having the delay timeout of 500 ms in DelayedRemoteFetchPurgatory can lead to
larger number of expired delayed remote fetch request when the remote storage
have any degradation to serve within the timeout.
We should introduce one config (preferably server config) to define the delay
timeout for DelayedRemoteFetch requests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
