[
https://issues.apache.org/jira/browse/KAFKA-48?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200159#comment-13200159
]
Jay Kreps commented on KAFKA-48:
--------------------------------
Two other issues with this patch, I forgot to mention:
- There is a race condition between checking the available bytes, and adding
the watchers for the topics. I *think* this is okay since the min_bytes is a
minimum not a maximum, so in the rare case that a produce comes in before the
watchers are added we will just wait slightly longer than we should have. I
think this is probably better than properly synchronizing and locking out all
produces on that partition.
- The other issues is that the delay queue is only emptied right now when the
delay expires. If the request is fulfilled before the delay expires, the
request is marked completed, but it remains in the delay queue until it
expires. This is a problem and needs to be fixed. The problem is that if the
client sets a low min_bytes and a high max_wait these requests may accumulate.
Currently we would have to do an O(N) walk of the waiting requests to fix this.
I am going to try to come up with an improved set of data structures to fix
this without requiring that.
> Implement optional "long poll" support in fetch request
> -------------------------------------------------------
>
> Key: KAFKA-48
> URL: https://issues.apache.org/jira/browse/KAFKA-48
> Project: Kafka
> Issue Type: Bug
> Reporter: Alan Cabrera
> Assignee: Jay Kreps
> Attachments: KAFKA-48.patch
>
>
> Currently, the fetch request is non-blocking. If there is nothing on the
> broker for the consumer to retrieve, the broker simply returns an empty set
> to the consumer. This can be inefficient, if you want to ensure low-latency
> because you keep polling over and over. We should make a blocking version of
> the fetch request so that the fetch request is not returned until the broker
> has at least one message for the fetcher or some timeout passes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
