[
https://issues.apache.org/jira/browse/KAFKA-48?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154069#comment-13154069
]
Taylor Gautier commented on KAFKA-48:
-------------------------------------
I've been staring at the code for a while - and I'm not sure I understand why
you need KAFKA-202 to implement this feature.
What I am thinking to do is:
1) Every thread has to open a local socket for read/write
2) Each thread puts the socket into the poll set for reading
3) If a read request fails to read any messages, when it comes back to the
handler, the handler adds a callback method to the appropriate log and puts the
read request into a special queue. When that log gets messages for write, it
calls the callback. The callback writes a byte into the special thread socket.
4) The byte wakes up the thread, which sees that the special socket had a byte
written to it, and so it goes and re-handles the read requests in the special
queue as if they had just come in from the network. Thus if there are any
messages available in the log for a given request, they are read just like
normal and transferred out onto the channel. If not, they're re-queued as per
step 3.
I think there is some pieces I haven't quite got right - in particular, I think
there can only be one active response at a time. Thus there will have to be
some sort of response queue built up as each request generates a response, but
I think that's simple - the handler just writes responses with non-zero
messages into a response queue and the write logic of the socketserver is
updated to drain this queue on write events (at the moment, it only deals with
one response at a time, but now it may have many to send out queued up).
Some other work that is probably going to be more difficult is that the binary
protocol has to change to include the topic name or else there is no way to
disambiguate the responses coming back.
> Implement optional "long poll" support in fetch request
> -------------------------------------------------------
>
> Key: KAFKA-48
> URL: https://issues.apache.org/jira/browse/KAFKA-48
> Project: Kafka
> Issue Type: Bug
> Reporter: Alan Cabrera
> Assignee: Jay Kreps
>
> Currently, the fetch request is non-blocking. If there is nothing on the
> broker for the consumer to retrieve, the broker simply returns an empty set
> to the consumer. This can be inefficient, if you want to ensure low-latency
> because you keep polling over and over. We should make a blocking version of
> the fetch request so that the fetch request is not returned until the broker
> has at least one message for the fetcher or some timeout passes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
