[
https://issues.apache.org/jira/browse/KAFKA-665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jay Kreps resolved KAFKA-665.
-----------------------------
Resolution: Fixed
We solved this by making the response queue unbounded. When the request queue
is full the network thread will block until some space frees up, but there is
no deadlock here, processors can still make progress. We do need to block the
processors to avoid continuing to read and enqueue an unbounded number of new
requests. You could imagine being smarter about this but I don't think the
current situation is too bad.
> Outgoing responses delayed on a busy Kafka broker
> --------------------------------------------------
>
> Key: KAFKA-665
> URL: https://issues.apache.org/jira/browse/KAFKA-665
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Neha Narkhede
> Priority: Critical
> Labels: replication-performance
> Fix For: 0.9.0
>
>
> In a long running test, I observed that after a few hours of operation, few
> requests start timing out, mainly because they spent very long time sitting
> in the response queue -
> [2012-12-07 22:05:56,670] TRACE Completed request with correlation id 3965966
> and client : TopicMetadataRequest:4009, queueTime:1, localTime:28,
> remoteTime:0, sendTime:3980 (kafka.network.RequestChannel$)
> [2012-12-07 22:04:12,046] TRACE Completed request with correlation id 3962561
> and client : TopicMetadataRequest:3449, queueTime:0, localTime:29,
> remoteTime:0, sendTime:3420 (kafka.network.RequestChannel$)
> [2012-12-07 22:05:56,670] TRACE Completed request with correlation id 3965966
> and client : TopicMetadataRequest:4009, queueTime:1, localTime:28,
> remoteTime:0, sendTime:3980 (kafka.network.RequestChannel$)
> We might have a problem in the way we process outgoing responses. Basically,
> if the processor thread blocks on enqueuing requests in the request queue, it
> doesn't come around to processing its responses which are ready to go out.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
