[
https://issues.apache.org/jira/browse/STORM-1674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371661#comment-15371661
]
ASF GitHub Bot commented on STORM-1674:
---------------------------------------
Github user moesol closed the pull request at:
https://github.com/apache/storm/pull/1287
> Idle KafkaSpout consumes more bandwidth than needed
> ---------------------------------------------------
>
> Key: STORM-1674
> URL: https://issues.apache.org/jira/browse/STORM-1674
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-kafka
> Affects Versions: 0.9.3, 0.10.1, 1.0.1
> Reporter: Robert Hastings
> Assignee: Robert Hastings
> Fix For: 2.0.0, 1.0.2, 1.1.0
>
>
> Discovered 30 megabits of traffic flowing between a set of KafkaSpouts
> and our kafka servers even though no Kafka messages were moving.
> Using the wireshark kafka dissector, we were able to see that
> each FetchRequest had maxWait set to 10000
> and minBytes set to 0. When binBytes is set to 0 the kafka server
> responds immediately when there are no messages. In turn the KafkaSpout
> polls without any delay causing a constant stream of FetchRequest/
> FetchResponse messages. Using a non-KafkaSpout client had a similar
> traffic pattern with two key differences
> 1) minBytes was 1
> 2) maxWait was 100
> With these FetchRequest parameters and no messages flowing,
> the kafka server delays the FetchResponse by 100 ms. This reduces
> the network traffic from megabits to the low kilobits. It also
> reduced the CPU utilization of our kafka server from 140% to 2%.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
