[ https://issues.apache.org/jira/browse/STORM-1674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371661#comment-15371661 ]
ASF GitHub Bot commented on STORM-1674: --------------------------------------- Github user moesol closed the pull request at: https://github.com/apache/storm/pull/1287 > Idle KafkaSpout consumes more bandwidth than needed > --------------------------------------------------- > > Key: STORM-1674 > URL: https://issues.apache.org/jira/browse/STORM-1674 > Project: Apache Storm > Issue Type: Bug > Components: storm-kafka > Affects Versions: 0.9.3, 0.10.1, 1.0.1 > Reporter: Robert Hastings > Assignee: Robert Hastings > Fix For: 2.0.0, 1.0.2, 1.1.0 > > > Discovered 30 megabits of traffic flowing between a set of KafkaSpouts > and our kafka servers even though no Kafka messages were moving. > Using the wireshark kafka dissector, we were able to see that > each FetchRequest had maxWait set to 10000 > and minBytes set to 0. When binBytes is set to 0 the kafka server > responds immediately when there are no messages. In turn the KafkaSpout > polls without any delay causing a constant stream of FetchRequest/ > FetchResponse messages. Using a non-KafkaSpout client had a similar > traffic pattern with two key differences > 1) minBytes was 1 > 2) maxWait was 100 > With these FetchRequest parameters and no messages flowing, > the kafka server delays the FetchResponse by 100 ms. This reduces > the network traffic from megabits to the low kilobits. It also > reduced the CPU utilization of our kafka server from 140% to 2%. -- This message was sent by Atlassian JIRA (v6.3.4#6332)