[
https://issues.apache.org/jira/browse/STORM-1674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294081#comment-15294081
]
ASF GitHub Bot commented on STORM-1674:
---------------------------------------
Github user ptgoetz commented on the pull request:
https://github.com/apache/storm/pull/1393#issuecomment-220705013
Thanks @moesol I merged this into master, 1.x-branch and 1.0.x-branch.
Note that with 1.0.x-branch this created a conflict, as
`KafkaConfig.minFetchByte` had been introduced to do the same thing as your
`KafkaConfig.fetchMinBytes` variable. These are public variables, so they make
up part of the API.
Since 1.0.1 had already been released with the `minFetchByte` varibable, to
ensure backward compatibility I used the `minFetchByte` variable name across
branches.
While I prefer the variable name you chose, I think it's more important to
ensure backward compatibility. :)
Thanks again!
> Idle KafkaSpout consumes more bandwidth than needed
> ---------------------------------------------------
>
> Key: STORM-1674
> URL: https://issues.apache.org/jira/browse/STORM-1674
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-kafka
> Affects Versions: 0.9.3
> Reporter: Robert Hastings
>
> Discovered 30 megabits of traffic flowing between a set of KafkaSpouts
> and our kafka servers even though no Kafka messages were moving.
> Using the wireshark kafka dissector, we were able to see that
> each FetchRequest had maxWait set to 10000
> and minBytes set to 0. When binBytes is set to 0 the kafka server
> responds immediately when there are no messages. In turn the KafkaSpout
> polls without any delay causing a constant stream of FetchRequest/
> FetchResponse messages. Using a non-KafkaSpout client had a similar
> traffic pattern with two key differences
> 1) minBytes was 1
> 2) maxWait was 100
> With these FetchRequest parameters and no messages flowing,
> the kafka server delays the FetchResponse by 100 ms. This reduces
> the network traffic from megabits to the low kilobits. It also
> reduced the CPU utilization of our kafka server from 140% to 2%.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
