[
https://issues.apache.org/jira/browse/STORM-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097475#comment-15097475
]
Garrick Dasbach commented on STORM-1471:
----------------------------------------
That's a fair point, however it's somewhat confusing then that the KafkaConfig
allows you to set fetchMaxWait, which defaults to 10s.
> Make FetchRequestBuilder.minBytes a configurable parameter in SpoutConfig
> -------------------------------------------------------------------------
>
> Key: STORM-1471
> URL: https://issues.apache.org/jira/browse/STORM-1471
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-kafka
> Reporter: Garrick Dasbach
>
> We currently have an issue in our storm cluster where our Kafka brokers are
> under heavy load due to too many fetch requests from storm. We've narrowed
> the problem to the way Fetch Requests are build in KafkaUtils. When using
> the FetchRequestBuilder, storm provides overrides for all the properties
> except minBytes. The default for that field is 0 (even though the Kafka
> default for the high-level consumer is 1). When paired with a maxWait > 0,
> this creates a situation where the broker can immediately return a response
> without waiting (due to minBytes 0). This puts a heavy load on the brokers
> and defeats the purpose of any long polling.
> By making this a SpoutConfig option, it will allow the user to set that as
> appropriate for their situation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
