[ https://issues.apache.org/jira/browse/STORM-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jungtaek Lim closed STORM-1471. ------------------------------- Resolution: Not A Problem > Make FetchRequestBuilder.minBytes a configurable parameter in SpoutConfig > ------------------------------------------------------------------------- > > Key: STORM-1471 > URL: https://issues.apache.org/jira/browse/STORM-1471 > Project: Apache Storm > Issue Type: Bug > Components: storm-kafka > Reporter: Garrick Dasbach > > We currently have an issue in our storm cluster where our Kafka brokers are > under heavy load due to too many fetch requests from storm. We've narrowed > the problem to the way Fetch Requests are build in KafkaUtils. When using > the FetchRequestBuilder, storm provides overrides for all the properties > except minBytes. The default for that field is 0 (even though the Kafka > default for the high-level consumer is 1). When paired with a maxWait > 0, > this creates a situation where the broker can immediately return a response > without waiting (due to minBytes 0). This puts a heavy load on the brokers > and defeats the purpose of any long polling. > By making this a SpoutConfig option, it will allow the user to set that as > appropriate for their situation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)