anand-h-codes opened a new pull request, #3679: URL: https://github.com/apache/storm/pull/3679
## What is the purpose of the change Currently 'KafkaTridentTransactionalSpoutEmitter' and 'KafkaTridentOpaqueEmitter' polls every partition assigned to the spout one by one while emitting new batches. But this can be improved by leveraging Kafka Consumer's usual polling strategy. That is Kafka Broker will take care of choosing the right partition. Advantages of this are 1. If a spout is assigned multiple Topic Partitions, the consumer doesn't have to waste time on polling partitions with no data. 2. This change will give better control over the Trident batch size, by adjusting Kafka Consumer properties. Note: This change will affect only when the batch is emitted for the first time. ## How was the change tested Tested by running a Kafka Trident topology locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@storm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
