Github user srdo commented on the issue: https://github.com/apache/storm/pull/1679 @jfenc91 Hope it works for you now. You're right though that the spout needs to handle unclean leader elections better. It's not great that the consumer group will remain "ahead" of the LEO and the spout will fail to commit offsets until it has caught up to where the unclean leader election happened. I think an option for a solution would be to have the spout check if the current LEO is behind committedOffset when it hits the else case in findNextCommitOffsets, and if so, it should reset the consumer group's committed offset to the either the LEO or the beginning of the partition, probably depending on a new configuration parameter. It's never going to be a nice situation to be in, but at least the spout shouldn't break when it happens, especially since it can happen with the default Kafka configuration.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---