Github user srdo commented on the issue:
https://github.com/apache/storm/pull/1679
@jfenc91 Hope it works for you now. You're right though that the spout
needs to handle unclean leader elections better. It's not great that the
consumer group will remain "ahead" of the LEO and the spout will fail to commit
offsets until it has caught up to where the unclean leader election happened. I
think an option for a solution would be to have the spout check if the current
LEO is behind committedOffset when it hits the else case in
findNextCommitOffsets, and if so, it should reset the consumer group's
committed offset to the either the LEO or the beginning of the partition,
probably depending on a new configuration parameter. It's never going to be a
nice situation to be in, but at least the spout shouldn't break when it
happens, especially since it can happen with the default Kafka configuration.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
