Github user HeartSaVioR commented on the issue:
https://github.com/apache/storm/pull/1924
+1
Once @hmcl finished the review and no outstanding comments there I'll merge.
Btw, regarding maxUncommittedOffsets, yes it provides more guarantees but
if we guarantee that commit occurs in time periodically (we're providing option
too), IMHO that's enough.
The thing I have in mind is that commit in nextTuple() doesn't strictly
guarantee commit in time. Two situations come in mind: reached max spout
pending, backpressure in place.
If spout reached max spout pending, since we are emitting **at most a
tuple** for nextTuple() so when tuple tree for a tuple is complete or tuple
fails, nextTuple() can be called. At least nextTuple() will be called after
tuple timeout (actually up to 2 * tuple timeout due to underlying
implementation).
So if users can tolerate this, it might be OK, but could be far from
configured value.
If backpressure is in place, nextTuple() will never be called. We can't
guarantee any time frame here.
If we want to strictly guarantee that commit occurs in time, it should be
apart from spout's lifecycle, like timer. It might also introduce some handling
of thread-safety so a bit more complicated, but IMO simpler than respecting
maxUncommittedOffsets.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
