Github user rick-kilgore commented on a diff in the pull request:
https://github.com/apache/storm/pull/254#discussion_r18382648
--- Diff: external/storm-kafka/src/jvm/storm/kafka/SpoutConfig.java ---
@@ -26,8 +26,19 @@
public Integer zkPort = null;
public String zkRoot = null;
public String id = null;
+
+ // setting for how often to save the current kafka offset to ZooKeeper
public long stateUpdateIntervalMs = 2000;
+ // Exponential back-off retry settings. These are used when retrying
messages after a bolt
+ // calls OutputCollector.fail().
+ //
+ // Note: be sure to set backtype.storm.Config.MESSAGE_TIMEOUT_SECS
appropriately to prevent
+ // resubmitting the message while still retrying.
--- End diff --
I created a version of PartitionManager that stops retrying when it would
run up against the TOPOLOGY_MESSAGE_TIMEOUT_SECS setting. But while I was
testing it, I discovered that the Storm framework is not actually failing the
message after this timeout. Did that behavior change recently? I thought I
remembered seeing it do that before.
Since it's not happening, the message ends up getting lost - neither failed
nor acknowledged. So maybe after printing the big nasty error message, I
should still handle the message in some way - either ack() it or just let it
keep retrying?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
