Github user sakanaou commented on the issue:
https://github.com/apache/storm/pull/1993
Thanks for your suggestions, @harshach & @hmcl. Maybe I didn't find the
correct way to handle our request yet in the provided architecture. But to make
it more clear: The records/tuples are not per se invalid and should go straight
to a "dead-letter-topic". The system is more about resilience in such a way
that it should not be a big problem if a system, that lies behind a downward
bolt, is temporarily unavailable. Instead of simply "skipping" such failed
records (after their retry count is reached) they should then - and only then -
be written to the mentioned "dead-letter-topic". We did so in the overwritten
`fail()` of our `KafkaSpout` and built a map from `KafkaSpoutMessageId` to the
`ConsumerRecord` with the help of the proposed `emittingTuple()`.
But I assume we could work out a solution solely based on the mentioned
`RecordTranslator` (or `KafkaSpoutTupleBuilder` in our case as we are still on
Storm 1.0.2).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
