I am using storm and kafka for replaying messages.
Now I want to save offset of each message and then use it later for
resending the message.
So my question is how can I fetch a single message using its offset ?
That is I know the offset of a message and I want to use the offset to
fetch that
Rather than fetching the message again you could cache it in the spout and
emit it again if the *fail* method is called and delete it when the
*ack*method is called. This is possible as Storm guarantees to call
the
*fail* and *ack* methods with the *messageId* on the exact same spout that
the
In Kafka you cannot fetch just one message based on the offset, what you
get instead is to start fetching from the given offset. Of course you can
just get the first one that you want and then discard the rest and stop
fetching immediately, but I think a better idea would be cache the
thanks for the suggestions.
The problem with caching is that I have to cache a lot of messages as I
don't know which one is going to fail.
If a message is processed at one go caching that message is unnecessary
that's why I want to replay it from the kafka itself.
And I want to use the offset as
