thanks for the suggestions. The problem with caching is that I have to cache a lot of messages as I don't know which one is going to fail. If a message is processed at one go caching that message is unnecessary that's why I want to replay it from the kafka itself. And I want to use the offset as message id so it can serve two purposes that is message id and offset for fetching the message from kafka. Caching also adds an overhead don't you think ? Your responses will be appreciated :) Thanks.
On Tue, Jan 28, 2014 at 3:45 AM, Guozhang Wang <wangg...@gmail.com> wrote: > In Kafka you cannot fetch just one message based on the offset, what you > get instead is to start fetching from the given offset. Of course you can > just get the first one that you want and then discard the rest and stop > fetching immediately, but I think a better idea would be cache the > not-acked-yet messages as Robert suggested. > > Guozhang > > > On Mon, Jan 27, 2014 at 1:39 PM, Robert Turner <rob...@bigfoot.com> wrote: > > > Rather than fetching the message again you could cache it in the spout > and > > emit it again if the *fail* method is called and delete it when the > > *ack*method is called. This is possible as Storm guarantees to call > > the > > *fail* and *ack* methods with the *messageId* on the exact same spout > that > > the message originated from. This means if you have cached the message > > there then it will still be available. > > > > > > > > > > On 27 January 2014 19:29, Abhishek Bhattacharjee < > > abhishek.bhattacharje...@gmail.com> wrote: > > > > > I am using storm and kafka for replaying messages. > > > Now I want to save offset of each message and then use it later for > > > resending the message. > > > So my question is how can I fetch a single message using its offset ? > > > That is I know the offset of a message and I want to use the offset to > > > fetch that message(only that message). > > > > > > > > > Thanks, > > > *Abhishek Bhattacharjee* > > > *Pune Institute of Computer Technology* > > > > > > > > > > > -- > > Cheers > > Rob. > > > > > > -- > -- Guozhang > -- *Abhishek Bhattacharjee* *Pune Institute of Computer Technology*
