Hello, Is there any way to identify that on rebalance *no messages are lost* and every message is processed at leat once in Storm 1.0.3 with stateful task backed by redis.
On Mon, Feb 20, 2017 at 9:57 AM, Arun Mahadevan <ar...@apache.org> wrote: > This is expected with in-memory state, which stores the state in a local > hash map and is not intended for any real use cases. And I don’t think > there is any value in serializing the in-memory state during rebalance. How > would you resurrect the state if the task gets reassigned to a different > host? > > > > Better use the redis state implementation or write a state implementation > that uses distributed memory like on top of memcached. > > > > Arun > > > > *From: *anshu shukla <anshushuk...@gmail.com> > *Reply-To: *"user@storm.apache.org" <user@storm.apache.org> > *Date: *Monday, February 20, 2017 at 8:50 AM > *To: *"user@storm.apache.org" <user@storm.apache.org> > *Subject: *Rebalancing Stateful bolts in storm 1.0.2 > > > > Hey, > > I was running in-memory stateful bolt and while doing *rebalance the > state for the tasks get lost. * > > Can anyone suggest some work around to retain the state while rebalancing > without using stores like Redis (like on rebalancing I should serialize the > state along with the task while regrouping task to other threads)? > > please suggest some general idea about the possibilities. > > > > -- > > Thanks & Regards, > Anshu Shukla > -- Thanks & Regards, Anshu Shukla
