RE: Cluster Failure and Data Recovery

Thanks Andrew. J From: Andrew Montalenti [mailto:and...@parsely.com] Sent: Thursday, June 05, 2014 3:30 PM To: user@storm.incubator.apache.org Subject: Re: Cluster Failure and Data Recovery Sounds like you might benefit from considering something like Kafka instead of a standard MQ. We

Re: Cluster Failure and Data Recovery

rinat...@gmail.com] > *Sent:* Wednesday, June 04, 2014 5:49 PM > *To:* user > *Subject:* Re: Cluster Failure and Data Recovery > > > > Hi Nima, > > Use the reliable message processing > <https://github.com/nathanmarz/storm/wiki/Guaranteeing-message-processing> >

RE: Cluster Failure and Data Recovery

to  frame to question better I should ask, if there a way to persist the intermediate data?   Thanks, Nima   From: Srinath C [mailto:srinat...@gmail.com] Sent: Wednesday, June 04, 2014 5:49 PM To: user Subject: Re: Cluster Failure and Data Recovery   Hi Nima,     Use the HYPERLINK "

Re: Cluster Failure and Data Recovery

Hi Nima, Use the reliable message processing mechanism to ensure that there is no data loss. You would need support for transactional semantics from the tuple source where spout can commit/abort a read (kestrel, kafka, r

Cluster Failure and Data Recovery

Hi everyone, We are in the process of designing a high available system with zero data loss tolerance. Plan is for the spouts to read from a queue and process them down in several different specialized bolts and then flush to DB. How can we guarantee no data loss here? Should we keep the que