Hi, It would be necessary to know a bit more about your application for specific recommendations, but from what I see above, a few things don't look right.
It appears that you are setting the processing mode on the input operator, which only reads from Kafka. Exactly-once is important for operators that affect the state of external systems, for example when writing to a database, or, producing messages to Kafka. There is also a problem with the documentation that may contribute to the confusion. The processing mode you are trying to use isn't really exactly-once and probably will be deprecated in a future release. Here are some better resources on fault tolerance and processing semantics: http://apex.apache.org/docs.html https://www.youtube.com/watch?v=FCMY6Ii89Nw http://www.slideshare.net/ApacheApexOrganizer/webinar-fault-toleranceandprocessingsemantics https://www.datatorrent.com/blog/end-to-end-exactly-once-with-apache-apex/ HTH, Thomas On Thu, Oct 6, 2016 at 11:26 AM, Bandaru, Srinivas < [email protected]> wrote: > Hi, Need some help! > > Need your advice in implementing the fault tolerance mechanism using > datatorrent. Currently we are trying to implement with EXACTLY_ONCE > scenario. > > We are referring the below documentation. The highlighted line says if we > need to use the EXACTLY_ONCE processing mode, the downstream operators > should have AT_MOST_ONCE. We added the operator level attributes in the > properties.xml (attached below) and when we launch the application, no > messages are coming through the last operator *topicUpdate *but we are > able to see the messages when we comment those attributes. > > *Please have a look at it, either we are specifying the attributes > incorrectly or is there any additional step to be done*. > > > > > > ยท *PROCESSING_MODE* > > static final Attribute > <https://www.datatorrent.com/docs/apidocs/com/datatorrent/api/Attribute.html><Operator.ProcessingMode > > <https://www.datatorrent.com/docs/apidocs/com/datatorrent/api/Operator.ProcessingMode.html>> > PROCESSING_MODE > > The payload processing mode for this operator - at most once, exactly > once, or default at least once. If the processing mode for an operator is > specified as AT_MOST_ONCE and no processing mode is specified for the > downstream operators if any, the processing mode of the downstream > operators is automatically set to AT_MOST_ONCE. If a different processing > mode is specified for the downstream operators it will result in an error. If > the processing mode for an operator is specified as EXACTLY_ONCE then the > processing mode for all downstream operators should be specified as > AT_MOST_ONCE otherwise it will result in an error. > > > > > > > > > > > > > > Thanks, > > Srinivas > > > This e-mail, including attachments, may include confidential and/or > proprietary information, and may be used only by the person or entity > to which it is addressed. If the reader of this e-mail is not the intended > recipient or his or her authorized agent, the reader is hereby notified > that any dissemination, distribution or copying of this e-mail is > prohibited. If you have received this e-mail in error, please notify the > sender by replying to this message and delete this e-mail immediately. >
