Thanks Thomas. We have set the set all three operator properties, Kafka input operator has set to “Exactly_once” and downstream two operators set as AT_MOST_ONCE same as mentioned in documentation. We have also referred the documentation you have shared but couldn’t find the right properties for exactly_once scenario. Is there anything we are missing or need to add additional properties if someone else used this functionality.
Any help would be greatly appreciated?? We tried with different combinations and trying to set all three properties with “EXACTLY_ONCE”, But application throwing an with error saying “If the processing mode for an operator is specified as EXACTLY_ONCE then the processing mode for all downstream operators should be specified as AT_MOST_ONCE otherwise it will result in an error.” Application behavior is same as the Datatorrent documentation but it’s not working. Thanks, Srinivas From: Thomas Weise [mailto:[email protected]] Sent: Thursday, October 06, 2016 3:03 PM To: [email protected] Subject: Re: Datatorrent fault tolerance Hi, It would be necessary to know a bit more about your application for specific recommendations, but from what I see above, a few things don't look right. It appears that you are setting the processing mode on the input operator, which only reads from Kafka. Exactly-once is important for operators that affect the state of external systems, for example when writing to a database, or, producing messages to Kafka. There is also a problem with the documentation that may contribute to the confusion. The processing mode you are trying to use isn't really exactly-once and probably will be deprecated in a future release. Here are some better resources on fault tolerance and processing semantics: http://apex.apache.org/docs.html https://www.youtube.com/watch?v=FCMY6Ii89Nw http://www.slideshare.net/ApacheApexOrganizer/webinar-fault-toleranceandprocessingsemantics https://www.datatorrent.com/blog/end-to-end-exactly-once-with-apache-apex/ HTH, Thomas On Thu, Oct 6, 2016 at 11:26 AM, Bandaru, Srinivas <[email protected]<mailto:[email protected]>> wrote: Hi, Need some help! Need your advice in implementing the fault tolerance mechanism using datatorrent. Currently we are trying to implement with EXACTLY_ONCE scenario. We are referring the below documentation. The highlighted line says if we need to use the EXACTLY_ONCE processing mode, the downstream operators should have AT_MOST_ONCE. We added the operator level attributes in the properties.xml (attached below) and when we launch the application, no messages are coming through the last operator topicUpdate but we are able to see the messages when we comment those attributes. Please have a look at it, either we are specifying the attributes incorrectly or is there any additional step to be done. • PROCESSING_MODE static final Attribute<https://www.datatorrent.com/docs/apidocs/com/datatorrent/api/Attribute.html><Operator.ProcessingMode<https://www.datatorrent.com/docs/apidocs/com/datatorrent/api/Operator.ProcessingMode.html>> PROCESSING_MODE The payload processing mode for this operator - at most once, exactly once, or default at least once. If the processing mode for an operator is specified as AT_MOST_ONCE and no processing mode is specified for the downstream operators if any, the processing mode of the downstream operators is automatically set to AT_MOST_ONCE. If a different processing mode is specified for the downstream operators it will result in an error. If the processing mode for an operator is specified as EXACTLY_ONCE then the processing mode for all downstream operators should be specified as AT_MOST_ONCE otherwise it will result in an error. [cid:[email protected]] [cid:[email protected]] Thanks, Srinivas This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately. This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately.
