After any of the operators fails during processing, it always recovers from the last checkpointed state. So it will reprocess all the tuples which were processed before failure but not checkpointed. What is a recommended way to to avoid this from happening? Is there any setting in Apex that enables the checkpoint creation just before the operator completely gets killed or fails? If not then how can it be achieved?
I tried tweaking the operator processing mode to EXACTLY_ONCE. Also checked details about CountStoreOperator but to make sure I cover all the individual operator failures, I will have to put CountStoreOperator after every operator. Not sure if this is really scalable solution. What is the best recommended way to achieve this? -- View this message in context: http://apache-apex-users-list.78494.x6.nabble.com/What-is-recommended-way-to-achieve-exactly-once-tuple-processing-in-case-of-operator-failure-scenario-tp1740.html Sent from the Apache Apex Users list mailing list archive at Nabble.com.
