Hi!
Yes, AbstractRichFunction.close() would be the right place to do
cleanup. This method is called both in case of successful finishing and
also in the case of failures.
For BATCH execution, Flink will do backtracking upwards from the failed
task(s) to see if intermediate results from previous tasks are still
available. If they are available, computation can restart from there.
Otherwise the whole job will have to be restarted.
Best,
Aljoscha
On 28.09.20 21:44, Boyuan Zhang wrote:
Hi team,
I'm building a UDF by implementing AbstractRichFunction, where I want to do
some resource cleanup per input element when the processing result is
committed. I can perform such cleanup in streaming by implementing
*CheckpointListener.notifyCheckpointComplete() *but it seems like there is
no checkpoint mechanism in batch processing.
I'm wondering is* AbstractRichFunction.close() *the good place to do so?
How does flink deal with fault tolerance in batch?
Thanks for your help!
