That is correct. The StreamingFileSink does not extend the TwoPhaseCommitSinkFunction because of some internal implementation details. But this, does not mean that it does not implement a two phase commit protocol (which is independent of the implementation).
Cheers, Kostas On Thu, Jan 10, 2019 at 10:33 AM Taher Koitawala <taher.koitaw...@gslab.com> wrote: > StreamingFileSink extends RichSinkFunction and implements > CheckpointedFunction, CheckpointListener and ProcessingTimeCallback however > TwoPhaseCommitSinkFunction is never used by StreamingFileSink. Hence I had > a question if the sink uses the TwoPhaseCommit protocol or not. > > Regards, > Taher Koitawala > > On Thu 10 Jan, 2019, 2:40 PM Kostas Kloudas <kklou...@gmail.com wrote: > >> Hi Taher, >> >> The StreamingFileSink implements a version of TwoPhaseCommit. Can you >> elaborate a bit on what do you mean by " TwoPhaseCommit is not being used >> "? >> >> Cheers, >> Kostas >> >> On Thu, Jan 10, 2019 at 9:29 AM Taher Koitawala < >> taher.koitaw...@gslab.com> wrote: >> >>> Hi All, >>> As per my understanding and the API of StreamingFileSink, >>> TwoPhaseCommit is not being used. Can someone please confirm is that's >>> right? Also if StreamingFileSink does not support >>> TwoPhaseCommits what is the best way to implement this? >>> >>> >>> Regards, >>> Taher Koitawala >>> GS Lab Pune >>> +91 8407979163 >>> >>
