In general a ParDo might be execute more than once. Most runners will only allow one of those executions to have any effect on the pipeline (e.g. if you are counting elements, this will make sure that you end up with the correct count), however if your ParDo has side effects outside of your pipeline then you need to make sure that those side effects are idempotent.
On Sat, Jun 1, 2019 at 11:16 PM Chaim Turkel <[email protected]> wrote: > Hi, > I have a case where at the end of my pipline i have one element > output and in the last method i write the status and some clean up. I > see that sometimes this code is run more than once (is this possible, > and how can i prevent it?) > > > Chaim > > -- > > > Loans are funded by > FinWise Bank, a Utah-chartered bank located in Sandy, > Utah, member FDIC, Equal > Opportunity Lender. Merchant Cash Advances are > made by Behalf. For more > information on ECOA, click here > <https://www.behalf.com/legal/ecoa/>. For important information about > opening a new > account, review Patriot Act procedures here > <https://www.behalf.com/legal/patriot/>. > Visit Legal > <https://www.behalf.com/legal/> to > review our comprehensive program terms, > conditions, and disclosures. >
