Re: [External Sender] Re: [Question] Error handling for IO Write Functions

2023-11-09 Thread Robert Bradshaw via dev
+1 Specifically, p.run().waitUntilFinish() would throw an exception if there were errors during pipeline execution. On Wed, Nov 8, 2023 at 8:05 AM John Casey via dev wrote: > Yep, thats a common misunderstanding with beam. > > The code that is actually executed in the try block is just for pipe

Re: [External Sender] Re: [Question] Error handling for IO Write Functions

2023-11-08 Thread John Casey via dev
Yep, thats a common misunderstanding with beam. The code that is actually executed in the try block is just for pipeline construction, and no data is processed at this point in time. Once the pipeline is constructed, the various pardos are serialized, and sent to the runners, where they are actua

Re: [External Sender] Re: [Question] Error handling for IO Write Functions

2023-11-08 Thread Ramya Prasad via dev
Hey John, Yes that's how my code is set up, I have the FileIO.write() in its own try-catch block. I took a second look at where exactly the code is failing, and it's actually in a ParDo function which is happening before I call FileIO.write(). But even within that, I've tried adding a try-catch bu