Hi All, Per https://github.com/apache/beam/issues/26521 and https://github.com/apache/beam/issues/26520, there is an issue in Beam versions 2.33 - 2.47 where data can be lost when using the Storage Write API in Batch. This issue is much more likely to occur in versions 2.44-2.47.
The bugs themselves have been resolved , thanks @Reuven Lax for the quick fixes. For now, the recommended workaround is to use the File Loads method of Bigquery Write instead. I am ooo for the next week, but we will be doing a postmortem when I return to identify how a gap like this occurred, and how to prevent it in the future. Thanks, John
