This is an automated email from the ASF dual-hosted git repository.
reuvenlax pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
from 428ec97e30c improve error message for mismatched pipelines (#24834)
add 7ad44c84585 Handle schema updates in Storage API writes.
new f5020e7ac2b Merge pull request #24145: Handle updates to table schema
when using Storage API writes.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../beam/sdk/io/gcp/bigquery/AppendClientInfo.java | 115 +++++++++--
.../beam/sdk/io/gcp/bigquery/BigQueryIO.java | 18 +-
.../beam/sdk/io/gcp/bigquery/BigQueryServices.java | 8 +
.../sdk/io/gcp/bigquery/BigQueryServicesImpl.java | 11 ++
.../beam/sdk/io/gcp/bigquery/BigQueryUtils.java | 29 ---
.../sdk/io/gcp/bigquery/SplittingIterable.java | 61 +++++-
.../bigquery/StorageApiDynamicDestinations.java | 2 +
.../StorageApiDynamicDestinationsBeamRow.java | 8 +-
.../StorageApiDynamicDestinationsTableRow.java | 24 ++-
.../beam/sdk/io/gcp/bigquery/StorageApiLoads.java | 20 +-
.../io/gcp/bigquery/StorageApiWritePayload.java | 25 ++-
.../StorageApiWriteRecordsInconsistent.java | 12 +-
.../bigquery/StorageApiWriteUnshardedRecords.java | 146 +++++++++++---
.../bigquery/StorageApiWritesShardedRecords.java | 173 +++++++++++------
.../io/gcp/bigquery/TableRowToStorageApiProto.java | 110 +++++++++--
.../sdk/io/gcp/testing/FakeDatasetService.java | 63 +++++-
.../sdk/io/gcp/bigquery/BigQueryIOWriteTest.java | 216 +++++++++++++++++++++
.../bigquery/TableRowToStorageApiProtoTest.java | 95 ++++++++-
18 files changed, 958 insertions(+), 178 deletions(-)
