reuvenlax commented on code in PR #38410:
URL: https://github.com/apache/beam/pull/38410#discussion_r3231350045
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiWritesShardedRecords.java:
##########
@@ -722,11 +713,24 @@ private void handleAppendFailure(
// Finalize the stream and clear streamName so a new stream will be
created.
o.get(flushTag)
.output(KV.of(failedContext.streamName, new
Operation(failedContext.offset - 1, true)));
+
+ // Clear streamName so a new stream will be created.
+ try {
+ streamName.write("");
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+
+ // Re-establish the client with the new stream.
+ clearClients.accept(failedContexts);
Review Comment:
clearClients no longer even looks at failedContexts. We should simplify it
and remove the parameter
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiWritesShardedRecords.java:
##########
@@ -722,11 +713,24 @@ private void handleAppendFailure(
// Finalize the stream and clear streamName so a new stream will be
created.
o.get(flushTag)
.output(KV.of(failedContext.streamName, new
Operation(failedContext.offset - 1, true)));
+
+ // Clear streamName so a new stream will be created.
+ try {
+ streamName.write("");
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+
+ // Re-establish the client with the new stream.
+ clearClients.accept(failedContexts);
Review Comment:
We should probably rename clearClients to resetClient
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
