scwhittle commented on code in PR #35523:
URL: https://github.com/apache/beam/pull/35523#discussion_r2226806550
##########
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/client/AbstractWindmillStream.java:
##########
@@ -122,13 +140,23 @@ protected AbstractWindmillStream(
StreamObserverFactory streamObserverFactory,
Set<AbstractWindmillStream<?, ?>> streamRegistry,
int logEveryNStreamFailures,
- String backendWorkerToken) {
+ String backendWorkerToken,
+ Duration halfClosePhysicalStreamAfter) {
+ checkArgument(!halfClosePhysicalStreamAfter.isNegative());
this.backendWorkerToken = backendWorkerToken;
this.physicalStreamFactory =
(StreamObserver<ResponseT> observer) ->
streamObserverFactory.from(clientFactory, observer);
this.physicalStreamDeadlineSeconds =
streamObserverFactory.getDeadlineSeconds();
+ if (!halfClosePhysicalStreamAfter.isZero()
+ &&
halfClosePhysicalStreamAfter.compareTo(Duration.ofSeconds(physicalStreamDeadlineSeconds))
+ >= 0) {
+ logger.debug("Not attempting to half-close cleanly as stream deadline is
shorter.");
+ halfClosePhysicalStreamAfter = Duration.ZERO;
+ }
+ this.halfClosePhysicalStreamAfter = halfClosePhysicalStreamAfter;
+ this.closingPhysicalStreams = new HashSet<>();
Review Comment:
done
##########
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/client/AbstractWindmillStream.java:
##########
@@ -472,17 +505,74 @@ private void clearPhysicalStreamForDebug() {
currentPhysicalStreamForDebug.set(null);
}
+ private void onHalfClosePhysicalStreamTimeout(PhysicalStreamHandler handler)
{
+ synchronized (this) {
+ if (currentPhysicalStream != handler || clientClosed || isShutdown) {
+ return;
+ }
+ try {
+ handler.streamDebugMetrics.recordHalfClose();
+ closingPhysicalStreams.add(handler);
+ currentPhysicalStream = null;
+ halfCloseFuture = null; // This is the currently running future.
+ clearPhysicalStreamForDebug();
+ requestObserver.onCompleted();
+ } catch (Exception e) {
+ // XXX figure out
+ }
+ try {
+ @NonNull PhysicalStreamHandler streamHandler = newResponseHandler();
Review Comment:
I didn't want to do it in the synchronized block. we also want to do it
after the handling of the error in case that will trigger new messages to be
sent on the new stream.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
