mutianf commented on code in PR #29548:
URL: https://github.com/apache/beam/pull/29548#discussion_r1417510336
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java:
##########
@@ -180,6 +181,27 @@
* .withBatchElements(100)); // every batch will have 100 elements
* }</pre>
*
+ * <p>Configure timeout for writes:
+ *
+ * <pre>{@code
+ * // Let each attempt run for 1 second, retry if the attempt failed.
+ * // Give up after the request is retried for 60 seconds.
+ * Duration attemptTimeout = Duration.millis(1000);
+ * Duration operationTimeout = Duration.millis(60 * 1000);
+ * data.apply("write",
+ * BigtableIO.write()
+ * .withProjectId("project")
+ * .withInstanceId("instance")
+ * .withTableId("table")
+ * .withAttemptTimeout(attemptTimeout)
Review Comment:
I think closing the service in tearDown() instead of finish_bundle() is the
root cause of the hanging issue in a customer ticket, so this fix should
significantly lower the chance of that happening. However, the underlying batch
also seems to hang occasionally, and closing it with a timeout can mitigate
that, which will cause the bundle the fail but dataflow will retry it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
