[ https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=385485&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385485 ]
ASF GitHub Bot logged work on BEAM-9269: ---------------------------------------- Author: ASF GitHub Bot Created on: 11/Feb/20 21:53 Start Date: 11/Feb/20 21:53 Worklog Time Spent: 10m Work Description: nielm commented on pull request #10752: [BEAM-9269] Add commit deadline for Spanner writes. URL: https://github.com/apache/beam/pull/10752#discussion_r377922130 ########## File path: sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerConfig.java ########## @@ -136,13 +163,51 @@ public SpannerConfig withHost(ValueProvider<String> host) { return toBuilder().setHost(host).build(); } + public SpannerConfig withCommitDeadline(Duration commitDeadline) { + return withCommitDeadline(ValueProvider.StaticValueProvider.of(commitDeadline)); + } + + public SpannerConfig withCommitDeadline(ValueProvider<Duration> commitDeadline) { + return toBuilder().setCommitDeadline(commitDeadline).build(); + } + + public SpannerConfig withMaxCumulativeBackoff(Duration maxCumulativeBackoff) { + return withMaxCumulativeBackoff(ValueProvider.StaticValueProvider.of(maxCumulativeBackoff)); + } + + public SpannerConfig withMaxCumulativeBackoff(ValueProvider<Duration> maxCumulativeBackoff) { + return toBuilder().setMaxCumulativeBackoff(maxCumulativeBackoff).build(); + } + @VisibleForTesting SpannerConfig withServiceFactory(ServiceFactory<Spanner, SpannerOptions> serviceFactory) { return toBuilder().setServiceFactory(serviceFactory).build(); } public SpannerAccessor connectToSpanner() { SpannerOptions.Builder builder = SpannerOptions.newBuilder(); + + if (getCommitDeadline() != null && getCommitDeadline().get().getMillis() > 0) { + + // In Spanner API version 1.21 or above, we can set the deadline / total Timeout on an API + // call using the following code: + // + // UnaryCallSettings.Builder commitSettings = + // builder.getSpannerStubSettingsBuilder().commitSettings(); + // RetrySettings.Builder commitRetrySettings = commitSettings.getRetrySettings().toBuilder() + // commitSettings.setRetrySettings( + // commitRetrySettings.setTotalTimeout( + // Duration.ofMillis(getCommitDeadlineMillis().get())) + // .build()); + // + // However, at time of this commit, the Spanner API is at only at v1.6.0, where the only + // method to set a deadline is with GRPC Interceptors, so we have to use that... + SpannerInterceptorProvider interceptorProvider = + SpannerInterceptorProvider.createDefault() + .with(new CommitDeadlineSettingInterceptor(getCommitDeadline().get())); Review comment: > Just to confirm, this deadline will not cause Dataflow workitems to fail but just that request will be retried by SpannerIO within the same workitem Correct, it will backoff/retry up to a configurable time limit (default 15 mins per workitem). ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 385485) Time Spent: 3h 50m (was: 3h 40m) > Set shorter Commit Deadline and handle with backoff/retry > --------------------------------------------------------- > > Key: BEAM-9269 > URL: https://issues.apache.org/jira/browse/BEAM-9269 > Project: Beam > Issue Type: Improvement > Components: io-java-gcp > Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0 > Reporter: Niel Markwick > Assignee: Niel Markwick > Priority: Major > Labels: google-cloud-spanner > Time Spent: 3h 50m > Remaining Estimate: 0h > > Default commit deadline in Spanner is 1hr, which can lead to a variety of > issues including database overload and session expiry. > Shorter deadline should be set with backoff/retry when deadline expires, so > that the Spanner database does not become overloaded. -- This message was sent by Atlassian Jira (v8.3.4#803005)