[jira] [Created] (BEAM-4378) Update javadoc for set/getRegion in DataflowPipelineOptions.
David Yan created BEAM-4378: --- Summary: Update javadoc for set/getRegion in DataflowPipelineOptions. Key: BEAM-4378 URL: https://issues.apache.org/jira/browse/BEAM-4378 Project: Beam Issue Type: Improvement Components: runner-dataflow Reporter: David Yan Assignee: Thomas Groh Dataflow now supports the region flag. We should update the javadoc here: https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/options/DataflowPipelineOptions.java#L107 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-3764) Add retry logic in ExampleUtils that handles network flakes.
David Yan created BEAM-3764: --- Summary: Add retry logic in ExampleUtils that handles network flakes. Key: BEAM-3764 URL: https://issues.apache.org/jira/browse/BEAM-3764 Project: Beam Issue Type: Improvement Components: examples-java Reporter: David Yan Assignee: Reuven Lax This exception is thrown when there is a network flake when running the traffic examples. Should we retry instead of throwing the exception and failing the job? {{Exception in thread "main" java.io.IOException: Error getting access token for service account: }} {{ at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:319)}} {{ at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:149)}} {{ at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:135)}} {{ at com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:96)}} {{ at com.google.cloud.hadoop.util.ChainingHttpRequestInitializer.initialize(ChainingHttpRequestInitializer.java:52)}} {{ at com.google.api.client.http.HttpRequestFactory.buildRequest(HttpRequestFactory.java:93)}} {{ at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:300)}} {{ at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)}} {{ at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)}} {{ at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)}} {{ at org.apache.beam.examples.common.ExampleUtils.executeNullIfNotFound(ExampleUtils.java:397)}} {{ at org.apache.beam.examples.common.ExampleUtils.setupPubsubTopic(ExampleUtils.java:283)}} {{ at org.apache.beam.examples.common.ExampleUtils.setupPubsub(ExampleUtils.java:131)}} {{ at org.apache.beam.examples.common.ExampleUtils.setup(ExampleUtils.java:105)}} {{ at org.apache.beam.examples.complete.TrafficMaxLaneFlow.main(TrafficMaxLaneFlow.java:334)}} {{Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake}} {{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:994)}} {{ at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1379)}} {{ at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1407)}} {{ at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1391)}} {{ at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)}} {{ at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)}} {{ at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1316)}} {{ at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1291)}} {{ at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250)}} {{ at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:77)}} {{ at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)}} {{ at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:317)}} {{ ... 14 more}} {{Caused by: java.io.EOFException: SSL peer shut down incorrectly}} {{ at sun.security.ssl.InputRecord.read(InputRecord.java:505)}} {{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}} {{ ... 25 more}} Add retry code somewhere here? https://github.com/apache/beam/blob/29859eb54d05b96a9db477e7bb04537510273bd2/examples/java/src/main/java/org/apache/beam/examples/common/ExampleUtils.java#L398 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-3613) SpannerIO: Typo in "witHost"
[ https://issues.apache.org/jira/browse/BEAM-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Yan resolved BEAM-3613. - Resolution: Fixed Fix Version/s: 2.4.0 > SpannerIO: Typo in "witHost" > > > Key: BEAM-3613 > URL: https://issues.apache.org/jira/browse/BEAM-3613 > Project: Beam > Issue Type: Bug > Components: sdk-java-gcp >Reporter: David Yan >Assignee: Chamikara Jayalath >Priority: Minor > Fix For: 2.4.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > There is a typo in SpannerIO.java. > "witHost" should be "withHost". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-3613) SpannerIO: Typo in "witHost"
[ https://issues.apache.org/jira/browse/BEAM-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16351178#comment-16351178 ] David Yan commented on BEAM-3613: - [https://github.com/apache/beam/pull/4592] open. > SpannerIO: Typo in "witHost" > > > Key: BEAM-3613 > URL: https://issues.apache.org/jira/browse/BEAM-3613 > Project: Beam > Issue Type: Bug > Components: sdk-java-gcp >Reporter: David Yan >Assignee: Chamikara Jayalath >Priority: Minor > > There is a typo in SpannerIO.java. > "witHost" should be "withHost". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-3613) SpannerIO: Typo in "witHost"
David Yan created BEAM-3613: --- Summary: SpannerIO: Typo in "witHost" Key: BEAM-3613 URL: https://issues.apache.org/jira/browse/BEAM-3613 Project: Beam Issue Type: Bug Components: sdk-java-gcp Reporter: David Yan Assignee: Chamikara Jayalath There is a typo in SpannerIO.java. "witHost" should be "withHost". -- This message was sent by Atlassian JIRA (v7.6.3#76005)