[
https://issues.apache.org/jira/browse/BEAM-12500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
steven zhang updated BEAM-12500:
--------------------------------
Priority: P1 (was: P2)
> Dataflow SocketException (SSLException) error while trying to send message
> from Cloud Pub/Sub to BigQuery
> ---------------------------------------------------------------------------------------------------------
>
> Key: BEAM-12500
> URL: https://issues.apache.org/jira/browse/BEAM-12500
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core, sdk-java-harness
> Affects Versions: 2.26.0, 2.27.0, 2.28.0, 2.29.0
> Environment: GCP services used (such as BigQuery): Pub/Sub
> -->Dataflow-->BigQuery
> Reporter: steven zhang
> Priority: P1
> Labels: SSLException, sdk-consistency
> Fix For: 2.20.0, 2.25.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> *Issue*: I found an issue of seeing SSLException when using beam version
> (>2.25) and could not upgrade with advanced version of beam in the dataflow
> job (streaming). Only versions below 2.25 is possible to work. Could not be
> sure if this issue is connected to [1]
> [1]https://bugs.openjdk.java.net/browse/JDK-8241372
> *Log*: {{Error message from worker: java.lang.RuntimeException:
> java.net.SocketException: Connection reset
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:932)
>
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:972)
>
> org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteFn.flushRows(StreamingWriteFn.java:195)
>
> org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteFn.finishBundle(StreamingWriteFn.java:149)
> Caused by: java.net.SocketException: Connection reset
> java.net.SocketInputStream.read(SocketInputStream.java:210)
> java.net.SocketInputStream.read(SocketInputStream.java:141)
> sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
> sun.security.ssl.InputRecord.read(InputRecord.java:503)
> sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
> sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940)
> sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
> java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:552)
> sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:609)
> sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:696)
> java.io.FilterInputStream.read(FilterInputStream.java:133)
> sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3393)
>
> com.google.api.client.http.javanet.NetHttpResponse$SizeValidatingInputStream.read(NetHttpResponse.java:164)
> java.io.FilterInputStream.read(FilterInputStream.java:133)
> java.io.FilterInputStream.read(FilterInputStream.java:107)
> com.google.common.io.ByteStreams.exhaust(ByteStreams.java:273)
> com.google.api.client.http.ConsumingInputStream.close(ConsumingInputStream.java:40)
> java.util.zip.InflaterInputStream.close(InflaterInputStream.java:227)
> java.util.zip.GZIPInputStream.close(GZIPInputStream.java:136)
> com.fasterxml.jackson.core.json.UTF8StreamJsonParser._closeInput(UTF8StreamJsonParser.java:254)
> com.fasterxml.jackson.core.base.ParserBase.close(ParserBase.java:369)
> com.google.api.client.json.jackson2.JacksonParser.close(JacksonParser.java:47)
> com.google.api.client.json.JsonParser.parse(JsonParser.java:363)
> com.google.api.client.json.JsonParser.parse(JsonParser.java:335)
> com.google.api.client.json.JsonObjectParser.parseAndClose(JsonObjectParser.java:79)
>
> com.google.api.client.json.JsonObjectParser.parseAndClose(JsonObjectParser.java:73)
> com.google.api.client.http.HttpResponse.parseAs(HttpResponse.java:451)
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
>
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.lambda$insertAll$1(BigQueryServicesImpl.java:857)
>
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$BoundedExecutorService$SemaphoreCallable.call(BigQueryServicesImpl.java:1304)
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> java.lang.Thread.run(Thread.java:748)}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
