[
https://issues.apache.org/jira/browse/SPARK-24476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16509708#comment-16509708
]
bharath kumar avusherla commented on SPARK-24476:
-------------------------------------------------
Our initial try was enable the speculative execution, reran the job and it is
working fine. Then replaced s3n with s3a, reran the job. It is working fine
too. Right now we are running two jobs simultaneously with these two different
settings, nothing got failed. Do you recommend enabling speculative execution
even after changing s3n from s3a?
> java.net.SocketTimeoutException: Read timed out under jets3t while running
> the Spark Structured Streaming
> ---------------------------------------------------------------------------------------------------------
>
> Key: SPARK-24476
> URL: https://issues.apache.org/jira/browse/SPARK-24476
> Project: Spark
> Issue Type: Bug
> Components: Structured Streaming
> Affects Versions: 2.3.0
> Reporter: bharath kumar avusherla
> Priority: Minor
> Attachments: socket-timeout-exception
>
>
> We are working on spark streaming application using spark structured
> streaming with checkpointing in s3. When we start the application, the
> application runs just fine for sometime then it crashes with the error
> mentioned below. The amount of time it will run successfully varies from time
> to time, sometimes it will run for 2 days without any issues then crashes,
> sometimes it will crash after 4hrs/ 24hrs.
> Our streaming application joins(left and inner) multiple sources from kafka
> and also s3 and aurora database.
> Can you please let us know how to solve this problem?
> Is it possible to somehow tweak the SocketTimeout-Time?
> Here, I'm pasting the few line of complete exception log below. Also attached
> the complete exception to the issue.
> *_Exception:_*
> *_Caused by: java.net.SocketTimeoutException: Read timed out_*
> _at java.net.SocketInputStream.socketRead0(Native Method)_
> _at java.net.SocketInputStream.read(SocketInputStream.java:150)_
> _at java.net.SocketInputStream.read(SocketInputStream.java:121)_
> _at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)_
> _at sun.security.ssl.InputRecord.read(InputRecord.java:503)_
> _at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:954)_
> _at
> sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1343)_
> _at
> sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1371)_
> _at
> sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1355)_
> _at
> org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:553)_
> _at
> org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:412)_
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
