[
https://issues.apache.org/jira/browse/SPARK-31199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
runnings updated SPARK-31199:
-----------------------------
Description:
io.connectionTimeout only used for connection timeout for connection setup
while io.idleTimeout is used to control how long to kill the connection if it
seems to be idle([https://github.com/apache/spark/pull/5584])
These 2 timeouts could be quite different. connectiontimeout could be short to
help fast fail the connection related problem when meet some high load/unstable
nodes. While idle timeout may be related to business function performance which
is more complicated.
was:
io.connectionTimeout only used for connection timeout for connection setup
while io.idleTimeout is used to control how long to kill the connection if it
seems to be idle([https://github.com/apache/spark/pull/5584])
These 2 timeouts could be quite different and shorten connectiontimeout could
help fast fail the connection related problem in some cases like when doing
shuffle, we could fast fail the task and retry.
> Separate connection timeout and idle timeout for netty
> ------------------------------------------------------
>
> Key: SPARK-31199
> URL: https://issues.apache.org/jira/browse/SPARK-31199
> Project: Spark
> Issue Type: Improvement
> Components: Shuffle
> Affects Versions: 3.1.0
> Reporter: runnings
> Priority: Major
>
> io.connectionTimeout only used for connection timeout for connection setup
> while io.idleTimeout is used to control how long to kill the connection if it
> seems to be idle([https://github.com/apache/spark/pull/5584])
>
> These 2 timeouts could be quite different. connectiontimeout could be short
> to help fast fail the connection related problem when meet some high
> load/unstable nodes. While idle timeout may be related to business function
> performance which is more complicated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
