This is an automated email from the ASF dual-hosted git repository.
mridulm80 pushed a commit to branch branch-3.5
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.5 by this push:
new 3a723a171b1 [SPARK-44937][CORE] Mark connection as timedOut in
TransportClient.close
3a723a171b1 is described below
commit 3a723a171b1b77a8a8b2ccce2bd489acb8db00a3
Author: Hasnain Lakhani <hasnain.lakh...@databricks.com>
AuthorDate: Thu Sep 28 18:16:49 2023 -0500
[SPARK-44937][CORE] Mark connection as timedOut in TransportClient.close
### What changes were proposed in this pull request?
This PR avoids a race condition where a connection which is in the process
of being closed could be returned by the TransportClientFactory only to be
immediately closed and cause errors upon use.
This race condition is rare and not easily triggered, but with the upcoming
changes to introduce SSL connection support, connection closing can take just a
slight bit longer and it's much easier to trigger this issue.
Looking at the history of the code I believe this was an oversight in
https://github.com/apache/spark/pull/9853.
### Why are the changes needed?
Without this change, some of the new tests added in
https://github.com/apache/spark/pull/42685 would fail
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existing tests were run in CI.
Without this change, some of the new tests added in
https://github.com/apache/spark/pull/42685 fail
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #43162 from hasnain-db/spark-tls-timeout.
Authored-by: Hasnain Lakhani <hasnain.lakh...@databricks.com>
Signed-off-by: Mridul Muralidharan <mridul<at>gmail.com>
(cherry picked from commit 2a88feadd4b7cec9e01bc744e589783e3390e5ce)
Signed-off-by: Mridul Muralidharan <mridulatgmail.com>
---
.../main/java/org/apache/spark/network/client/TransportClient.java | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git
a/common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java
b/common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java
index 4a0a1566998..40825e06b82 100644
---
a/common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java
+++
b/common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java
@@ -325,7 +325,10 @@ public class TransportClient implements Closeable {
@Override
public void close() {
- // close is a local operation and should finish with milliseconds; timeout
just to be safe
+ // Mark the connection as timed out, so we do not return a connection
that's being closed
+ // from the TransportClientFactory if closing takes some time (e.g. with
SSL)
+ this.timedOut = true;
+ // close should not take this long; use a timeout just to be safe
channel.close().awaitUninterruptibly(10, TimeUnit.SECONDS);
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
