This is an automated email from the ASF dual-hosted git repository. hvanhovell pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 8cc8dd6f67a [SPARK-42272][CONNEC][TESTS][FOLLOW-UP] Do not cache local port in SparkConnectService 8cc8dd6f67a is described below commit 8cc8dd6f67a31cb46a228a7fd06ab0c18a462c07 Author: Hyukjin Kwon <gurwls...@apache.org> AuthorDate: Wed Feb 22 11:02:35 2023 -0400 [SPARK-42272][CONNEC][TESTS][FOLLOW-UP] Do not cache local port in SparkConnectService ### What changes were proposed in this pull request? This PR proposes to do not cache local port. ### Why are the changes needed? When Spark Context is stopped, and started again, the Spark Connect server shuts down and starts up again too (while JVM itself is alive). So, we should not cache the local port but have the new local port. For example, in https://github.com/apache/spark/pull/40109, the Spark Connect server at `ReadwriterTestsMixin` stops after the tests. And then, `ReadwriterV2TestsMixin` starts the new Spark Connect server which causes failures on any actual protobuf message exchanges to the server. ### Does this PR introduce _any_ user-facing change? No, test-only. ### How was this patch tested? I tested it on the top of https://github.com/apache/spark/pull/40109. That PR should validate it. Closes #40123 from HyukjinKwon/SPARK-42272-followup. Authored-by: Hyukjin Kwon <gurwls...@apache.org> Signed-off-by: Herman van Hovell <her...@databricks.com> --- .../org/apache/spark/sql/connect/service/SparkConnectService.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectService.scala b/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectService.scala index 0a5b4197b78..959aceaf46a 100644 --- a/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectService.scala +++ b/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectService.scala @@ -241,7 +241,7 @@ object SparkConnectService { private[connect] var server: Server = _ // For testing purpose, it's package level private. - private[connect] lazy val localPort = { + private[connect] def localPort: Int = { assert(server != null) // Return the actual local port being used. This can be different from the csonfigured port // when the server binds to the port 0 as an example. --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org