IsisPolei created SPARK-48334:
---------------------------------
Summary: NettyServer doesn't shutdown if SparkContext initialize
failed
Key: SPARK-48334
URL: https://issues.apache.org/jira/browse/SPARK-48334
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 3.1.3
Reporter: IsisPolei
When obtaining a SparkContext instance using SparkContext.getOrCreate(), if an
exception occurs during initialization (such as using incorrect Spark
parameters, e.g., spark.executor.memory=1 without units), the RpcServer started
during this period will not be shut down, resulting in the port being occupied
indefinitely.
The action to close the RpcServer happens in _env.stop(), where
rpcEnv.shutdown() is executed, but this action only occurs when _env != null
(SparkContext.scala:2106, version 3.1.3). However, the error occurs during
initialization, and _env is not instantiated, so _env.stop() will not be
executed, leading to the RpcServer not being closed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
