IsisPolei created SPARK-48334: --------------------------------- Summary: NettyServer doesn't shutdown if SparkContext initialize failed Key: SPARK-48334 URL: https://issues.apache.org/jira/browse/SPARK-48334 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 3.1.3 Reporter: IsisPolei
When obtaining a SparkContext instance using SparkContext.getOrCreate(), if an exception occurs during initialization (such as using incorrect Spark parameters, e.g., spark.executor.memory=1 without units), the RpcServer started during this period will not be shut down, resulting in the port being occupied indefinitely. The action to close the RpcServer happens in _env.stop(), where rpcEnv.shutdown() is executed, but this action only occurs when _env != null (SparkContext.scala:2106, version 3.1.3). However, the error occurs during initialization, and _env is not instantiated, so _env.stop() will not be executed, leading to the RpcServer not being closed. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org