Rafal Wojdyla created SPARK-48711:
-------------------------------------
Summary: OOM killer may leave SparkContext in broken state causing
ConnectionRefusedError
Key: SPARK-48711
URL: https://issues.apache.org/jira/browse/SPARK-48711
Project: Spark
Issue Type: Bug
Components: PySpark, Spark Core
Affects Versions: 3.5.0
Reporter: Rafal Wojdyla
Related to https://issues.apache.org/jira/browse/SPARK-18523, and
https://github.com/apache/spark/pull/15961. I'm currently on:
{code}
pyspark 3.5.0 pyhd8ed1ab_0 conda-forge
py4j 0.10.9.7 pyhd8ed1ab_0 conda-forge
{code}
When Spark JVM process gets OOM-Killed, `SparkContext.stop` fails with
`ConnectionRefusedError`, which leaves the `SparkSession/Context` in a "dirty"
state. https://issues.apache.org/jira/browse/SPARK-18523 addressed this by
catching the {{Py4JError}} it looks like the code now raises
{{ConnectionRefusedError}}:
{code}
Traceback (most recent call last):
...
File "<TRUNC>/lib/python3.11/site-packages/pyspark/sql/session.py", line
1796, in stop
self._sc.stop()
File "<TRUNC>/lib/python3.11/site-packages/pyspark/context.py", line 654, in
stop
self._jsc.stop()
File "<TRUNC>/lib/python3.11/site-packages/py4j/java_gateway.py", line 1321,
in __call__
answer = self.gateway_client.send_command(command)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<TRUNC>/lib/python3.11/site-packages/py4j/java_gateway.py", line 1036,
in send_command
connection = self._get_connection()
^^^^^^^^^^^^^^^^^^^^^^
File "<TRUNC>/lib/python3.11/site-packages/py4j/clientserver.py", line 284,
in _get_connection
connection = self._create_new_connection()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<TRUNC>/lib/python3.11/site-packages/py4j/clientserver.py", line 291,
in _create_new_connection
connection.connect_to_java_server()
File "<TRUNC>/lib/python3.11/site-packages/py4j/clientserver.py", line 438,
in connect_to_java_server
self.socket.connect((self.java_address, self.java_port))
ConnectionRefusedError: [Errno 111] Connection refused
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
