[ https://issues.apache.org/jira/browse/SPARK-40122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean R. Owen resolved SPARK-40122. ---------------------------------- Resolution: Invalid This itself doesn't mean anything - means the Python process died. It'd have to be more specific to be actionable > py4j-0.10.9.5 often produces "Connection reset by peer" in Spark 3.3.0 > ----------------------------------------------------------------------- > > Key: SPARK-40122 > URL: https://issues.apache.org/jira/browse/SPARK-40122 > Project: Spark > Issue Type: Bug > Components: PySpark > Affects Versions: 3.3.0 > Reporter: Ihor Bobak > Priority: Major > > Without any visible reason I am getting this error in my Jupyter notebook > (see stacktrace below) with pyspark kernel. Often it occurs even if no Spark > operations are made, e.g. when I am working with multiprocessing Pool for a > local piece of code that should parallelize on the cores of the driver, with > no spark transformations/actions done in that jupyter cell. > INFO:py4j.clientserver:Error while sending or receiving. > Traceback (most recent call last): > File > "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py", > line 503, in send_command > self.socket.sendall(command.encode("utf-8")) > ConnectionResetError: [Errno 104] Connection reset by peer > INFO:py4j.clientserver:Closing down clientserver connection > INFO:root:Exception while sending command. > Traceback (most recent call last): > File > "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py", > line 503, in send_command > self.socket.sendall(command.encode("utf-8")) > ConnectionResetError: [Errno 104] Connection reset by peer > During handling of the above exception, another exception occurred: > Traceback (most recent call last): > File > "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py", > line 1038, in send_command > response = connection.send_command(command) > File > "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py", > line 506, in send_command > raise Py4JNetworkError( > py4j.protocol.Py4JNetworkError: Error while sending > INFO:py4j.clientserver:Closing down clientserver connection -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org