[ https://issues.apache.org/jira/browse/SPARK-40122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ihor Bobak updated SPARK-40122: ------------------------------- Description: Without any visible reason I am getting this error in my Jupyter notebook (see stacktrace below) with pyspark kernel. Often it occurs even if no Spark operations are made, e.g. when I am working with multiprocessing Pool for a local piece of code that should parallelize on the cores of the driver, with no spark transformations/actions done in that jupyter cell. INFO:py4j.clientserver:Error while sending or receiving. Traceback (most recent call last): File "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py", line 503, in send_command self.socket.sendall(command.encode("utf-8")) ConnectionResetError: [Errno 104] Connection reset by peer INFO:py4j.clientserver:Closing down clientserver connection INFO:root:Exception while sending command. Traceback (most recent call last): File "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py", line 503, in send_command self.socket.sendall(command.encode("utf-8")) ConnectionResetError: [Errno 104] Connection reset by peer During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py", line 1038, in send_command response = connection.send_command(command) File "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py", line 506, in send_command raise Py4JNetworkError( py4j.protocol.Py4JNetworkError: Error while sending INFO:py4j.clientserver:Closing down clientserver connection was: Without any visible reason I am getting this error in my Jupyter notebook (see stacktrace below) with pyspark kernel. Often it occurs even if no Spark operations are made, e.g. when I am working with multiprocessing Pool for a local piece of code that should parallelize on the cores of the driver, with no spark transformations/actions done in that jupyter cell. {{INFO:py4j.clientserver:Error while sending or receiving. Traceback (most recent call last): File "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py", line 503, in send_command self.socket.sendall(command.encode("utf-8")) ConnectionResetError: [Errno 104] Connection reset by peer INFO:py4j.clientserver:Closing down clientserver connection INFO:root:Exception while sending command. Traceback (most recent call last): File "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py", line 503, in send_command self.socket.sendall(command.encode("utf-8")) ConnectionResetError: [Errno 104] Connection reset by peer During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py", line 1038, in send_command response = connection.send_command(command) File "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py", line 506, in send_command raise Py4JNetworkError( py4j.protocol.Py4JNetworkError: Error while sending INFO:py4j.clientserver:Closing down clientserver connection }} > py4j-0.10.9.5 often produces "Connection reset by peer" in Spark 3.3.0 > ----------------------------------------------------------------------- > > Key: SPARK-40122 > URL: https://issues.apache.org/jira/browse/SPARK-40122 > Project: Spark > Issue Type: Bug > Components: PySpark > Affects Versions: 3.3.0 > Reporter: Ihor Bobak > Priority: Major > > Without any visible reason I am getting this error in my Jupyter notebook > (see stacktrace below) with pyspark kernel. Often it occurs even if no Spark > operations are made, e.g. when I am working with multiprocessing Pool for a > local piece of code that should parallelize on the cores of the driver, with > no spark transformations/actions done in that jupyter cell. > INFO:py4j.clientserver:Error while sending or receiving. > Traceback (most recent call last): > File > "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py", > line 503, in send_command > self.socket.sendall(command.encode("utf-8")) > ConnectionResetError: [Errno 104] Connection reset by peer > INFO:py4j.clientserver:Closing down clientserver connection > INFO:root:Exception while sending command. > Traceback (most recent call last): > File > "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py", > line 503, in send_command > self.socket.sendall(command.encode("utf-8")) > ConnectionResetError: [Errno 104] Connection reset by peer > During handling of the above exception, another exception occurred: > Traceback (most recent call last): > File > "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py", > line 1038, in send_command > response = connection.send_command(command) > File > "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py", > line 506, in send_command > raise Py4JNetworkError( > py4j.protocol.Py4JNetworkError: Error while sending > INFO:py4j.clientserver:Closing down clientserver connection -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org