Hayden Jeune created SPARK-26080:
------------------------------------
Summary: Unable to run worker.py on Windows
Key: SPARK-26080
URL: https://issues.apache.org/jira/browse/SPARK-26080
Project: Spark
Issue Type: Bug
Components: PySpark
Affects Versions: 2.4.0
Environment: Windows 10 Education 64 bit
Reporter: Hayden Jeune
Use of the resource module in python means worker.py cannot run on a windows
system. This package is only available in unix based environments.
[https://github.com/apache/spark/blob/9a5fda60e532dc7203d21d5fbe385cd561906ccb/python/pyspark/worker.py#L25]
{code:python}
textFile = sc.textFile("README.md")
textFile.first()
{code}
When the above commands are run I receive the error 'worker failed to connect
back', and I can see an exception in the console coming from worker.py saying
'ModuleNotFoundError: No module named resource'
I do not really know enough about what I'm doing to fix this myself. Apologies
if there's something simple I'm missing here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
