Thank you Andrew for you reply!
I am very intested in having this feature. It is possible to run PySpark on
AWS EMR in client mode(https://aws.amazon.com/articles/4926593393724923),
but that kills the whole idea of running batch jobs in EMR on PySpark.
Could you please (help to) create a
Hi all!
I found this problem when I tried running python application on Amazon's
EMR yarn cluster.
It is possible to run bundled example applications on EMR but I cannot
figure out how to run a little bit more complex python application which
depends on some other python scripts. I tried adding
Hi all!
I found this problem when I tried running python application on Amazon's
EMR yarn cluster.
It is possible to run bundled example applications on EMR but I cannot
figure out how to run a little bit more complex python application which
depends on some other python scripts. I tried adding
Hi Vladimir,
Yes, as the error messages suggests, PySpark currently only supports local
files. This does not mean it only runs in local mode, however; you can
still run PySpark on any cluster manager (though only in client mode). All
this means is that your python files must be on your local file
-spark-user-list.1001560.n3.nabble.com/spark-submit-py-files-remote-Only-local-additional-python-files-are-supported-tp21216.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user
