I tried your program in yarn-client mode and it worked with no
exception. This is the command I used:
spark-submit --master yarn-client --py-files work.py main.py
(Spark 1.2.1)
On 20.3.2015. 9:47, Guillaume Charhon wrote:
Hi Davies,
I am already using --py-files. The system does use the other file. The
error I am getting is not trivial. Please check the error log.
On Thu, Mar 19, 2015 at 8:03 PM, Davies Liu <dav...@databricks.com
<mailto:dav...@databricks.com>> wrote:
You could submit additional Python source via --py-files , for
example:
$ bin/spark-submit --py-files work.py main.py
On Tue, Mar 17, 2015 at 3:29 AM, poiuytrez
<guilla...@databerries.com <mailto:guilla...@databerries.com>> wrote:
> Hello guys,
>
> I am having a hard time to understand how spark-submit behave
with multiple
> files. I have created two code snippets. Each code snippet is
composed of a
> main.py and work.py. The code works if I paste work.py then
main.py in a
> pyspark shell. However both snippets do not work when using
spark submit and
> generate different errors.
>
> Function add_1 definition outside
> http://www.codeshare.io/4ao8B
> https://justpaste.it/jzvj
>
> Embedded add_1 function definition
> http://www.codeshare.io/OQJxq
> https://justpaste.it/jzvn
>
> I am trying a way to make it work.
>
> Thank you for your support.
>
>
>
> --
> View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-submit-and-multiple-files-tp22097.html
> Sent from the Apache Spark User List mailing list archive at
Nabble.com.
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
<mailto:user-unsubscr...@spark.apache.org>
> For additional commands, e-mail: user-h...@spark.apache.org
<mailto:user-h...@spark.apache.org>
>
