You could submit additional Python source via --py-files , for example: $ bin/spark-submit --py-files work.py main.py
On Tue, Mar 17, 2015 at 3:29 AM, poiuytrez <guilla...@databerries.com> wrote: > Hello guys, > > I am having a hard time to understand how spark-submit behave with multiple > files. I have created two code snippets. Each code snippet is composed of a > main.py and work.py. The code works if I paste work.py then main.py in a > pyspark shell. However both snippets do not work when using spark submit and > generate different errors. > > Function add_1 definition outside > http://www.codeshare.io/4ao8B > https://justpaste.it/jzvj > > Embedded add_1 function definition > http://www.codeshare.io/OQJxq > https://justpaste.it/jzvn > > I am trying a way to make it work. > > Thank you for your support. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-submit-and-multiple-files-tp22097.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
