How to compile Python and use How to compile Python and use spark-submit

2016-01-08 Thread Ascot Moss
Hi, Instead of using Spark-shell, does anyone know how to build .zip (or .egg) for Python and use Spark-submit to run? Regards

Re: How to compile Python and use How to compile Python and use spark-submit

2016-01-08 Thread Denny Lee
Per http://spark.apache.org/docs/latest/submitting-applications.html: For Python, you can use the --py-files argument of spark-submit to add .py, .zip or .egg files to be distributed with your application. If you depend on multiple Python files we recommend packaging them into a .zip or .egg.