How to compile Python and use How to compile Python and use spark-submit
Hi, Instead of using Spark-shell, does anyone know how to build .zip (or .egg) for Python and use Spark-submit to run? Regards
Re: How to compile Python and use How to compile Python and use spark-submit
Per http://spark.apache.org/docs/latest/submitting-applications.html: For Python, you can use the --py-files argument of spark-submit to add .py, .zip or .egg files to be distributed with your application. If you depend on multiple Python files we recommend packaging them into a .zip or .egg.