Re: SPARK-SUBMIT and optional args like -h etc

2016-11-30 Thread Daniel van der Ende
Hi, Looks like the ordering of your parameters to spark submit is different on Windows vs EMR. I assume the -h flag is for an arguments for your python script? In that case you'll need to put the arguments after the python script. Daniel On 1 Dec 2016 6:24 a.m., "Patnaik, Vandana"

SPARK-SUBMIT and optional args like -h etc

2016-11-30 Thread Patnaik, Vandana
Hello All, I am new to spark and am wondering how to pass an optional argument to my python program using SPARK-SUBMIT. This works fine on my local machine but not on AWS EMR: On Windows: C:\Vandana\spark\examples>..\bin\spark-submit new_profile_csv1.py -h 0 -t exam ple_float.txt On EMR: