Re: spark-submit question

ot;success" : true > } > ./test3.sh: line 15: --num-decimals=1000: command not found > ./test3.sh: line 16: --second-argument=Arg2: command not found > > > ________ > From: Marcelo Vanzin <van...@cloudera.com> > Sent: Tuesday, February 28, 2017 12:17:49 P

Re: spark-submit question

not found ./test3.sh: line 16: --second-argument=Arg2: command not found From: Marcelo Vanzin <van...@cloudera.com> Sent: Tuesday, February 28, 2017 12:17:49 PM To: Joe Olson Cc: user@spark.apache.org Subject: Re: spark-submit question Everything after the jar path is passe

Re: spark-submit question

Everything after the jar path is passed to the main class as parameters. So if it's not working you're probably doing something wrong in your code (that you haven't posted). On Tue, Feb 28, 2017 at 7:05 AM, Joe Olson wrote: > For spark-submit, I know I can submit application

spark-submit question

For spark-submit, I know I can submit application level command line parameters to my .jar. However, can I prefix them with switches? My command line params are processed in my applications using JCommander. I've tried several variations of the below with no success. An example of what I am

Re: spark-submit question

I figured it out. I had to use pyspark.files.SparkFiles to get the locations of files loaded into Spark. On Mon, Nov 17, 2014 at 1:26 PM, Sean Owen so...@cloudera.com wrote: You are changing these paths and filenames to match your own actual scripts and file locations right? On Nov 17, 2014

Re: spark-submit question

You are changing these paths and filenames to match your own actual scripts and file locations right? On Nov 17, 2014 4:59 AM, Samarth Mailinglist mailinglistsama...@gmail.com wrote: I am trying to run a job written in python with the following command: bin/spark-submit --master