Turns out one of the other developers wrapped the jobs in script and did a
cd to another folder in the script before executing spark-submit.
On 12 June 2015 at 14:20, Matthew Jones mle...@gmail.com wrote:
Hmm either spark-submit isn't picking up the relative path or Chronos is
not setting your
That's a great idea. I did what you suggested and added the url to the
props file in the uri of the json. The properties file now shows up in the
sandbox. But when it goes to run spark-submit with --properties-file
props.properties it fails to find it:
Exception in thread main
Hmm either spark-submit isn't picking up the relative path or Chronos is
not setting your working directory to your sandbox. Try using cd
$MESOS_SANDBOX spark-submit --properties-file props.properties
On Fri, Jun 12, 2015 at 12:32 PM Gary Ogden gog...@gmail.com wrote:
That's a great idea. I
I have a properties file that is hosted at a url. I would like to be able
to use the url in the --properties-file parameter when submitting a job to
mesos using spark-submit via chronos
I would rather do this than use a file on the local server.
This doesn't seem to work though when submitting
That's not supported. You could use wget / curl to download the file to a
temp location before running spark-submit, though.
On Thu, Jun 11, 2015 at 12:48 PM, Gary Ogden gog...@gmail.com wrote:
I have a properties file that is hosted at a url. I would like to be able
to use the url in the
