Hello,
Can this be used as a library from within another application?
Thanks!
Best, Oliver
From: Matt Chu [mailto:[email protected]]
Sent: Thursday, September 04, 2014 2:46 AM
To: Vicky Kak
Cc: user
Subject: Re: Programatically running of the Spark Jobs.
https://github.com/spark-jobserver/spark-jobserver
Ooyala's Spark jobserver is the current de facto standard, IIUC. I just added
it to our prototype stack, and will begin trying it out soon. Note that you can
only do standalone or Mesos; YARN isn't quite there yet.
(The repo just moved from https://github.com/ooyala/spark-jobserver, so don't
trust Google on this one (yet); development is happening in the first repo.)
On Wed, Sep 3, 2014 at 11:39 PM, Vicky Kak
<[email protected]<mailto:[email protected]>> wrote:
I have been able to submit the spark jobs using the submit script but I would
like to do it via code.
I am unable to search anything matching to my need.
I am thinking of using org.apache.spark.deploy.SparkSubmit to do so, may be
have to write some utility that passes the parameters required for this class.
I would be interested to know how community is doing.
Thanks,
Vicky
***********************************************************************************************************************
This email message and any attachments are intended solely for the use of the
addressee. If you are not the intended recipient, you are prohibited from
reading, disclosing, reproducing, distributing, disseminating or otherwise
using this transmission. If you have received this message in error, please
promptly notify the sender by reply email and immediately delete this message
from your system. This message and any attachments may contain information that
is confidential, privileged or exempt from disclosure. Delivery of this message
to any person other than the intended recipient is not intended to waive any
right or privilege. Message transmission is not guaranteed to be secure or free
of software viruses.
***********************************************************************************************************************
