Hi Jörn, thanks for your reply
Oozie starts ooze java action as single "long running" MapReduce Mapper.
This mapper is responsible for calling main class. Main class belongs to
user and this main class starts spark job.
yarn-cluster is not an option for me. I have to do something special to
maintai
Maybe you should better run it in yarn cluster mode. Yarn client would start
the driver on the oozie server.
> On 19. Mar 2018, at 12:58, Serega Sheypak wrote:
>
> I'm trying to run it as Oozie java action and reduce env dependency. The only
> thing I need is Hadoop Configuration to talk to hd
I'm trying to run it as Oozie java action and reduce env dependency. The
only thing I need is Hadoop Configuration to talk to hdfs and yarn.
Spark submit is a shell thing. Trying to do all from jvm.
Oozie java action starts main class which inststiates SparkConf and
session. It works well in local
Hi,
What's the deployment process then (if not using spark-submit)? How is the
AM deployed? Why would you want to skip spark-submit?
Jacek
On 19 Mar 2018 00:20, "Serega Sheypak" wrote:
> Hi, Is it even possible to run spark on yarn as usual java application?
> I've built jat using maven with s