I think it should be possible for Oozie to poll S3. Check out this <https://www.cloudera.com/documentation/enterprise/5-9-x/topics/admin_oozie_s3.html> description on how to make it work in jobs, something similar should work on the server side as well
On Tue, May 15, 2018 at 4:43 PM, purna pradeep <purna2prad...@gmail.com> wrote: > Thanks Andras, > > Also I also would like to know if oozie supports Aws S3 as input events to > poll for a dependency file before kicking off a spark action > > > For example: I don’t want to kick off a spark action until a file is > arrived on a given AWS s3 location > > On Tue, May 15, 2018 at 10:17 AM Andras Piros <andras.pi...@cloudera.com> > wrote: > > > Hi, > > > > Oozie needs HDFS to store workflow, coordinator, or bundle definitions, > as > > well as sharelib files in a safe, distributed and scalable way. Oozie > needs > > YARN to run almost all of its actions, Spark action being no exception. > > > > At the moment it's not feasible to install Oozie without those Hadoop > > components. How to install Oozie please *find here > > <https://oozie.apache.org/docs/5.0.0/AG_Install.html>*. > > > > Regards, > > > > Andras > > > > On Tue, May 15, 2018 at 4:11 PM, purna pradeep <purna2prad...@gmail.com> > > wrote: > > > > > Hi, > > > > > > Would like to know if I can use sparkaction in oozie without having > > Hadoop > > > cluster? > > > > > > I want to use oozie to schedule spark jobs on Kubernetes cluster > > > > > > I’m a beginner in oozie > > > > > > Thanks > > > > > > -- *Peter Cseh *| Software Engineer cloudera.com <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> ------------------------------
