That's strange, this exception should not happen in that case. Can you check the server logs for messages like this? LOG.info("Loaded urihandlers {0}", Arrays.toString(classes)); LOG.info("Loaded default urihandler {0}", defaultHandler.getClass().getName()); Thanks
On Wed, May 16, 2018 at 5:47 PM, purna pradeep <purna2prad...@gmail.com> wrote: > This is what I already have in my oozie-site.xml > > <property> > > <name>oozie.service.HadoopAccessorService. > supported.filesystems</name> > > <value>*</value> > > </property> > > On Wed, May 16, 2018 at 11:37 AM Peter Cseh <gezap...@cloudera.com> wrote: > >> You'll have to configure >> oozie.service.HadoopAccessorService.supported.filesystems >> hdfs,hftp,webhdfs Enlist >> the different filesystems supported for federation. If wildcard "*" is >> specified, then ALL file schemes will be allowed.properly. >> >> For testing purposes it's ok to put * in there in oozie-site.xml >> >> On Wed, May 16, 2018 at 5:29 PM, purna pradeep <purna2prad...@gmail.com> >> wrote: >> >> > Peter, >> > >> > I have tried to specify dataset with uri starting with s3://, s3a:// and >> > s3n:// and I am getting exception >> > >> > >> > >> > Exception occurred:E0904: Scheme [s3] not supported in uri >> > [s3://mybucket/input.data] Making the job failed >> > >> > org.apache.oozie.dependency.URIHandlerException: E0904: Scheme [s3] not >> > supported in uri [s3:// mybucket /input.data] >> > >> > at >> > org.apache.oozie.service.URIHandlerService.getURIHandler( >> > URIHandlerService.java:185) >> > >> > at >> > org.apache.oozie.service.URIHandlerService.getURIHandler( >> > URIHandlerService.java:168) >> > >> > at >> > org.apache.oozie.service.URIHandlerService.getURIHandler( >> > URIHandlerService.java:160) >> > >> > at >> > org.apache.oozie.command.coord.CoordCommandUtils.createEarlyURIs( >> > CoordCommandUtils.java:465) >> > >> > at >> > org.apache.oozie.command.coord.CoordCommandUtils. >> > separateResolvedAndUnresolved(CoordCommandUtils.java:404) >> > >> > at >> > org.apache.oozie.command.coord.CoordCommandUtils. >> > materializeInputDataEvents(CoordCommandUtils.java:731) >> > >> > at >> > org.apache.oozie.command.coord.CoordCommandUtils. >> materializeOneInstance( >> > CoordCommandUtils.java:546) >> > >> > at >> > org.apache.oozie.command.coord.CoordMaterializeTransitionXCom >> > mand.materializeActions(CoordMaterializeTransitionXCommand.java:492) >> > >> > at >> > org.apache.oozie.command.coord.CoordMaterializeTransitionXCom >> > mand.materialize(CoordMaterializeTransitionXCommand.java:362) >> > >> > at >> > org.apache.oozie.command.MaterializeTransitionXCommand.execute( >> > MaterializeTransitionXCommand.java:73) >> > >> > at >> > org.apache.oozie.command.MaterializeTransitionXCommand.execute( >> > MaterializeTransitionXCommand.java:29) >> > >> > at org.apache.oozie.command.XCommand.call(XCommand.java:290) >> > >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> > >> > at >> > org.apache.oozie.service.CallableQueueService$CallableWrapper.run( >> > CallableQueueService.java:181) >> > >> > at >> > java.util.concurrent.ThreadPoolExecutor.runWorker( >> > ThreadPoolExecutor.java:1149) >> > >> > at >> > java.util.concurrent.ThreadPoolExecutor$Worker.run( >> > ThreadPoolExecutor.java:624) >> > >> > at java.lang.Thread.run(Thread.java:748) >> > >> > >> > >> > Is S3 support specific to CDH distribution or should it work in Apache >> > Oozie as well? I’m not using CDH yet so >> > >> > On Wed, May 16, 2018 at 10:28 AM Peter Cseh <gezap...@cloudera.com> >> wrote: >> > >> > > I think it should be possible for Oozie to poll S3. Check out this >> > > < >> > > https://www.cloudera.com/documentation/enterprise/5-9- >> > x/topics/admin_oozie_s3.html >> > > > >> > > description on how to make it work in jobs, something similar should >> work >> > > on the server side as well >> > > >> > > On Tue, May 15, 2018 at 4:43 PM, purna pradeep < >> purna2prad...@gmail.com> >> > > wrote: >> > > >> > > > Thanks Andras, >> > > > >> > > > Also I also would like to know if oozie supports Aws S3 as input >> events >> > > to >> > > > poll for a dependency file before kicking off a spark action >> > > > >> > > > >> > > > For example: I don’t want to kick off a spark action until a file is >> > > > arrived on a given AWS s3 location >> > > > >> > > > On Tue, May 15, 2018 at 10:17 AM Andras Piros < >> > andras.pi...@cloudera.com >> > > > >> > > > wrote: >> > > > >> > > > > Hi, >> > > > > >> > > > > Oozie needs HDFS to store workflow, coordinator, or bundle >> > definitions, >> > > > as >> > > > > well as sharelib files in a safe, distributed and scalable way. >> Oozie >> > > > needs >> > > > > YARN to run almost all of its actions, Spark action being no >> > exception. >> > > > > >> > > > > At the moment it's not feasible to install Oozie without those >> Hadoop >> > > > > components. How to install Oozie please *find here >> > > > > <https://oozie.apache.org/docs/5.0.0/AG_Install.html>*. >> > > > > >> > > > > Regards, >> > > > > >> > > > > Andras >> > > > > >> > > > > On Tue, May 15, 2018 at 4:11 PM, purna pradeep < >> > > purna2prad...@gmail.com> >> > > > > wrote: >> > > > > >> > > > > > Hi, >> > > > > > >> > > > > > Would like to know if I can use sparkaction in oozie without >> having >> > > > > Hadoop >> > > > > > cluster? >> > > > > > >> > > > > > I want to use oozie to schedule spark jobs on Kubernetes cluster >> > > > > > >> > > > > > I’m a beginner in oozie >> > > > > > >> > > > > > Thanks >> > > > > > >> > > > > >> > > > >> > > >> > > >> > > >> > > -- >> > > *Peter Cseh *| Software Engineer >> > > cloudera.com <https://www.cloudera.com> >> > > >> > > [image: Cloudera] <https://www.cloudera.com/> >> > > >> > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: >> > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: >> > Cloudera >> > > on LinkedIn] <https://www.linkedin.com/company/cloudera> >> > > ------------------------------ >> > > >> > >> >> >> >> -- >> *Peter Cseh *| Software Engineer >> cloudera.com <https://www.cloudera.com> >> >> [image: Cloudera] <https://www.cloudera.com/> >> >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: >> Cloudera >> on LinkedIn] <https://www.linkedin.com/company/cloudera> >> ------------------------------ >> > -- *Peter Cseh *| Software Engineer cloudera.com <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> ------------------------------