This is what is in the logs
2018-05-16 14:06:13,500 INFO URIHandlerService:520 - SERVER[localhost]
Loaded urihandlers [org.apache.oozie.dependency.FSURIHandler]
2018-05-16 14:06:13,501 INFO URIHandlerService:520 - SERVER[localhost]
Loaded default urihandler org.apache.oozie.dependency.FSURIHandler
On Wed, May 16, 2018 at 12:27 PM Peter Cseh <gezap...@cloudera.com> wrote:
> That's strange, this exception should not happen in that case.
> Can you check the server logs for messages like this?
> LOG.info("Loaded urihandlers {0}", Arrays.toString(classes));
> LOG.info("Loaded default urihandler {0}",
> defaultHandler.getClass().getName());
> Thanks
>
> On Wed, May 16, 2018 at 5:47 PM, purna pradeep <purna2prad...@gmail.com>
> wrote:
>
>> This is what I already have in my oozie-site.xml
>>
>> <property>
>>
>>
>> <name>oozie.service.HadoopAccessorService.supported.filesystems</name>
>>
>> <value>*</value>
>>
>> </property>
>>
>> On Wed, May 16, 2018 at 11:37 AM Peter Cseh <gezap...@cloudera.com>
>> wrote:
>>
>>> You'll have to configure
>>> oozie.service.HadoopAccessorService.supported.filesystems
>>> hdfs,hftp,webhdfs Enlist
>>> the different filesystems supported for federation. If wildcard "*" is
>>> specified, then ALL file schemes will be allowed.properly.
>>>
>>> For testing purposes it's ok to put * in there in oozie-site.xml
>>>
>>> On Wed, May 16, 2018 at 5:29 PM, purna pradeep <purna2prad...@gmail.com>
>>> wrote:
>>>
>>> > Peter,
>>> >
>>> > I have tried to specify dataset with uri starting with s3://, s3a://
>>> and
>>> > s3n:// and I am getting exception
>>> >
>>> >
>>> >
>>> > Exception occurred:E0904: Scheme [s3] not supported in uri
>>> > [s3://mybucket/input.data] Making the job failed
>>> >
>>> > org.apache.oozie.dependency.URIHandlerException: E0904: Scheme [s3] not
>>> > supported in uri [s3:// mybucket /input.data]
>>> >
>>> > at
>>> > org.apache.oozie.service.URIHandlerService.getURIHandler(
>>> > URIHandlerService.java:185)
>>> >
>>> > at
>>> > org.apache.oozie.service.URIHandlerService.getURIHandler(
>>> > URIHandlerService.java:168)
>>> >
>>> > at
>>> > org.apache.oozie.service.URIHandlerService.getURIHandler(
>>> > URIHandlerService.java:160)
>>> >
>>> > at
>>> > org.apache.oozie.command.coord.CoordCommandUtils.createEarlyURIs(
>>> > CoordCommandUtils.java:465)
>>> >
>>> > at
>>> > org.apache.oozie.command.coord.CoordCommandUtils.
>>> > separateResolvedAndUnresolved(CoordCommandUtils.java:404)
>>> >
>>> > at
>>> > org.apache.oozie.command.coord.CoordCommandUtils.
>>> > materializeInputDataEvents(CoordCommandUtils.java:731)
>>> >
>>> > at
>>> >
>>> org.apache.oozie.command.coord.CoordCommandUtils.materializeOneInstance(
>>> > CoordCommandUtils.java:546)
>>> >
>>> > at
>>> > org.apache.oozie.command.coord.CoordMaterializeTransitionXCom
>>> > mand.materializeActions(CoordMaterializeTransitionXCommand.java:492)
>>> >
>>> > at
>>> > org.apache.oozie.command.coord.CoordMaterializeTransitionXCom
>>> > mand.materialize(CoordMaterializeTransitionXCommand.java:362)
>>> >
>>> > at
>>> > org.apache.oozie.command.MaterializeTransitionXCommand.execute(
>>> > MaterializeTransitionXCommand.java:73)
>>> >
>>> > at
>>> > org.apache.oozie.command.MaterializeTransitionXCommand.execute(
>>> > MaterializeTransitionXCommand.java:29)
>>> >
>>> > at org.apache.oozie.command.XCommand.call(XCommand.java:290)
>>> >
>>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> >
>>> > at
>>> > org.apache.oozie.service.CallableQueueService$CallableWrapper.run(
>>> > CallableQueueService.java:181)
>>> >
>>> > at
>>> > java.util.concurrent.ThreadPoolExecutor.runWorker(
>>> > ThreadPoolExecutor.java:1149)
>>> >
>>> > at
>>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(
>>> > ThreadPoolExecutor.java:624)
>>> >
>>> > at java.lang.Thread.run(Thread.java:748)
>>> >
>>> >
>>> >
>>> > Is S3 support specific to CDH distribution or should it work in Apache
>>> > Oozie as well? I’m not using CDH yet so
>>> >
>>> > On Wed, May 16, 2018 at 10:28 AM Peter Cseh <gezap...@cloudera.com>
>>> wrote:
>>> >
>>> > > I think it should be possible for Oozie to poll S3. Check out this
>>> > > <
>>> > > https://www.cloudera.com/documentation/enterprise/5-9-
>>> > x/topics/admin_oozie_s3.html
>>> > > >
>>> > > description on how to make it work in jobs, something similar should
>>> work
>>> > > on the server side as well
>>> > >
>>> > > On Tue, May 15, 2018 at 4:43 PM, purna pradeep <
>>> purna2prad...@gmail.com>
>>> > > wrote:
>>> > >
>>> > > > Thanks Andras,
>>> > > >
>>> > > > Also I also would like to know if oozie supports Aws S3 as input
>>> events
>>> > > to
>>> > > > poll for a dependency file before kicking off a spark action
>>> > > >
>>> > > >
>>> > > > For example: I don’t want to kick off a spark action until a file
>>> is
>>> > > > arrived on a given AWS s3 location
>>> > > >
>>> > > > On Tue, May 15, 2018 at 10:17 AM Andras Piros <
>>> > andras.pi...@cloudera.com
>>> > > >
>>> > > > wrote:
>>> > > >
>>> > > > > Hi,
>>> > > > >
>>> > > > > Oozie needs HDFS to store workflow, coordinator, or bundle
>>> > definitions,
>>> > > > as
>>> > > > > well as sharelib files in a safe, distributed and scalable way.
>>> Oozie
>>> > > > needs
>>> > > > > YARN to run almost all of its actions, Spark action being no
>>> > exception.
>>> > > > >
>>> > > > > At the moment it's not feasible to install Oozie without those
>>> Hadoop
>>> > > > > components. How to install Oozie please *find here
>>> > > > > <https://oozie.apache.org/docs/5.0.0/AG_Install.html>*.
>>> > > > >
>>> > > > > Regards,
>>> > > > >
>>> > > > > Andras
>>> > > > >
>>> > > > > On Tue, May 15, 2018 at 4:11 PM, purna pradeep <
>>> > > purna2prad...@gmail.com>
>>> > > > > wrote:
>>> > > > >
>>> > > > > > Hi,
>>> > > > > >
>>> > > > > > Would like to know if I can use sparkaction in oozie without
>>> having
>>> > > > > Hadoop
>>> > > > > > cluster?
>>> > > > > >
>>> > > > > > I want to use oozie to schedule spark jobs on Kubernetes
>>> cluster
>>> > > > > >
>>> > > > > > I’m a beginner in oozie
>>> > > > > >
>>> > > > > > Thanks
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> > >
>>> > >
>>> > > --
>>> > > *Peter Cseh *| Software Engineer
>>> > > cloudera.com <https://www.cloudera.com>
>>> > >
>>> > > [image: Cloudera] <https://www.cloudera.com/>
>>> > >
>>> > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>>> > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>>> > Cloudera
>>> > > on LinkedIn] <https://www.linkedin.com/company/cloudera>
>>> > > ------------------------------
>>> > >
>>> >
>>>
>>>
>>>
>>> --
>>> *Peter Cseh *| Software Engineer
>>> cloudera.com <https://www.cloudera.com>
>>>
>>> [image: Cloudera] <https://www.cloudera.com/>
>>>
>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>>> Cloudera
>>> on LinkedIn] <https://www.linkedin.com/company/cloudera>
>>> ------------------------------
>>>
>>
>
>
> --
> *Peter Cseh *| Software Engineer
> cloudera.com <https://www.cloudera.com>
>
> [image: Cloudera] <https://www.cloudera.com/>
>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> ------------------------------
>
>
