Re: Oozie for spark jobs without Hadoop

Ok I fixed this by adding aws keys in oozie But I’m getting below error I have tried setting proxy in core-site.xml but no luck 2018-05-17 15:39:20,602 ERROR CoordInputLogicEvaluatorPhaseOne:517 - SERVER[localhost] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[000-180517144113498-oozie-xjt0-C]

Re: Oozie for spark jobs without Hadoop

Can you try configuring the access keys via environment variables in the server? https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html#Authenticating_via_environment_variables It's possible that we don't propagate the coordinator action's configuration properly to the

Re: Oozie for spark jobs without Hadoop

Ok I got passed this error By rebuilding oozie with Dhttpclient.version=4.5.5 -Dhttpcore.version=4.4.9 now getting this error ACTION[000-180517144113498-oozie-xjt0-C@1] org.apache.oozie.service.HadoopAccessorException: E0902: Exception occurred: [doesBucketExist on

Re: Oozie for spark jobs without Hadoop

Peter, Also When I submit a job with new http client jar, I get ```Error: IO_ERROR : java.io.IOException: Error while connecting Oozie server. No of retries = 1. Exception = Could not authenticate, Authentication failed, status: 500, message: Server Error``` On Thu, May 17, 2018 at 12:14 PM

Re: Oozie for spark jobs without Hadoop

Ok I have tried this It appears that s3a support requires httpclient 4.4.x and oozie is bundled with httpclient 4.3.6. When httpclient is upgraded, the ext UI stops loading. On Thu, May 17, 2018 at 10:28 AM Peter Cseh wrote: > Purna, > > Based on >

Re: Oozie for spark jobs without Hadoop

Purna, Based on https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html#S3 you should try to go for s3a. You'll have to include the aws-jdk as well if I see it correctly: https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html#S3A Also, the property names