mr-framework directory always empty

2016-11-08 Thread Benjamin Ross
I'm running into an infuriating issue using HDP 2.3.6.0-3796. I've set mapreduce.application.framework.path appropriately, to /hdp/apps/2.3.6.0-3796/mapreduce/mapreduce.tar.gz#mr-framework, and set the mapreduce.application.classpath properly so it pulls in the localized mr-framework. But for

allow all users to decrypt?

2016-11-08 Thread Benjamin Ross
All, I'm in the process of configuring our system for hadoop encryption. We're nearly complete - one of the last issues is that we have a build user that needs to decrypt data to read it from hdfs. The issue is that the build user is an Active Directory user, so the username is DOMAIN\build, r

RE: Authentication Failure talking to Ranger KMS

2016-10-11 Thread Benjamin Ross
re on the right track... Thanks in advance, Ben From: Benjamin Ross Sent: Tuesday, October 11, 2016 12:02 PM To: Wei-Chiu Chuang Cc: user@hadoop.apache.org; u...@ranger.incubator.apache.org Subject: RE: Authentication Failure talking to Ranger KMS That seems

RE: Authentication Failure talking to Ranger KMS

2016-10-11 Thread Benjamin Ross
AM To: Benjamin Ross Cc: user@hadoop.apache.org; u...@ranger.incubator.apache.org Subject: Re: Authentication Failure talking to Ranger KMS Somes to me you encountered this bug? HDFS-10481<https://issues.apache.org/jira/browse/HDFS-10481> If you’re using CDH, this is fixed in CDH5.5.5, C

Authentication Failure talking to Ranger KMS

2016-10-11 Thread Benjamin Ross
All, I'm trying to use httpfs to write to an encryption zone with security off. I can read from an encryption zone, but I can't write to one. Here's the applicable namenode logs. httpfs and root both have all possible privileges in the KMS. What am I missing? 2016-10-07 15:48:16,164 DEBUG i

Help with WebHDFS authentication: simple vs simple-dt

2016-09-27 Thread Benjamin Ross
All, I'm in the process of setting up encryption at rest on a cluster, but I want to make sure that everything else remains permissive - otherwise it will break existing processes that we have in place. I'm very close to getting this working - the last piece is that webhdfs is not permissive:

RE: Help starting RangerKMS

2016-08-26 Thread Benjamin Ross
I actually figured this out. It was pretty simple - the issue was that the ranger KMS master key was generated when the system didn't have JCE installed. So I had to delete the master key and let the system regenerate it. From: Benjamin Ross Sent: F

Help starting RangerKMS

2016-08-26 Thread Benjamin Ross
Hey guys, I'm trying to start the RangerKMS server and I'm running into this very obscure error. Any help would be appreciated. We have confirmed JCE is installed on the node running RangerKMS. We're using Java JDK 1.7 and Ranger 0.5.0.2.3 (HDP 2.3.6.0-3796). [root@bodcdevhdp6 kms]# cat cat

RE: Issue with Hadoop Job History Server

2016-08-18 Thread Benjamin Ross
Turns out we made a stupid mistake - our system was managing to mix configuration between an old cluster and a new cluster. So, things are working now. Thanks, Ben From: Benjamin Ross Sent: Thursday, August 18, 2016 10:05 AM To: Rohith Sharma K S; Gao, Yunlong

RE: Issue with Hadoop Job History Server

2016-08-18 Thread Benjamin Ross
pp.mapreduce.am.resource.mb 6144 yarn.app.mapreduce.am.staging-dir /user Thanks, Ben From: Rohith Sharma K S [ksrohithsha...@gmail.com] Sent: Thursday, August 18, 2016 3:17 AM To: Gao, Yunlong Cc: user@hadoop.apache.org; Benjamin Ross Subject: Re: Is

RE: anyone seen this weird "setXIncludeAware is not supported" error?

2016-07-29 Thread Benjamin Ross
First thought is that I would check to see if you're somehow pulling in a xerces library that your version of Hadoop wasn't built against. Can you provide your pom file? Also, I would do a mvn dependencies:list and see if something looks off. You should probably paste the output of that for ot

RE: Question regarding WebHDFS security

2016-07-05 Thread Benjamin Ross
e the help. Ben From: Larry McCay [lmc...@hortonworks.com] Sent: Tuesday, July 05, 2016 4:23 PM To: Benjamin Ross Cc: David Morel; user@hadoop.apache.org Subject: Re: Question regarding WebHDFS security For consuming REST APIs like webhdfs, where kerberos is inconvenient

RE: Question regarding WebHDFS security

2016-07-05 Thread Benjamin Ross
a possible solution, is there any way to allow unprotected access from just those machines until we can upgrade them? Thanks, Ben From: David Morel [dmo...@amakuru.net] Sent: Tuesday, July 05, 2016 2:33 PM To: Benjamin Ross Cc: user@hadoop.apache.org S

Question regarding WebHDFS security

2016-07-05 Thread Benjamin Ross
All, We're planning the rollout of kerberizing our hadoop cluster. The issue is that we have several single tenant services that rely on contacting the HDFS cluster over WebHDFS without credentials. So, the concern is that once we kerberize the cluster, we will no longer be able to access it w