Re: Authentication and security with hadoop

Thanks for the clarification Arpit. Will check the docs. Ravi On Thu, Jul 14, 2016 at 1:44 AM, Arpit Agarwal wrote: > Hi Ravi, > > > > Kerberos is the only supported mechanism for strong identity. Most Hadoop > access controls are easily bypassed without Kerberos authentication. > > > > Kerbero

How to request specific user for running YARN Application master?

Hi, Can YARN client request specific user for running application master? I see that it is being run as "yarn" user OR "nobody" When I add following in yarn-site, it tries to run it as user who submitted yarn client. Is it possible to request specific user for running application master? yarn.no

How to restart an HDFS standby namenode dead for a very long time

Hi - we have an HDFS (version 2.0.0-cdh4.4.0) cluster setup in HA with 2 namenodes and 5 journal nodes. This cluster has been somewhat neglected (long story) and the standby namenode process has been dead for several months. Recently we tried to just start the standby namenode process again, but s

Re: Authentication and security with hadoop

Hi Ravi, Kerberos is the only supported mechanism for strong identity. Most Hadoop access controls are easily bypassed without Kerberos authentication. Kerberos setup can be difficult. Most Kerberos complications arise with multi-homed hosts or if DNS/reverse DNS is broken. If you run into spec

Fwd: FileSystem object's close method is not called. hadoop 2.7.2

Hi, I have implemented an HC file system by extending the org.apache.hadoop.fs.FileSystem class. Things seem to work fine, except the fact that when I run mapreduce jobs with yarn (simple DFSIO tests) I keep noticing that my Filesystem's "close" method is not called. That is, I see in logs that my

Fwd: hadoop statistacally logs out while running MR job in hadoop 2.7.2

Hi, I am having some trouble with running hadoop jobs in Pseudo cluster and in cluster mode in ubuntu 16.04. While running a vanila hadoop/hdfs installation - my hadoop user gets logged out and all of the processes that are run by this user are closed. I don't see anything indicating in logs (/var

Couldn't view logs page in native UI as logged in as super user.

Hi, I am not able to view the logs file in hadoop admin UI,even am logged with super user credentials. Please find the details below: Open native hadoop UI with super user credentials ,I can clearly view the UI page,at same time same super user credentials am not able to view the logs page of nat

Authentication and security with hadoop

Hi Community, We wanted to have authentication on hadoop, means want to make sure the user is what he claims to be and doesn't proxy another users using env variables. >From many links , I see that the default choice is kerberos with hadoop. And as far i understand ,I see that ranger is more like