What are the thoughts on hdfsproxy ? http://hadoop.apache.org/hdfs/docs/r0.21.0/hdfsproxy.html
Will this be useful in my scenario ? -----Original Message----- From: Stuti Awasthi Sent: Wednesday, February 15, 2012 12:28 PM To: hdfs-user@hadoop.apache.org Subject: RE: Security in Hadoop-1.0.0 Thanks Harsh, My usecase is : there will be multiple users which will connect to HDFS cluster using an application for all filesystem operation. Normally if user login to app, their permissions on HDFS cluster will be handle by the application front. For any user who wants to connect to hdfs cluster directly without application, I want to provide a security layer using LDAP in between so that only authenticated users can access the cluster. Since there will be n numbers of users, I do not want to add them to unix groups. I hope this clears out my scenario. Thanks for providing links and explaining them in detail. I will read more and try to implement this at my end. Stuti Awasthi -----Original Message----- From: Harsh J [mailto:ha...@cloudera.com] Sent: Tuesday, February 14, 2012 7:14 PM To: hdfs-user@hadoop.apache.org Subject: Re: Security in Hadoop-1.0.0 Stuti, What are you looking for, exactly? Are all you asking for is strong authentication for your HDFS clusters such that no external user may connect to it and read files (even those marked o+r)? If so, that is what a HDFS security configuration, which we have pointed you to already, aims to provide. Know that LDAP isn't an "authentication" mechanism - and thats not what you want to "integrate" HDFS with, for security. You need a functional Kerberos environment that integrates with your LDAP, for strong authentication of users (token based security). To setup Kerberos integrated with your existing LDAP service, please follow articles such as http://www.linux-mag.com/id/4738/ Once your Kerberos instance is setup to talk and authenticate users on your LDAP instance, carry on with the guide pointed out earlier at https://ccp.cloudera.com/display/CDHDOC/Configuring+Hadoop+Security+in+CDH3 - which will essentially work for Apache Hadoop 1.x too. You only need to bother with Kerberos after this point. Hope this clears it up for you. P.s. If your environment already uses Active Directory to manage users, you can use that directly as well: https://ccp.cloudera.com/display/CDHDOC/Integrating+Hadoop+Security+with+Active+Directory P.p.s. The doc page at https://ccp.cloudera.com/display/CDHDOC/CDH3+Security+Guide carries further articles on Kerberos and other security configs if you want to read more - and all of the instructions would work with most upstream releases too. On Tue, Feb 14, 2012 at 1:48 PM, Stuti Awasthi <stutiawas...@hcl.com> wrote: > After some googling I found the following link : > http://mapredit.blogspot.in/2011/10/secure-your-hadoop-cluster-part-i. > html > http://mapredit.blogspot.in/2011/10/secure-your-hadoop-cluster-part-ii > .html > > But these mainly deals with applying LDAP for map-reduce. I want to configure > LDAP for HDFS as well as mapreduce. Please suggest me some links through > which I can configure dfs with LDP also. > > Thanks > > -----Original Message----- > From: Stuti Awasthi > Sent: Tuesday, February 14, 2012 12:28 PM > To: hdfs-user@hadoop.apache.org > Subject: RE: Security in Hadoop-1.0.0 > > Thanks Patrick, > > The concept is clear to me now. As a first step I would like to configure > LDAP with Hadoop. > I am using Apache Hadoop 1.0.0 but not able to find configuration steps in > this version documentation. > It would be really helpful if someone can point me to relevant documentation > of configuring this version of Hadoop with LDAP. > > Thanks > > From: Patrick Angeles [mailto:patrickange...@gmail.com] > Sent: Monday, February 13, 2012 8:29 PM > To: hdfs-user@hadoop.apache.org > Subject: Re: Security in Hadoop-1.0.0 > > LDAP and Kerberos are orthogonal in Hadoop, but both are often used together. > LDAP allows for centralized user/group management (sort of like DNS for your > users). Kerberos is for strong authentication of users. > > When using Kerberos in Hadoop, you want to propagate user/group identities to > all your cluster nodes. (Otherwise, you might authenticate strongly, but your > user ID doesn't exist in a Tasktracker so your job fails.) LDAP happens to be > a common way to do this. > > Typically when you set up Kerberos, you also set up your cluster nodes to do > LDAP authentication. You do this setup at the operating system level (via > PAM). > > Note that you can also use Hue as your user-gateway to Hadoop. In this > scenario, you can use an LDAP backend to authenticate users. You do not have > to (but can) configure Hadoop with Kerberos. > > - P > On Mon, Feb 13, 2012 at 3:11 AM, Stuti Awasthi <stutiawas...@hcl.com> wrote: > Hi, > I am bit confused on Security part of Hadoop. Cluster is behind the firewall. > I have read that Hadoop can be configured with LDAP also. > I want to know which is better : configure Hadoop security with LDAP or > Kerberos as both provide authentication. > > Please provide me more details on this as I am newbee in this part. > > Thanks > > > -----Original Message----- > From: alo alt [mailto:wget.n...@googlemail.com] > Sent: Monday, February 06, 2012 3:56 PM > To: hdfs-user@hadoop.apache.org > Subject: Re: Security in Hadoop-1.0.0 > > Kerberos tokens and lifetime: > http://hortonworks.com/the-role-of-delegation-tokens-in-apache-hadoop- > security/ > > Security in CDH3 (the same as hadoop) > https://ccp.cloudera.com/display/CDHDOC/CDH3+Security+Guide > > best, > Alex > > -- > Alexander Lorenz > http://mapredit.blogspot.com > > On Feb 6, 2012, at 11:19 AM, Stuti Awasthi wrote: > >> Hi all, >> I started looking into configure security in Hadoop-1.0.0 but do not find >> concrete documentation on which kind of security is provided in this release >> and how to configure them. >> Currently I am following >> "http://hadoop.apache.org/common/docs/r1.0.0/"; documentation >> >> As per knowledge, Proxy authentication and Kerberos security is provided in >> this release of Hadoop. Please point me to some good documentation or give >> me some pointers from where I can start this work. >> >> Thanks >> Stuti Awasthi >> >> >> >> ::DISCLAIMER:: >> --------------------------------------------------------------------- >> - >> ------------------------------------------------- >> >> The contents of this e-mail and any attachment(s) are confidential and >> intended for the named recipient(s) only. >> It shall not attach any liability on the originator or HCL or its >> affiliates. Any views or opinions presented in this email are solely those >> of the author and may not necessarily reflect the opinions of HCL or its >> affiliates. >> Any form of reproduction, dissemination, copying, disclosure, >> modification, distribution and / or publication of this message >> without the prior written consent of the author of this e-mail is >> strictly prohibited. If you have received this email in error please delete >> it and notify the sender immediately. Before opening any mail and >> attachments please check them for viruses and defect. >> >> --------------------------------------------------------------------- >> - >> ------------------------------------------------- > -- Harsh J Customer Ops. Engineer Cloudera | http://tiny.cloudera.com/about
