That error has been resolved. Following is the latest error:
class org.apache.hadoop.hdfs.web.resources.UserProvider class org.apache.hadoop.hdfs.web.resources.ExceptionHandler Jul 28, 2015 10:39:46 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Jul 28, 2015 10:39:51 PM com.sun.jersey.spi.inject.Errors processErrorMessages WARNING: The following warnings have been detected with resource and/or provider classes: WARNING: A sub-resource method, public javax.ws.rs.core.Response org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.getRoot(org.apache.hadoop.security.UserGroupInformation,org.apache.hadoop.hdfs.web.resources.DelegationParam,org.apache.hadoop.hdfs.web.resources.UserParam,org.apache.hadoop.hdfs.web.resources.DoAsParam,org.apache.hadoop.hdfs.web.resources.GetOpParam,org.apache.hadoop.hdfs.web.resources.OffsetParam,org.apache.hadoop.hdfs.web.resources.LengthParam,org.apache.hadoop.hdfs.web.resources.RenewerParam,org.apache.hadoop.hdfs.web.resources.BufferSizeParam,java.util.List,org.apache.hadoop.hdfs.web.resources.XAttrEncodingParam,org.apache.hadoop.hdfs.web.resources.ExcludeDatanodesParam,org.apache.hadoop.hdfs.web.resources.FsActionParam,org.apache.hadoop.hdfs.web.resources.TokenKindParam,org.apache.hadoop.hdfs.web.resources.TokenServiceParam) throws java.io.IOException,java.lang.InterruptedException, with URI template, "/", is treated as a resource method On Wed, Jul 29, 2015 at 12:50 AM, Don Bosco Durai <bo...@apache.org> wrote: > I have seen this issue before in OpenStack environment where the DFS > folders were configured to /tmp/… and they were wiped out after restart. > > If it is a test env, you can try formatting. First check dfs.name.dir > property in HDFS. > > % $HADOOP_INSTALL/hadoop/bin/hadoop namenode -format > > http://wiki.apache.org/hadoop/GettingStartedWithHadoop > Formatting the Namenode > > The first step to starting up your Hadoop installation is formatting the > Hadoop filesystem, which is implemented on top of the local filesystems of > your cluster. You need to do this the first time you set up a Hadoop > installation. *Do not* format a running Hadoop filesystem, this will > cause all your data to be erased. Before formatting, ensure that the > dfs.name.dir directory exists. If you just used the default, then > mkdir -p /tmp/hadoop-username/dfs/name will create the directory. To > format the filesystem (which simply initializes the directory specified by > the dfs.name.dir variable), run the command: > % $HADOOP_INSTALL/hadoop/bin/hadoop namenode -format > > If asked to [re]format, you must reply Y (not just y) if you want to > reformat, else Hadoop will abort the format. > > > Thanks > > Bosco > > From: Aneela Saleem <ane...@platalytics.com> > Reply-To: "user@ranger.incubator.apache.org" < > user@ranger.incubator.apache.org> > Date: Tuesday, July 28, 2015 at 12:40 PM > > To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org> > Subject: Re: Error Setting up Ranger-Admin > > NameNode error Logs: > > 2015-06-30 00:30:05,933 FATAL > org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode. > java.io.IOException: NameNode is not formatted. > > On Wed, Jul 29, 2015 at 12:22 AM, Don Bosco Durai <bo...@apache.org> > wrote: > >> Anything in the namenode log file? >> >> Thanks >> >> Bosco >> >> From: Aneela Saleem <ane...@platalytics.com> >> Reply-To: "user@ranger.incubator.apache.org" < >> user@ranger.incubator.apache.org> >> Date: Tuesday, July 28, 2015 at 12:18 PM >> To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org> >> Subject: Re: Error Setting up Ranger-Admin >> >> Hdfs Plugin request is not being shown in Audit > Plugins tab. I followed >> all steps and verified with your steps. Restarted namenode but still can't >> see any plugin request >> >> On Wed, Jul 29, 2015 at 12:03 AM, Aneela Saleem <ane...@platalytics.com> >> wrote: >> >>> Thank you so much Bosco. I created JIRA for this. Can be found here >>> <https://issues.apache.org/jira/browse/RANGER-582> >>> >>> Let me try UserSync, then i will approach yyou in case of any problem. >>> >>> On Tue, Jul 28, 2015 at 11:28 PM, Don Bosco Durai < >>> bdu...@hortonworks.com> wrote: >>> >>>> Good. I already typed it, so let me paste it for the benefit of others. >>>> We need to update the doc also. If you don’t mind, can you create a JIRA to >>>> track this? >>>> >>>> 1. Go to RangerAdmin UI -> AccessManager (first menu tab is already >>>> opened by default) >>>> 2. Click on “+” in the HDFS panel title >>>> 3. Enter Service Name: e.g. “${cluster_name}_hdfs. Cluster name >>>> could be anything. You can call the repo “test_hdfs” if you want to >>>> 4. UserName: ranger (doesn’t matter whether it exists because you >>>> are not using kerberos) >>>> 5. Password: ranger (doesn’t matter what you give) >>>> 6. Namenode URL: hdfs://<namenode_host>:8020 >>>> 7. Authorization Enabled: No >>>> 8. Authentication Type: Simple >>>> 9. Rest you can leave it as default. >>>> >>>> Save this screen. Then use the same name e.g “test_hdfs” in your >>>> install.properties for enabling Ranger in HDFS. Then restart namenode. >>>> >>>> Is the plugin request showing up in the Audit->Plugin tab? >>>> >>>> If the plugin is showing up, then that means authorization and auditing >>>> is already working. >>>> >>>> For configuring policies, we need to configure UserSync. The >>>> install/configure process is same for all processes. You need to update the >>>> install.properties with the appropriate values. >>>> Set SYNC_SOURCE = ldap >>>> And check the section "Table: LDAP/AD Properties with sample values” >>>> for reference values. >>>> >>>> Please note that for using LDAP within Hadoop, you need to configure >>>> Hadoop core-site.xml to use LDAP. You might be already knowing it. Just >>>> want to make sure. >>>> >>>> Thanks >>>> >>>> Bosco >>>> >>>> >>>> From: Aneela Saleem <ane...@platalytics.com> >>>> Reply-To: "user@ranger.incubator.apache.org" < >>>> user@ranger.incubator.apache.org> >>>> Date: Tuesday, July 28, 2015 at 11:13 AM >>>> >>>> To: "user@ranger.incubator.apache.org" < >>>> user@ranger.incubator.apache.org> >>>> Subject: Re: Error Setting up Ranger-Admin >>>> >>>> Luckily i have created the service myself. Thanks for your timely help. >>>> >>>> Can you please tell me now what should be the next step. I want to >>>> explore all the functionality i.e., >>>> >>>> sync LDAP contacts >>>> perform authorization etc >>>> >>>> On Tue, Jul 28, 2015 at 11:07 PM, Aneela Saleem <ane...@platalytics.com >>>> > wrote: >>>> >>>>> There are no errors in NameNode logs. >>>>> >>>>> Can you please tell me how to create service/repo using Ranger Admin? >>>>> >>>>> On Tue, Jul 28, 2015 at 10:48 PM, Don Bosco Durai <bo...@apache.org> >>>>> wrote: >>>>> >>>>>> Yes, this is link I was about to redirect you to. >>>>>> >>>>>> Sorry, I might have assumed incorrectly on your initial question. >>>>>> ./setup.sh is called only for installing RangerAdmin. Good it seems you >>>>>> have made progress on installing RangerAdmin. >>>>>> >>>>>> I also assume, your ./enable-hdfs-plugin.sh went through fine. >>>>>> >>>>>> Looking into the Apache Ranger document, it seems it doesn’t >>>>>> explicitly call out to create a repo/service for HDFS in RangerAdmin. In >>>>>> the install.properties for HDFS, what value have you given for the >>>>>> property >>>>>> “REPOSITORY_NAME”? You need to create a service/repo in RangerAdmin >>>>>> with the same name. >>>>>> >>>>>> Also, after you restart HDFS namenode, can you check the namenode >>>>>> logs for any error? >>>>>> >>>>>> Thanks >>>>>> >>>>>> Bosco >>>>>> >>>>>> >>>>>> From: Aneela Saleem <ane...@platalytics.com> >>>>>> Reply-To: "user@ranger.incubator.apache.org" < >>>>>> user@ranger.incubator.apache.org> >>>>>> Date: Tuesday, July 28, 2015 at 10:16 AM >>>>>> >>>>>> To: "user@ranger.incubator.apache.org" < >>>>>> user@ranger.incubator.apache.org> >>>>>> Subject: Re: Error Setting up Ranger-Admin >>>>>> >>>>>> And i'm using following link for installation: >>>>>> >>>>>> >>>>>> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation >>>>>> >>>>>> I'm upto *Enabling Ranger HDFS Plugins *section (e) >>>>>> >>>>>> - You can verify by logging into the Ranger Admin Web interface >>>>>> > Audit > Agents. >>>>>> - >>>>>> >>>>>> You can verify the plugin is communicating to Ranger admin in >>>>>> Audit->plugins tab >>>>>> >>>>>> >>>>>> where i don't see any audits ensuring hdfs plugin is communicating to >>>>>> Ranger Admin. >>>>>> >>>>>> On Tue, Jul 28, 2015 at 10:11 PM, Aneela Saleem < >>>>>> ane...@platalytics.com> wrote: >>>>>> >>>>>>> I'm planning to secure HDFS and i'm using Knox for authentication >>>>>>> using REST API. >>>>>>> >>>>>>> Yes i'm still able to access the RangerAdmin UI. I'm just curious >>>>>>> how to sync LDAP contacts in APache Ranger. >>>>>>> >>>>>>> Secondly if the document is related to 0.4 and i'm linked with 0.5, >>>>>>> then how should i proceed? Do you have any fruitful link, i should >>>>>>> follow? >>>>>>> (being a novice) >>>>>>> >>>>>>> On Tue, Jul 28, 2015 at 10:04 PM, Don Bosco Durai <bo...@apache.org> >>>>>>> wrote: >>>>>>> >>>>>>>> Aneela >>>>>>>> >>>>>>>> The document you linked is for Apache Ranger 0.4. There subtle >>>>>>>> changes in Apache Ranger 0.5. >>>>>>>> >>>>>>>> Just curious, how you were able to access RangerAdmin UI? Did the >>>>>>>> installation continued after the install failure? >>>>>>>> >>>>>>>> Also, can you give additional information of your env? I can try >>>>>>>> reproducing it. >>>>>>>> >>>>>>>> Also, which components are you planning secure? (HDFS, Hive, Hbase, >>>>>>>> Solr, YARN, Storm, etc) >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> Bosco >>>>>>>> >>>>>>>> >>>>>>>> From: Aneela Saleem <ane...@platalytics.com> >>>>>>>> Reply-To: "user@ranger.incubator.apache.org" < >>>>>>>> user@ranger.incubator.apache.org> >>>>>>>> Date: Tuesday, July 28, 2015 at 5:19 AM >>>>>>>> To: "user@ranger.incubator.apache.org" < >>>>>>>> user@ranger.incubator.apache.org> >>>>>>>> Subject: Re: Error Setting up Ranger-Admin >>>>>>>> >>>>>>>> And secondly my Ranger UI does not Match the UI as shown in the >>>>>>>> following link: >>>>>>>> >>>>>>>> >>>>>>>> http://pivotalhd.docs.pivotal.io/docs/ranger-user-guide.html#Item1.4.2 >>>>>>>> >>>>>>>> There is no Policy Manager Tab and no Manage Repository Thing. >>>>>>>> >>>>>>>> On Tue, Jul 28, 2015 at 5:17 PM, Aneela Saleem < >>>>>>>> ane...@platalytics.com> wrote: >>>>>>>> >>>>>>>>> Hi Bosco, >>>>>>>>> >>>>>>>>> I did as you suggested but still getting same error. >>>>>>>>> >>>>>>>>> On Tue, Jul 28, 2015 at 4:49 AM, Don Bosco Durai <bo...@apache.org >>>>>>>>> > wrote: >>>>>>>>> >>>>>>>>>> The properties file looks good.. >>>>>>>>>> >>>>>>>>>> I am not sure whether it is because of python 2.7. Is it possible >>>>>>>>>> for you to use python 2.6? >>>>>>>>>> >>>>>>>>>> Try changing the below to where 2.6 is installed. >>>>>>>>>> >>>>>>>>>> PYTHON_COMMAND_INVOKER=python >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> Bosco >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> From: Aneela Saleem <ane...@platalytics.com> >>>>>>>>>> Reply-To: "user@ranger.incubator.apache.org" < >>>>>>>>>> user@ranger.incubator.apache.org> >>>>>>>>>> Date: Monday, July 27, 2015 at 2:02 PM >>>>>>>>>> To: "user@ranger.incubator.apache.org" < >>>>>>>>>> user@ranger.incubator.apache.org> >>>>>>>>>> Subject: Error Setting up Ranger-Admin >>>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> I have followed this link >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation >>>>>>>>>> >>>>>>>>>> But when i run *./setupsh, *i get the following error at the end: >>>>>>>>>> >>>>>>>>>> Traceback (most recent call last): >>>>>>>>>> File "update_property.py", line 40, in <module> >>>>>>>>>> >>>>>>>>>> write_properties_to_xml(ranger_admin_site_xml_path,parameter_name,parameter_value) >>>>>>>>>> File "update_property.py", line 21, in write_properties_to_xml >>>>>>>>>> if(os.path.isfile(xml_path)): >>>>>>>>>> File "/usr/lib/python2.7/genericpath.py", line 29, in isfile >>>>>>>>>> st = os.stat(path) >>>>>>>>>> TypeError: coercing to Unicode: need string or buffer, NoneType >>>>>>>>>> found >>>>>>>>>> 2015-07-28 01:38:57,308 [E] Update property failed for: >>>>>>>>>> >>>>>>>>>> Attached is the *install.proprties* file. >>>>>>>>>> >>>>>>>>>> I'm new to Apache Knox, can anyone please guide me. I'm having >>>>>>>>>> troubles in installing Apache Ranger. >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
