HI Larry,
Thanks for quick response. the value of contextFactory I missed somehow..
Now I don't see the contextFactory undefined error but .... When I try to
curl the default gateway for webhdfs still I am seeing the same console.
I tried issuing the following curl command with valid TGT in the cache and
after kdestroy and removing the TGT for both I am seeing the same output.
curl -iv -k -u [email protected]:#123Password -X GET "
https://gateway:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS";
also tried
curl -iv -k -X GET "
https://gateway:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS";
I am attaching the default gateway topology file with the email to avoid
lot of texts.
In the *gateway.log *I don't see any entry while hitting the curl
In the *gateway-audit *I see following :
15/12/09 13:44:47
||d96572dd-a988-4392-b7c8-fcf7e1d154f7|audit|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|unavailable|
15/12/09 13:44:48
||d96572dd-a988-4392-b7c8-fcf7e1d154f7|audit|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|success|Response
status: 401
I am not sure what I am missing!!!
*Thank you very much for the help.*
Regards,
DP
*Console Output:*
[root@gateway knox-server]# curl -iv -k -u [email protected]:KnoxPassword@123
-X GET "https://gateway:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS";
* About to connect() to gateway port 8443 (#0)
* Trying 192.168.197.8...
* Connected to gateway (192.168.197.8) port 8443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_DHE_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
* subject:
CN=FQDN_OF_My_gateway_HOST,OU=Test,O=Hadoop,L=Test,ST=Test,C=US
* start date: Nov 27 20:36:22 2015 GMT
* expire date: Nov 26 20:36:22 2016 GMT
* common name: FQDN_OF_My_gateway_HOST
* issuer:
CN=FQDN_OF_My_gateway_HOST,OU=Test,O=Hadoop,L=Test,ST=Test,C=US
* Server auth using Basic with user '[email protected]'
> GET /gateway/default/webhdfs/v1/?op=LISTSTATUS HTTP/1.1
> Authorization: Basic a25veEB0ZXN0LmNvbTojMTIzUGFzc3dvcmQ=
> User-Agent: curl/7.29.0
> Host: gateway:8443
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
HTTP/1.1 401 Unauthorized
* Authentication problem. Ignoring this.
< WWW-Authenticate: BASIC realm="application"
WWW-Authenticate: BASIC realm="application"
< Content-Length: 0
Content-Length: 0
< Server: Jetty(8.1.14.v20131031)
Server: Jetty(8.1.14.v20131031)
<
* Connection #0 to host gateway left intact
On 9 December 2015 at 13:24, larry mccay <[email protected]> wrote:
> I meant the version of the topology that I sent you.
> Note the order of the following to config items:
>
> <param>
> <name>main.ldapContextFactory</name>
>
> <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory</value>
> </param>
>
> <param>
> <name>main.ldapRealm.contextFactory</name>
> <value>$ldapContextFactory</value>
> </param>
>
> Do you have them in that order in the topology that you are using?
>
> On Wed, Dec 9, 2015 at 8:06 AM, Darpan Patel <[email protected]> wrote:
>
>> When we keep :
>>
>> <param>
>> <name>main.ldapRealm.contextFactory</name>
>> <value>$ldapContextFactory</value>
>> </param>
>>
>> in the log I see that the context Factory object is not defined
>> previously and hence could not be referred. Any idea for AD 2008/2012
>> Windows Server what should be the value?
>>
>> I am knox : 0.6.0.2 version.
>>
>> 2015-12-09 12:39:45,185 ERROR env.EnvironmentLoader
>> (EnvironmentLoader.java:initEnvironment(146)) - Shiro environment
>> initialization failed
>> org.apache.shiro.config.UnresolveableReferenceException: The object with
>> id [ldapContextFactory] has not yet been defined and therefore cannot
>> be referenced. Please ensure objects are defined in the order in which
>> they should be created and made available for future reference.
>>
>> Many thanks,
>> DP
>>
>>
>>
>> On 9 December 2015 at 07:58, Darpan Patel <[email protected]> wrote:
>>
>>> Hi Larry,
>>>
>>> I am using the version : 0.6.0.2.3.0.0-2557 of Knox .
>>>
>>>
>>> Checked through curl -u admin:admin-password -i -k
>>> https://localhost:8443/gateway/admin/api/v1/version
>>>
>>>
>>>
>>>
>>> On 8 December 2015 at 23:42, larry mccay <[email protected]> wrote:
>>>
>>>> In the version that I sent you the main.ldapContextFactory is set
>>>> before this entry.
>>>> Is that true in the version that you are using?
>>>>
>>>> On Tue, Dec 8, 2015 at 5:16 PM, Darpan Patel <[email protected]>
>>>> wrote:
>>>>
>>>>> Well when I am keeping the param to the following value we get an
>>>>> error.
>>>>>
>>>>> <param>
>>>>>> <name>main.ldapRealm.contextFactory</name>
>>>>>> <value>$ldapContextFactory</value>
>>>>>> </param>
>>>>>>
>>>>>>
>>>>>
>>>>> Copying from the gateway.log. (It made me think we need to define the
>>>>> value for ldapContextFactory)
>>>>>
>>>>> 2015-12-08 22:13:58,003 ERROR env.EnvironmentLoader
>>>>> (EnvironmentLoader.java:initEnvironment(146)) - Shiro environment
>>>>> initialization failed
>>>>> org.apache.shiro.config.UnresolveableReferenceException: *The object
>>>>> with id [ldapContextFactory] has not yet been defined and therefore cannot
>>>>> be referenced. * Please ensure objects are defined in the order in
>>>>> which they should be created and made available for future reference.
>>>>> at
>>>>> org.apache.shiro.config.ReflectionBuilder.getReferencedObject(ReflectionBuilder.java:224)
>>>>> at
>>>>> org.apache.shiro.config.ReflectionBuilder.resolveReference(ReflectionBuilder.java:239)
>>>>>
>>>>>
>>>>> Regards,
>>>>> DP
>>>>>
>>>>>
>>>>>>
>>>>>> On Tue, Dec 8, 2015 at 4:59 PM, Darpan Patel <[email protected]>
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks for the merged template. I made modifications to it and
>>>>>>>
>>>>>>> I am not sure what value should I fill
>>>>>>> for main.ldapRealm.contextFactory ?
>>>>>>> We are running on windows 2008/2012 Active directory.
>>>>>>>
>>>>>>> <param>
>>>>>>> <name>main.ldapRealm.contextFactory</name>
>>>>>>> <value>$ldapContextFactory</value>
>>>>>>> </param>
>>>>>>>
>>>>>>>
>>>>>> I think that you leave it exactly like that.
>>>>>> It is some sort of shiro injection thing - it references the value
>>>>>> defined above it that way.
>>>>>>
>>>>>>
>>>>>>> I removed this parameter and I see the in the logs:
>>>>>>>
>>>>>>> 2015-12-08 21:56:51,806 ERROR hadoop.gateway
>>>>>>> (KnoxLdapRealm.java:getUserDn(574)) - Failed to get system ldap
>>>>>>> connection:
>>>>>>> javax.naming.AuthenticationException: [LDAP: error code 49 - 80090308:
>>>>>>> LdapErr: DSID-0C0903A9, comment: AcceptSecurityContext error, data 52e,
>>>>>>> v1db1]
>>>>>>>
>>>>>>>
>>>>>>> ( I am happy to see new error after 3 days phew!!!)
>>>>>>>
>>>>>>>
>>>>>> Glad that you are happy but let's getting working and see how you
>>>>>> feel. :)
>>>>>> We'll also roll it into some better documentation for the AD specific
>>>>>> usecase.
>>>>>>
>>>>>>
>>>>>>> Regards,
>>>>>>> DP
>>>>>>>
>>>>>>> On 8 December 2015 at 14:52, Darpan Patel <[email protected]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Thanks Larray.
>>>>>>>> I will check this and update you.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> DP
>>>>>>>>
>>>>>>>> On 8 December 2015 at 12:18, larry mccay <[email protected]> wrote:
>>>>>>>>
>>>>>>>>> Hi Darpan -
>>>>>>>>>
>>>>>>>>> The following topology is probably a better starting point for you
>>>>>>>>> AD configuration - I've tried to merge yours with it as best I can:
>>>>>>>>>
>>>>>>>>> <gateway>
>>>>>>>>> <provider>
>>>>>>>>> <role>authentication</role>
>>>>>>>>> <name>ShiroProvider</name>
>>>>>>>>> <enabled>true</enabled>
>>>>>>>>> <param>
>>>>>>>>> <name>sessionTimeout</name>
>>>>>>>>> <value>30</value>
>>>>>>>>> </param>
>>>>>>>>> <param>
>>>>>>>>> <name>main.ldapRealm</name>
>>>>>>>>>
>>>>>>>>> <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm</value>
>>>>>>>>> </param>
>>>>>>>>>
>>>>>>>>> <param>
>>>>>>>>> <name>main.ldapContextFactory</name>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory</value>
>>>>>>>>> </param>
>>>>>>>>>
>>>>>>>>> <param>
>>>>>>>>> <name>main.ldapRealm.contextFactory</name>
>>>>>>>>> <value>$ldapContextFactory</value>
>>>>>>>>> </param>
>>>>>>>>> <param>
>>>>>>>>> <name>main.ldapRealm.contextFactory.url</name>
>>>>>>>>> <!-- ADJUST host, port for your AD setup-->
>>>>>>>>> <value>ldap://IP_OF_WINDOWS_AD:389</value>
>>>>>>>>> </param>
>>>>>>>>> <!-- ignored due to use of
>>>>>>>>> main.ldapRealm.userSearchAttributeName -->
>>>>>>>>> <param>
>>>>>>>>> <name>main.ldapRealm.userDnTemplate</name>
>>>>>>>>> <value>cn={0},CN=users,DC=test,DC=com</value>
>>>>>>>>> <!-- also tried following values -->
>>>>>>>>> <value>uid={0},CN=users,DC=test,DC=com</value>
>>>>>>>>> <value>cn={0},DC=test,DC=com</value>
>>>>>>>>> </param>
>>>>>>>>>
>>>>>>>>> <!-- Param above is ignored sAMAccount is usually used for
>>>>>>>>> AD -->
>>>>>>>>> <param>
>>>>>>>>> <name>main.ldapRealm.userSearchAttributeName</name>
>>>>>>>>> <value>sAMAccountName</value>
>>>>>>>>> </param>
>>>>>>>>>
>>>>>>>>> <!-- adjust as appropriate -->
>>>>>>>>> <param>
>>>>>>>>> <name>main.ldapRealm.userObjectClass</name>
>>>>>>>>> <value>person</value>
>>>>>>>>> </param>
>>>>>>>>>
>>>>>>>>> <!-- adjust the dn below to match your environment -->
>>>>>>>>> <param>
>>>>>>>>> <name>main.ldapRealm.contextFactory.systemUsername</name>
>>>>>>>>>
>>>>>>>>> <value>cn={systemuser},ou=process,ou=accounts,dc=test,dc=com</value>
>>>>>>>>> </param>
>>>>>>>>>
>>>>>>>>> <!-- should be moved to the credential store for the gateway to be
>>>>>>>>> more secure -->
>>>>>>>>> <param>
>>>>>>>>> <name>main.ldapRealm.contextFactory.systemPassword</name>
>>>>>>>>> <value>{systemuser_password}/value>
>>>>>>>>> </param>
>>>>>>>>>
>>>>>>>>> <!-- let's disable for now since you have no
>>>>>>>>> authorization policies defined anyway -->
>>>>>>>>> <param>
>>>>>>>>> <name>main.ldapRealm.authorizationEnabled</name>
>>>>>>>>> <value>false</value>
>>>>>>>>> </param>
>>>>>>>>>
>>>>>>>>> <param>
>>>>>>>>> <name>main.ldapRealm.searchBase</name>
>>>>>>>>> <value>cn=users,dc=test,dc=com</value>
>>>>>>>>> </param>
>>>>>>>>>
>>>>>>>>> <param>
>>>>>>>>>
>>>>>>>>> <param>
>>>>>>>>>
>>>>>>>>> <name>main.ldapRealm.memberAttributeValueTemplate</name>
>>>>>>>>> <value>cn={0},cn=users,dc=test,dc=com</value>
>>>>>>>>> <!-- also tried uid={0} -->
>>>>>>>>> </param>
>>>>>>>>>
>>>>>>>>> <param>
>>>>>>>>>
>>>>>>>>> <name>main.ldapRealm.contextFactory.authenticationMechanism</name>
>>>>>>>>> <value>simple</value>
>>>>>>>>> </param>
>>>>>>>>>
>>>>>>>>> <param>
>>>>>>>>> <name>urls./**</name>
>>>>>>>>> <value>authcBasic</value>
>>>>>>>>> </param>
>>>>>>>>> </provider>
>>>>>>>>>
>>>>>>>>> <!-- the group principal mapping below is not likely what
>>>>>>>>> you want
>>>>>>>>> note that mapping of the hdfs group to admin. Also, we have
>>>>>>>>> disabled authorization above so there is no need for
>>>>>>>>> groups -->
>>>>>>>>> <provider>
>>>>>>>>> <role>identity-assertion</role>
>>>>>>>>> <name>Default</name>
>>>>>>>>> <enabled>true</enabled>
>>>>>>>>> <!--param>
>>>>>>>>> <name>group.principal.mapping</name>
>>>>>>>>> <value>*=users;hdfs=admin</value>
>>>>>>>>> </param-->
>>>>>>>>> </provider>
>>>>>>>>>
>>>>>>>>> <provider>
>>>>>>>>> <role>authorization</role>
>>>>>>>>> <name>AclsAuthz</name>
>>>>>>>>> <enabled>true</enabled>
>>>>>>>>> </provider>
>>>>>>>>>
>>>>>>>>> </gateway>
>>>>>>>>>
>>>>>>>>> We need to better document the difference between LDAP and AD for
>>>>>>>>> such deployments.
>>>>>>>>>
>>>>>>>>> I've also tried to document some of the changes that I made.
>>>>>>>>> Note that you don't have any authorization ACLs defined in the
>>>>>>>>> AclsAuthz provider so I disabled group lookup.
>>>>>>>>> That will only add complexity to your config - we can re-enable
>>>>>>>>> once authentication is working.
>>>>>>>>>
>>>>>>>>> Please go through this config and ensure that DNs, host and ports
>>>>>>>>> and system usernames match your environment.
>>>>>>>>>
>>>>>>>>> Hope this helps.
>>>>>>>>>
>>>>>>>>> --larry
>>>>>>>>>
>>>>>>>>> On Tue, Dec 8, 2015 at 5:16 AM, Darpan Patel <[email protected]>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi All,
>>>>>>>>>>
>>>>>>>>>> For this blocker issue let more information if it can help fixing
>>>>>>>>>> the authorization problem.
>>>>>>>>>> Please let me know if more details required.
>>>>>>>>>> (+ dev list)
>>>>>>>>>>
>>>>>>>>>> */etc/krb5.conf*
>>>>>>>>>>
>>>>>>>>>> [libdefaults]
>>>>>>>>>> renew_lifetime = 7d
>>>>>>>>>> forwardable = true
>>>>>>>>>> default_realm = HORTONWORKS.COM
>>>>>>>>>> ticket_lifetime = 24h
>>>>>>>>>> dns_lookup_realm = false
>>>>>>>>>> dns_lookup_kdc = false
>>>>>>>>>> #default_tgs_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5
>>>>>>>>>> #default_tkt_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5
>>>>>>>>>>
>>>>>>>>>> [domain_realm]
>>>>>>>>>> .hortonworks.com = HORTONWORKS.COM
>>>>>>>>>> HORTONWORKS.COm = HORTONWORKS.COM
>>>>>>>>>>
>>>>>>>>>> [logging]
>>>>>>>>>> default = FILE:/var/log/krb5kdc.log
>>>>>>>>>> admin_server = FILE:/var/log/kadmind.log
>>>>>>>>>> kdc = FILE:/var/log/krb5kdc.log
>>>>>>>>>>
>>>>>>>>>> [realms]
>>>>>>>>>> HORTONWORKS.COM = {
>>>>>>>>>> admin_server = KDC_SERVER_HOST
>>>>>>>>>> kdc = KDC_SERVER_HOST
>>>>>>>>>> }
>>>>>>>>>> *TEST.COM <http://TEST.COM>* = {
>>>>>>>>>> admin_server = WINDOWS_12_SERVER_AD_HOST
>>>>>>>>>> kdc = WINDOWS_12_SERVER_AD_HOST
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> */usr/hdp/current/knox-server/conf/gateway-site.xml*
>>>>>>>>>>
>>>>>>>>>> <configuration>
>>>>>>>>>> <property>
>>>>>>>>>> <name>*gateway.gateway.conf.dir*</name>
>>>>>>>>>> <value>deployments</value>
>>>>>>>>>> </property>
>>>>>>>>>> <property>
>>>>>>>>>> <name>*gateway.hadoop.kerberos.secured*</name>
>>>>>>>>>> <value>true</value>
>>>>>>>>>> </property>
>>>>>>>>>> <property>
>>>>>>>>>> <name>*gateway.path*</name>
>>>>>>>>>> <value>gateway</value>
>>>>>>>>>> </property>
>>>>>>>>>> <property>
>>>>>>>>>> <name>*gateway.port*</name>
>>>>>>>>>> <value>8443</value>
>>>>>>>>>> </property>
>>>>>>>>>> <property>
>>>>>>>>>> <name>*java.security.auth.login.config*</name>
>>>>>>>>>> <value>/*etc/knox/conf/krb5JAASLogin.conf*</value>
>>>>>>>>>> </property>
>>>>>>>>>> <property>
>>>>>>>>>> <name>*java.security.krb5.conf*</name>
>>>>>>>>>> <value>*/etc/krb5.conf*</value>
>>>>>>>>>> </property>
>>>>>>>>>> <property>
>>>>>>>>>> <name>sun.security.krb5.debug</name>
>>>>>>>>>> <value>true</value>
>>>>>>>>>> </property>
>>>>>>>>>> </configuration>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> */etc/knox/conf/krb5JAASLogin.conf*
>>>>>>>>>>
>>>>>>>>>> com.sun.security.jgss.initiate {
>>>>>>>>>> com.sun.security.auth.module.Krb5LoginModule required
>>>>>>>>>> renewTGT=true
>>>>>>>>>> doNotPrompt=true
>>>>>>>>>> useKeyTab=true
>>>>>>>>>> keyTab="/etc/security/keytabs/knox.service.keytab"
>>>>>>>>>> principal="knox/[email protected]"
>>>>>>>>>> isInitiator=true
>>>>>>>>>> storeKey=true
>>>>>>>>>> useTicketCache=true
>>>>>>>>>> client=true;
>>>>>>>>>> };
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> DP
>>>>>>>>>>
>>>>>>>>>> ---------- Forwarded message ----------
>>>>>>>>>> From: Darpan Patel <[email protected]>
>>>>>>>>>> Date: 7 December 2015 at 17:59
>>>>>>>>>> Subject: Need help setting up Knox for A/D integrated Kerberized
>>>>>>>>>> Cluster
>>>>>>>>>> To: [email protected]
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hi All,
>>>>>>>>>>
>>>>>>>>>> I am stuck on an issue from last two days. I would be really
>>>>>>>>>> grateful if someone can help on this.
>>>>>>>>>>
>>>>>>>>>> We have HDP 2.3 implemented over 8 node cluster and the same
>>>>>>>>>> cluster has been Kerberized and later on we have integrated it with
>>>>>>>>>> Active
>>>>>>>>>> Directory (Which runs in the same VPN). We also verified that
>>>>>>>>>> Windows 2012
>>>>>>>>>> A/D integration with Ranger works fine for defining policies and
>>>>>>>>>> audit log.
>>>>>>>>>> But I am stuck at Knox bit. I am trying to replicate the same
>>>>>>>>>> configuration
>>>>>>>>>> properties which I have set for Ranger LDAP-AD Integration.
>>>>>>>>>>
>>>>>>>>>> I am taking reference of the Hortonworks documentation and also
>>>>>>>>>> Apache Knox documentation.
>>>>>>>>>>
>>>>>>>>>> The A/D domain name is TEST.COM and all the users are under
>>>>>>>>>> Users
>>>>>>>>>>
>>>>>>>>>> [image: Inline images 1]
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Under the Users we have few users one of the them is knox,
>>>>>>>>>> darpan, test,etc.
>>>>>>>>>>
>>>>>>>>>> When we issue following command on the node on which Knox Server
>>>>>>>>>> is running (topology name is default)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *curl -iv -k -u [email protected]:#123Password -X GET
>>>>>>>>>> "https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS
>>>>>>>>>> <https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS>"
>>>>>>>>>> OR*
>>>>>>>>>>
>>>>>>>>>> *curl -iv -k -u knox:#123Password -X GET
>>>>>>>>>> "https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS
>>>>>>>>>> <https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS>"*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Every time I see < HTTP/1.1 401 Unauthorized HTTP/1.1 401
>>>>>>>>>> Unauthorized on the console.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Entries in the *gateway-audit.log *are like this :
>>>>>>>>>>
>>>>>>>>>> gateway-audit.log
>>>>>>>>>> ==================
>>>>>>>>>> 15/12/07 17:11:08
>>>>>>>>>> ||38606993-17e2-4c3e-ad4b-e3faea293aae|audit|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|unavailable|
>>>>>>>>>> 15/12/07 17:11:09
>>>>>>>>>> ||38606993-17e2-4c3e-ad4b-e3faea293aae|audit|WEBHDFS||||authentication|
>>>>>>>>>> *principal*|*[email protected] <[email protected]>*|failure|*LDAP
>>>>>>>>>> authentication failed.*
>>>>>>>>>> 15/12/07 17:11:09
>>>>>>>>>> ||38606993-17e2-4c3e-ad4b-e3faea293aae|audit|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|success|Response
>>>>>>>>>> status: 401
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 15/12/07 17:05:28
>>>>>>>>>> ||5b436e43-b874-40f7-b111-7b262fe5125d|audit|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|unavailable|
>>>>>>>>>> 15/12/07 17:05:29
>>>>>>>>>> ||5b436e43-b874-40f7-b111-7b262fe5125d|audit|WEBHDFS||||authentication|
>>>>>>>>>> *principal*|knox|failure|*LDAP authentication failed.*
>>>>>>>>>> 15/12/07 17:05:29
>>>>>>>>>> ||5b436e43-b874-40f7-b111-7b262fe5125d|audit|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|success|Response
>>>>>>>>>> status: 401
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *Gateway.log*
>>>>>>>>>> *===========*
>>>>>>>>>> 2015-12-07 17:05:28,620 INFO hadoop.gateway
>>>>>>>>>> (KnoxLdapRealm.java:getUserDn(550)) - Computed userDn:
>>>>>>>>>> cn=knox,CN=users,DC=test,DC=com using dnTemplate for principal: knox
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Following is the part of our *default.xml *topology:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> <gateway>
>>>>>>>>>> <provider>
>>>>>>>>>> <role>authentication</role>
>>>>>>>>>> <name>ShiroProvider</name>
>>>>>>>>>> <enabled>true</enabled>
>>>>>>>>>> <param>
>>>>>>>>>> <name>sessionTimeout</name>
>>>>>>>>>> <value>30</value>
>>>>>>>>>> </param>
>>>>>>>>>> <param>
>>>>>>>>>> <name>*main.ldapRealm*</name>
>>>>>>>>>>
>>>>>>>>>> <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm</value>
>>>>>>>>>> </param>
>>>>>>>>>>
>>>>>>>>>> <param>
>>>>>>>>>> <name>*main.ldapContextFactory*</name>
>>>>>>>>>>
>>>>>>>>>> <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory</value>
>>>>>>>>>> </param>
>>>>>>>>>>
>>>>>>>>>> <param>
>>>>>>>>>> <name>*main.ldapRealm.userDnTemplate*
>>>>>>>>>> </name>
>>>>>>>>>>
>>>>>>>>>> <value>cn={0},CN=users,DC=test,DC=com</value>
>>>>>>>>>> <!-- also tried following values -->
>>>>>>>>>>
>>>>>>>>>> <value>uid={0},CN=users,DC=test,DC=com</value>
>>>>>>>>>> <value>cn={0},DC=test,DC=com</value>
>>>>>>>>>> </param>
>>>>>>>>>> <param>
>>>>>>>>>> <name>*main.ldapRealm.contextFactory.url*
>>>>>>>>>> </name>
>>>>>>>>>> <!-- IP Address of the WINDOSWS 2012
>>>>>>>>>> Acive Directory Server which works for Ranger -->
>>>>>>>>>> <value>*ldap://IP_OF_WINDOWS_AD:389*
>>>>>>>>>> </value>
>>>>>>>>>> </param>
>>>>>>>>>> <param>
>>>>>>>>>> <name>*main.ldapRealm.authorizationEnabled*
>>>>>>>>>> </name>
>>>>>>>>>> <value>true</value>
>>>>>>>>>> </param>
>>>>>>>>>> <param>
>>>>>>>>>> <name>*main.ldapRealm.searchBase*</name>
>>>>>>>>>> <value>cn=users,dc=test,dc=com</value>
>>>>>>>>>> </param>
>>>>>>>>>> <param>
>>>>>>>>>> <param>
>>>>>>>>>> <name>
>>>>>>>>>> *main.ldapRealm.memberAttributeValueTemplate*</name>
>>>>>>>>>>
>>>>>>>>>> <value>cn={0},cn=users,dc=test,dc=com</value>
>>>>>>>>>> <!-- also tried uid={0} -->
>>>>>>>>>> </param>
>>>>>>>>>> <param>
>>>>>>>>>> <name>
>>>>>>>>>> *main.ldapRealm.contextFactory.authenticationMechanism<*/name>
>>>>>>>>>> <value>simple</value>
>>>>>>>>>> </param>
>>>>>>>>>> <param>
>>>>>>>>>> <name>urls./**</name>
>>>>>>>>>> <value>authcBasic</value>
>>>>>>>>>> </param>
>>>>>>>>>> </provider>
>>>>>>>>>>
>>>>>>>>>> <provider>
>>>>>>>>>> <role>*identity-assertion*</role>
>>>>>>>>>> <name>Default</name>
>>>>>>>>>> <enabled>true</enabled>
>>>>>>>>>> <param>
>>>>>>>>>> <name>*group.principal.mapping*</name>
>>>>>>>>>> <value>*=users;hdfs=admin</value>
>>>>>>>>>> </param>
>>>>>>>>>> </provider>
>>>>>>>>>>
>>>>>>>>>> <provider>
>>>>>>>>>> <role>*authorization*</role>
>>>>>>>>>> <name>AclsAuthz</name>
>>>>>>>>>> <enabled>true</enabled>
>>>>>>>>>> </provider>
>>>>>>>>>>
>>>>>>>>>> </gateway>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> And following is the console output while trying to access
>>>>>>>>>> webhdfs using curl
>>>>>>>>>>
>>>>>>>>>> curl -iv -k -u knox:#123Password -X GET "
>>>>>>>>>> https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS";
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *Console Output:*
>>>>>>>>>> ----------------
>>>>>>>>>>
>>>>>>>>>> * About to connect() to localhost port 8443 (#0)
>>>>>>>>>> * Trying ::1...
>>>>>>>>>> * Connected to localhost (::1) port 8443 (#0)
>>>>>>>>>> * Initializing NSS with certpath: sql:/etc/pki/nssdb
>>>>>>>>>> * skipping SSL peer certificate verification
>>>>>>>>>> * SSL connection using TLS_DHE_RSA_WITH_AES_128_CBC_SHA
>>>>>>>>>> * Server certificate:
>>>>>>>>>> * subject:
>>>>>>>>>> CN=FQDN_OF_My_gateway_HOST,OU=Test,O=Hadoop,L=Test,ST=Test,C=US
>>>>>>>>>> * start date: Nov 27 20:36:22 2015 GMT
>>>>>>>>>> * expire date: Nov 26 20:36:22 2016 GMT
>>>>>>>>>> * common name: FQDN_OF_My_gateway_HOST
>>>>>>>>>> * issuer:
>>>>>>>>>> CN=FQDN_OF_My_gateway_HOST,OU=Test,O=Hadoop,L=Test,ST=Test,C=US
>>>>>>>>>> * Server auth using Basic with user 'knox'
>>>>>>>>>> > GET /gateway/default/webhdfs/v1/?op=LISTSTATUS HTTP/1.1
>>>>>>>>>> > Authorization: Basic a25veDojMTIzUGFzc3dvcmQ=
>>>>>>>>>> > User-Agent: curl/7.29.0
>>>>>>>>>> > Host: localhost:8443
>>>>>>>>>> > Accept: */*
>>>>>>>>>> >
>>>>>>>>>> < HTTP/1.1 401 Unauthorized
>>>>>>>>>> HTTP/1.1 401 Unauthorized
>>>>>>>>>> * Authentication problem. Ignoring this.
>>>>>>>>>> < WWW-Authenticate: BASIC realm="application"
>>>>>>>>>> WWW-Authenticate: BASIC realm="application"
>>>>>>>>>> < Content-Length: 0
>>>>>>>>>> Content-Length: 0
>>>>>>>>>> < Server: Jetty(8.1.14.v20131031)
>>>>>>>>>> Server: Jetty(8.1.14.v20131031)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Please let me know if any additional information is required.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> DP
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
<topology>
<gateway>
<provider>
<role>authentication</role>
<name>ShiroProvider</name>
<enabled>true</enabled>
<param>
<name>sessionTimeout</name>
<value>30</value>
</param>
<param>
<name>main.ldapRealm</name>
<value>org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm</value>
</param>
<param>
<name>main.ldapContextFactory</name>
<value>org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory</value>
</param>
<param>
<name>main.ldapRealm.contextFactory</name>
<value>$ldapContextFactory</value>
</param>
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://AD_HOST:389</value>
<!-- Active Directory Host IP -->
</param>
<param>
<name>main.ldapRealm.userSearchAttributeName</name>
<value>sAMAccountName</value>
</param>
<param>
<name>main.ldapRealm.userObjectClass</name>
<value>person</value>
</param>
<!-- Not sure if the CN is correct. I've a user named "Knox Knox" id [email protected] in the AD -->
<!-- Tried with cn=knox,ou=accounts,dc=test,dc=com did not work -->
<!-- Also tried with cn=knox,cn=Users,dc=test,dc=com did not work -->
<param>
<name>main.ldapRealm.contextFactory.systemUsername</name>
<value>CN=Knox Knox,CN=Users,DC=test,DC=com</value>
</param>
<param>
<name>main.ldapRealm.contextFactory.systemPassword</name>
<value>KnoxPassword@123</value>
</param>
<param>
<name>main.ldapRealm.contextFactory.authenticationMechanism</name>
<value>simple</value>
</param>
<param>
<name>urls./**</name>
<value>authcBasic</value>
</param>
<param>
<name>main.ldapRealm.authorizationEnabled</name>
<value>false</value>
</param>
<param>
<name>main.ldapRealm.searchBase</name>
<value>CN=Users,DC=test,DC=com</value>
</param>
<param>
<name>main.ldapRealm.memberAttributeValueTemplate</name>
<value>cn={0},cn=Users,dc=test,dc=com</value>
</param>
</provider>
<provider>
<role>identity-assertion</role>
<name>Default</name>
<enabled>true</enabled>
</provider>
<provider>
<role>authorization</role>
<name>AclsAuthz</name>
</provider>
</gateway>
