Thanks Larray. I will check this and update you. Regards, DP
On 8 December 2015 at 12:18, larry mccay <[email protected]> wrote: > Hi Darpan - > > The following topology is probably a better starting point for you AD > configuration - I've tried to merge yours with it as best I can: > > <gateway> > <provider> > <role>authentication</role> > <name>ShiroProvider</name> > <enabled>true</enabled> > <param> > <name>sessionTimeout</name> > <value>30</value> > </param> > <param> > <name>main.ldapRealm</name> > > <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm</value> > </param> > > <param> > <name>main.ldapContextFactory</name> > > <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory</value> > </param> > > <param> > <name>main.ldapRealm.contextFactory</name> > <value>$ldapContextFactory</value> > </param> > <param> > <name>main.ldapRealm.contextFactory.url</name> > <!-- ADJUST host, port for your AD setup--> > <value>ldap://IP_OF_WINDOWS_AD:389</value> > </param> > <!-- ignored due to use of > main.ldapRealm.userSearchAttributeName --> > <param> > <name>main.ldapRealm.userDnTemplate</name> > <value>cn={0},CN=users,DC=test,DC=com</value> > <!-- also tried following values --> > <value>uid={0},CN=users,DC=test,DC=com</value> > <value>cn={0},DC=test,DC=com</value> > </param> > > <!-- Param above is ignored sAMAccount is usually used for AD --> > > <param> > <name>main.ldapRealm.userSearchAttributeName</name> > <value>sAMAccountName</value> > </param> > > <!-- adjust as appropriate --> > <param> > <name>main.ldapRealm.userObjectClass</name> > <value>person</value> > </param> > > <!-- adjust the dn below to match your environment --> > <param> > <name>main.ldapRealm.contextFactory.systemUsername</name> > <value>cn={systemuser},ou=process,ou=accounts,dc=test,dc=com</value> > </param> > > <!-- should be moved to the credential store for the gateway to be more > secure --> > <param> > <name>main.ldapRealm.contextFactory.systemPassword</name> > <value>{systemuser_password}/value> > </param> > > <!-- let's disable for now since you have no authorization > policies defined anyway --> > <param> > <name>main.ldapRealm.authorizationEnabled</name> > <value>false</value> > </param> > > <param> > <name>main.ldapRealm.searchBase</name> > <value>cn=users,dc=test,dc=com</value> > </param> > > <param> > > <param> > <name>main.ldapRealm.memberAttributeValueTemplate</name> > <value>cn={0},cn=users,dc=test,dc=com</value> > <!-- also tried uid={0} --> > </param> > > <param> > > <name>main.ldapRealm.contextFactory.authenticationMechanism</name> > <value>simple</value> > </param> > > <param> > <name>urls./**</name> > <value>authcBasic</value> > </param> > </provider> > > <!-- the group principal mapping below is not likely what you want > note that mapping of the hdfs group to admin. Also, we have > disabled authorization above so there is no need for groups > --> > <provider> > <role>identity-assertion</role> > <name>Default</name> > <enabled>true</enabled> > <!--param> > <name>group.principal.mapping</name> > <value>*=users;hdfs=admin</value> > </param--> > </provider> > > <provider> > <role>authorization</role> > <name>AclsAuthz</name> > <enabled>true</enabled> > </provider> > > </gateway> > > We need to better document the difference between LDAP and AD for such > deployments. > > I've also tried to document some of the changes that I made. > Note that you don't have any authorization ACLs defined in the AclsAuthz > provider so I disabled group lookup. > That will only add complexity to your config - we can re-enable once > authentication is working. > > Please go through this config and ensure that DNs, host and ports and > system usernames match your environment. > > Hope this helps. > > --larry > > On Tue, Dec 8, 2015 at 5:16 AM, Darpan Patel <[email protected]> wrote: > >> Hi All, >> >> For this blocker issue let more information if it can help fixing the >> authorization problem. >> Please let me know if more details required. >> (+ dev list) >> >> */etc/krb5.conf* >> >> [libdefaults] >> renew_lifetime = 7d >> forwardable = true >> default_realm = HORTONWORKS.COM >> ticket_lifetime = 24h >> dns_lookup_realm = false >> dns_lookup_kdc = false >> #default_tgs_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5 >> #default_tkt_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5 >> >> [domain_realm] >> .hortonworks.com = HORTONWORKS.COM >> HORTONWORKS.COm = HORTONWORKS.COM >> >> [logging] >> default = FILE:/var/log/krb5kdc.log >> admin_server = FILE:/var/log/kadmind.log >> kdc = FILE:/var/log/krb5kdc.log >> >> [realms] >> HORTONWORKS.COM = { >> admin_server = KDC_SERVER_HOST >> kdc = KDC_SERVER_HOST >> } >> *TEST.COM <http://TEST.COM>* = { >> admin_server = WINDOWS_12_SERVER_AD_HOST >> kdc = WINDOWS_12_SERVER_AD_HOST >> } >> >> >> */usr/hdp/current/knox-server/conf/gateway-site.xml* >> >> <configuration> >> <property> >> <name>*gateway.gateway.conf.dir*</name> >> <value>deployments</value> >> </property> >> <property> >> <name>*gateway.hadoop.kerberos.secured*</name> >> <value>true</value> >> </property> >> <property> >> <name>*gateway.path*</name> >> <value>gateway</value> >> </property> >> <property> >> <name>*gateway.port*</name> >> <value>8443</value> >> </property> >> <property> >> <name>*java.security.auth.login.config*</name> >> <value>/*etc/knox/conf/krb5JAASLogin.conf*</value> >> </property> >> <property> >> <name>*java.security.krb5.conf*</name> >> <value>*/etc/krb5.conf*</value> >> </property> >> <property> >> <name>sun.security.krb5.debug</name> >> <value>true</value> >> </property> >> </configuration> >> >> >> */etc/knox/conf/krb5JAASLogin.conf* >> >> com.sun.security.jgss.initiate { >> com.sun.security.auth.module.Krb5LoginModule required >> renewTGT=true >> doNotPrompt=true >> useKeyTab=true >> keyTab="/etc/security/keytabs/knox.service.keytab" >> principal="knox/[email protected]" >> isInitiator=true >> storeKey=true >> useTicketCache=true >> client=true; >> }; >> >> Regards, >> DP >> >> ---------- Forwarded message ---------- >> From: Darpan Patel <[email protected]> >> Date: 7 December 2015 at 17:59 >> Subject: Need help setting up Knox for A/D integrated Kerberized Cluster >> To: [email protected] >> >> >> Hi All, >> >> I am stuck on an issue from last two days. I would be really grateful if >> someone can help on this. >> >> We have HDP 2.3 implemented over 8 node cluster and the same cluster has >> been Kerberized and later on we have integrated it with Active Directory >> (Which runs in the same VPN). We also verified that Windows 2012 A/D >> integration with Ranger works fine for defining policies and audit log. But >> I am stuck at Knox bit. I am trying to replicate the same configuration >> properties which I have set for Ranger LDAP-AD Integration. >> >> I am taking reference of the Hortonworks documentation and also Apache >> Knox documentation. >> >> The A/D domain name is TEST.COM and all the users are under Users >> >> [image: Inline images 1] >> >> >> Under the Users we have few users one of the them is knox, darpan, >> test,etc. >> >> When we issue following command on the node on which Knox Server is >> running (topology name is default) >> >> >> >> *curl -iv -k -u [email protected]:#123Password -X GET >> "https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS >> <https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS>" OR* >> >> *curl -iv -k -u knox:#123Password -X GET >> "https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS >> <https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS>"* >> >> >> Every time I see < HTTP/1.1 401 Unauthorized HTTP/1.1 401 Unauthorized >> on the console. >> >> >> Entries in the *gateway-audit.log *are like this : >> >> gateway-audit.log >> ================== >> 15/12/07 17:11:08 >> ||38606993-17e2-4c3e-ad4b-e3faea293aae|audit|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|unavailable| >> 15/12/07 17:11:09 >> ||38606993-17e2-4c3e-ad4b-e3faea293aae|audit|WEBHDFS||||authentication| >> *principal*|*[email protected] <[email protected]>*|failure|*LDAP authentication >> failed.* >> 15/12/07 17:11:09 >> ||38606993-17e2-4c3e-ad4b-e3faea293aae|audit|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|success|Response >> status: 401 >> >> >> 15/12/07 17:05:28 >> ||5b436e43-b874-40f7-b111-7b262fe5125d|audit|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|unavailable| >> 15/12/07 17:05:29 >> ||5b436e43-b874-40f7-b111-7b262fe5125d|audit|WEBHDFS||||authentication| >> *principal*|knox|failure|*LDAP authentication failed.* >> 15/12/07 17:05:29 >> ||5b436e43-b874-40f7-b111-7b262fe5125d|audit|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|success|Response >> status: 401 >> >> >> *Gateway.log* >> *===========* >> 2015-12-07 17:05:28,620 INFO hadoop.gateway >> (KnoxLdapRealm.java:getUserDn(550)) - Computed userDn: >> cn=knox,CN=users,DC=test,DC=com using dnTemplate for principal: knox >> >> >> Following is the part of our *default.xml *topology: >> >> >> <gateway> >> <provider> >> <role>authentication</role> >> <name>ShiroProvider</name> >> <enabled>true</enabled> >> <param> >> <name>sessionTimeout</name> >> <value>30</value> >> </param> >> <param> >> <name>*main.ldapRealm*</name> >> >> <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm</value> >> </param> >> >> <param> >> <name>*main.ldapContextFactory*</name> >> >> <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory</value> >> </param> >> >> <param> >> <name>*main.ldapRealm.userDnTemplate*</name> >> <value>cn={0},CN=users,DC=test,DC=com</value> >> <!-- also tried following values --> >> <value>uid={0},CN=users,DC=test,DC=com</value> >> <value>cn={0},DC=test,DC=com</value> >> </param> >> <param> >> <name>*main.ldapRealm.contextFactory.url*</name> >> <!-- IP Address of the WINDOSWS 2012 Acive >> Directory Server which works for Ranger --> >> <value>*ldap://IP_OF_WINDOWS_AD:389*</value> >> </param> >> <param> >> <name>*main.ldapRealm.authorizationEnabled*</name> >> <value>true</value> >> </param> >> <param> >> <name>*main.ldapRealm.searchBase*</name> >> <value>cn=users,dc=test,dc=com</value> >> </param> >> <param> >> <param> >> <name>*main.ldapRealm.memberAttributeValueTemplate* >> </name> >> <value>cn={0},cn=users,dc=test,dc=com</value> >> <!-- also tried uid={0} --> >> </param> >> <param> >> <name> >> *main.ldapRealm.contextFactory.authenticationMechanism<*/name> >> <value>simple</value> >> </param> >> <param> >> <name>urls./**</name> >> <value>authcBasic</value> >> </param> >> </provider> >> >> <provider> >> <role>*identity-assertion*</role> >> <name>Default</name> >> <enabled>true</enabled> >> <param> >> <name>*group.principal.mapping*</name> >> <value>*=users;hdfs=admin</value> >> </param> >> </provider> >> >> <provider> >> <role>*authorization*</role> >> <name>AclsAuthz</name> >> <enabled>true</enabled> >> </provider> >> >> </gateway> >> >> >> And following is the console output while trying to access webhdfs using >> curl >> >> curl -iv -k -u knox:#123Password -X GET " >> https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS"; >> >> >> *Console Output:* >> ---------------- >> >> * About to connect() to localhost port 8443 (#0) >> * Trying ::1... >> * Connected to localhost (::1) port 8443 (#0) >> * Initializing NSS with certpath: sql:/etc/pki/nssdb >> * skipping SSL peer certificate verification >> * SSL connection using TLS_DHE_RSA_WITH_AES_128_CBC_SHA >> * Server certificate: >> * subject: >> CN=FQDN_OF_My_gateway_HOST,OU=Test,O=Hadoop,L=Test,ST=Test,C=US >> * start date: Nov 27 20:36:22 2015 GMT >> * expire date: Nov 26 20:36:22 2016 GMT >> * common name: FQDN_OF_My_gateway_HOST >> * issuer: >> CN=FQDN_OF_My_gateway_HOST,OU=Test,O=Hadoop,L=Test,ST=Test,C=US >> * Server auth using Basic with user 'knox' >> > GET /gateway/default/webhdfs/v1/?op=LISTSTATUS HTTP/1.1 >> > Authorization: Basic a25veDojMTIzUGFzc3dvcmQ= >> > User-Agent: curl/7.29.0 >> > Host: localhost:8443 >> > Accept: */* >> > >> < HTTP/1.1 401 Unauthorized >> HTTP/1.1 401 Unauthorized >> * Authentication problem. Ignoring this. >> < WWW-Authenticate: BASIC realm="application" >> WWW-Authenticate: BASIC realm="application" >> < Content-Length: 0 >> Content-Length: 0 >> < Server: Jetty(8.1.14.v20131031) >> Server: Jetty(8.1.14.v20131031) >> >> >> Please let me know if any additional information is required. >> >> Thanks, >> DP >> >> >
