Larry McCay created KNOX-762:
--------------------------------
Summary: Remove dependency on httpcomponents httpclient 4.5.2
Key: KNOX-762
URL: https://issues.apache.org/jira/browse/KNOX-762
Project: Apache Knox
Issue Type: Bug
Components: Server
Reporter: Larry McCay
Assignee: Larry McCay
Fix For: 0.10.0
Reported by Benjamin Ruland:
I am experiencing problems with Knox while using WebHDFS in a cluster with
Kerberos and SSL.
The KDC is an Microsoft AD 2012. Kerberos-Encryption is set to AES256. Knox is
connected to AD via LDAP sync (this is working fine for other Knox services).
I am running HDP 2.5 with Knox 0.9.0
In general, the cluster runs fine. WebHDFS using SPNEGO is working.
But when accessing WebHDFS over Knox, I get an 401 error and some strange logs.
I suspect that Knox is trying to get a ticket for a HTTPS/namenode@REALM
principal, which does not exist. Although running SSL, all principals for
SPNEGO are HTTP/...
I this a Knox Bug or is this a misconfiguration at some point?
It would be great, if someone has advice.
Best regards,
Benjamin
The used command is:
[root@utilitynode ~]# curl -ik -u validuser
"https://utilitynode:8443/gateway/default/webhdfs/v1/?OP=LISTSTATUS";
Enter host password for user 'validuser':
HTTP/1.1 401 Unauthorized
Date: Wed, 12 Oct 2016 07:47:41 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/default; Max-Age=0;
Expires=Tue,11-Oct-2016 07:47:41 GMT
WWW-Authenticate: BASIC realm="application"
Content-Length: 0
Server: Jetty(9.2.15.v20160210)
Debug Log in knox gateway.log
2016-10-12 09:51:49,735 DEBUG hadoop.gateway (GatewayFilter.java:doFilter(116))
- Received request: GET /webhdfs/v1/
2016-10-12 09:51:49,740 DEBUG hadoop.gateway
(KnoxLdapRealm.java:getUserDn(673)) - Searching from
OU=someOU,DC=somedomain,DC=de where
(&(objectclass=person)(sAMAccountName=validuser)) scope subtree
2016-10-12 09:51:49,745 INFO hadoop.gateway
(KnoxLdapRealm.java:getUserDn(679)) - Computed userDn:
CN=validuser,OU=Users,OU=someOU,DC=somedomain,DC=de using ldapSearch for
principal: validuser
2016-10-12 09:51:49,749 DEBUG hadoop.gateway
(UrlRewriteProcessor.java:rewrite(166)) - Rewrote URL:
https://utilitynode:8443/gateway/default/webhdfs/v1/?OP=LISTSTATUS, direction:
IN via explicit rule: WEBHDFS/webhdfs/inbound/namenode/root to URL:
https://utilitynode.somedomain.de:50470/webhdfs/v1/?OP=LISTSTATUS
2016-10-12 09:51:49,749 DEBUG hadoop.gateway
(DefaultDispatch.java:executeOutboundRequest(120)) - Dispatch request: GET
https://utilitynode.somedomain.de:50470/webhdfs/v1/?OP=LISTSTATUS&doAs=validuser
2016-10-12 09:51:49,781 WARN auth.HttpAuthenticator
(HttpAuthenticator.java:generateAuthResponse(207)) - NEGOTIATE authentication
error: No valid credentials provided (Mechanism level: No valid credentials
provided (Mechanism level: Server not found in Kerberos database (7)))
2016-10-12 09:51:49,782 DEBUG hadoop.gateway
(DefaultDispatch.java:executeOutboundRequest(133)) - Dispatch response status:
401
2016-10-12 09:51:49,783 DEBUG hadoop.gateway
(DefaultDispatch.java:getInboundResponseContentType(202)) - Using explicit
character set ISO-8859-1 for entity of type text/html
2016-10-12 09:51:49,783 DEBUG hadoop.gateway
(DefaultDispatch.java:getInboundResponseContentType(210)) - Inbound response
entity content type: text/html; charset=iso-8859-1
Log in knox gateway.out
Found ticket for knox/utilitynode.somedomain...@somedomain.de to go to
krbtgt/somedomain...@somedomain.de expiring on Wed Oct 12 19:53:51 CEST 2016
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
>>> Credentials acquireServiceCreds: same realm
default etypes for default_tgs_enctypes: 18.
>>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
getKDCFromDNS using UDP
>>> KrbKdcReq send: kdc=domaincontroller.somedomain.de. TCP:88, timeout=30000,
>>> number of retries =3, #bytes=1661
>>> KDCCommunication: kdc=domaincontroller.somedomain.de. TCP:88,
>>> timeout=30000,Attempt =1, #bytes=1661
>>>DEBUG: TCPClient reading 127 bytes
>>> KrbKdcReq send: #bytes read=127
>>> KdcAccessibility: remove domaincontroller.somedomain.de.:88
>>> KDCRep: init() encoding tag is 126 req type is 13
>>>KRBError:
sTime is Wed Oct 12 09:53:51 CEST 2016 1476258831000
suSec is 8354 suSec is 8354
error code is 7
error Message is Server not found in Kerberos database
sname is HTTPS/namenode.somedomain...@somedomain.de
msgType is 30
Extracts from topology config:
<topology>
<gateway>
<provider>
<role>authentication</role>
<name>ShiroProvider</name>
<enabled>true</enabled>
<!-- LDAP Sync properties sit here -->
<provider>
<role>identity-assertion</role>
<name>Default</name>
<enabled>true</enabled>
</provider>
<provider>
<role>authorization</role>
<name>XASecurePDPKnox</name>
<enabled>true</enabled>
</provider>
<provider>
<role>ha</role>
<name>HaProvider</name>
<enabled>true</enabled>
<param>
<name>WEBHDFS</name>
<value>maxFailoverAttempts=3;failoverSleep=1000;maxRetryAttempts=300;retrySleep=1000;enabled=true</value>
</param>
</provider>
</gateway>
<service>
<role>NAMENODE</role>
<url>hdfs://namenode.somedomain.de:8020</url>
<url>hdfs://namenode2.somedomain.de:8020</url>
</service>
<service>
<role>WEBHDFS</role>
<url>https://namenode.somedomain.de:50470/webhdfs</url>
<url>https://namenode2.somedomain.de:50470/webhdfs</url>
</service>
</topology>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
