[jira] [Commented] (HADOOP-9363) AuthenticatedURL will NPE if server closes connection
[
https://issues.apache.org/jira/browse/HADOOP-9363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17802964#comment-17802964
]
Shilun Fan commented on HADOOP-9363:
Bulk update: moved all 3.4.0 non-blocker issues, please move back if it is a
blocker. Retarget 3.5.0.
> AuthenticatedURL will NPE if server closes connection
> -
>
> Key: HADOOP-9363
> URL: https://issues.apache.org/jira/browse/HADOOP-9363
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0-alpha1
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Major
>
> A NPE occurs if the server unexpectedly closes the connection for an
> {{AuthenticatedURL}} w/o sending a response.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-9363) AuthenticatedURL will NPE if server closes connection
[
https://issues.apache.org/jira/browse/HADOOP-9363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15210530#comment-15210530
]
Steve Loughran commented on HADOOP-9363:
Looking at this. So the root cause is the back end isn't sending back a valid
response; the client is NPEing.
I'm looking at the code in the JDK to see how the NPE could be triggered, but
it's not immediately obvious. I don't see any diffs between Java 7u45 and Java
8, so have to assume that if there is a problem, it's still there.
What about catching any RuntimeException raised as this point, rethrow it as an
IOE, and including the URL at fault in the message?
> AuthenticatedURL will NPE if server closes connection
> -
>
> Key: HADOOP-9363
> URL: https://issues.apache.org/jira/browse/HADOOP-9363
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
>
> A NPE occurs if the server unexpectedly closes the connection for an
> {{AuthenticatedURL}} w/o sending a response.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9363) AuthenticatedURL will NPE if server closes connection
[
https://issues.apache.org/jira/browse/HADOOP-9363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14255007#comment-14255007
]
Carl Steinbach commented on HADOOP-9363:
Hi [~daryn],
bq. the 401 sans WWW-Authenticate header triggers a NPE in the client due to a
JDK bug that assumes a RFC-compliant response
Can you post a link to this JDK bug? I remember finding it the last time I
looked at this ticket, but right my now Google powers are failing me. Thanks!
AuthenticatedURL will NPE if server closes connection
-
Key: HADOOP-9363
URL: https://issues.apache.org/jira/browse/HADOOP-9363
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical
A NPE occurs if the server unexpectedly closes the connection for an
{{AuthenticatedURL}} w/o sending a response.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9363) AuthenticatedURL will NPE if server closes connection
[
https://issues.apache.org/jira/browse/HADOOP-9363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14247078#comment-14247078
]
Anthony Hsu commented on HADOOP-9363:
-
Hi [~daryn],
Any updates on this ticket and/or a patch? We have seen this issue
intermittently at LinkedIn, too.
Best,
Anthony
AuthenticatedURL will NPE if server closes connection
-
Key: HADOOP-9363
URL: https://issues.apache.org/jira/browse/HADOOP-9363
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical
A NPE occurs if the server unexpectedly closes the connection for an
{{AuthenticatedURL}} w/o sending a response.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9363) AuthenticatedURL will NPE if server closes connection
[
https://issues.apache.org/jira/browse/HADOOP-9363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13873798#comment-13873798
]
Daryn Sharp commented on HADOOP-9363:
-
I've determined the problem for this old issue and am working on a patch. If
the the spnego auth fails, it returns a 401 with no WWW-Authenticate header.
This violates the RFC which mandates a 401 always contains a WWW-Authenticate
header.
If any kerberos error occurs, the 401 sans WWW-Authenticate header triggers a
NPE in the client due to a JDK bug that assumes a RFC-compliant response.
Examples of errors include but are not limited to:
# server's kerberos principal/keytab and the http server started anyway
# server's TGT expires
# server principal's kvno is stale
# client has no TGT
# client sends an invalid service ticket - expired, wrong enc_type, wrong kvno,
etc)
# kdc is not available - down or transient network failure
# clock skew causes a kerberos failure
# replay attack is triggered (another JDK bug triggered by AuthenticatedURL)
The NPE is particularly bad for daemon services. For example, the RM will
crash if an NPE occurs while renewing/canceling a token.
AuthenticatedURL will NPE if server closes connection
-
Key: HADOOP-9363
URL: https://issues.apache.org/jira/browse/HADOOP-9363
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
A NPE occurs if the server unexpectedly closes the connection for an
{{AuthenticatedURL}} w/o sending a response.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HADOOP-9363) AuthenticatedURL will NPE if server closes connection
[
https://issues.apache.org/jira/browse/HADOOP-9363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13615388#comment-13615388
]
Daryn Sharp commented on HADOOP-9363:
-
This also occurs for unexpected kerberos errors such as a kvno version mismatch
between the client's service ticket and the server's HTTP principal in its
keytab.
{noformat}
Caused by: GSSException: Failure unspecified at GSS-API level (Mechanism level:
Specified version of key is not available (44))
at
sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:788)
at
sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:342)
at
sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:285)
at
sun.security.jgss.spnego.SpNegoContext.GSS_acceptSecContext(SpNegoContext.java:871)
at
sun.security.jgss.spnego.SpNegoContext.acceptSecContext(SpNegoContext.java:544)
at
sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:342)
at
sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:285)
at
org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler$2.run(KerberosAuthenticationHandler.java:278)
at
org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler$2.run(KerberosAuthenticationHandler.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.authenticate(KerberosAuthenticationHandler.java:270)
... 23 more
Caused by: KrbException: Specified version of key is not available (44)
at sun.security.krb5.EncryptionKey.findKey(EncryptionKey.java:588)
at sun.security.krb5.KrbApReq.authenticate(KrbApReq.java:270)
at sun.security.krb5.KrbApReq.init(KrbApReq.java:144)
at
sun.security.jgss.krb5.InitSecContextToken.init(InitSecContextToken.java:108)
at
sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:771)
{noformat}
I sniffed the packets and the SPNEGO exchange proceeds as expected: server
sends 401 with WWW-Authenticate header, client responds with Authorization
header, server responds with 401 with status message set to the kerberos
exception - client then NPEs on that response. It's unclear (I haven't
investigated) if it's a JDK bug, or if AuthenticatedURL's twiddling of the
URLConnection is causing the issue.
AuthenticatedURL will NPE if server closes connection
-
Key: HADOOP-9363
URL: https://issues.apache.org/jira/browse/HADOOP-9363
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
A NPE occurs if the server unexpectedly closes the connection for an
{{AuthenticatedURL}} w/o sending a response.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9363) AuthenticatedURL will NPE if server closes connection
[
https://issues.apache.org/jira/browse/HADOOP-9363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13593781#comment-13593781
]
Daryn Sharp commented on HADOOP-9363:
-
Example stack trace from 23, although the line numbers should be similar for
trunk. Problem was found while attempting to inject faults to force
acquisition of a new SPNEGO token.
In this particular case, a kerberos replay attack exception caused the server
to abruptly close the connection. The issue could of course happen for other
reasons.
{noformat}
Exception in thread main java.lang.RuntimeException:
java.lang.NullPointerException
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1014)
at
sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:2211)
at
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:382)
at
org.apache.hadoop.security.authentication.client.AuthenticatedURL.extractToken(AuthenticatedURL.java:251)
at
org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:61)
at
org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:143)
at
org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:217)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getHttpUrlConnection(WebHdfsFileSystem.java:360)
[]
Caused by: java.lang.NullPointerException
at
sun.net.www.protocol.http.NegotiateAuthentication.setHeaders(NegotiateAuthentication.java:161)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1171)
at
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:373)
{noformat}
AuthenticatedURL will NPE if server closes connection
-
Key: HADOOP-9363
URL: https://issues.apache.org/jira/browse/HADOOP-9363
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
A NPE occurs if the server unexpectedly closes the connection for an
{{AuthenticatedURL}} w/o sending a response.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
