[jira] [Updated] (HDFS-4548) Webhdfs doesn't renegotiate SPNEGO token
[ https://issues.apache.org/jira/browse/HDFS-4548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-4548: - Resolution: Fixed Fix Version/s: 2.0.4-alpha 0.23.7 3.0.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I've committed to trunk, branch-2 and branch-0.23. Thanks all for the work! > Webhdfs doesn't renegotiate SPNEGO token > > > Key: HDFS-4548 > URL: https://issues.apache.org/jira/browse/HDFS-4548 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha, 3.0.0, 0.23.7 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Blocker > Fix For: 3.0.0, 0.23.7, 2.0.4-alpha > > Attachments: HDFS-4548.branch-23.patch, HDFS-4548.branch-23.patch, > HDFS-4548.branch-23.patch, HDFS-4548.branch-23.patch, > HDFS-4548.branch-23.patch, HDFS-4548.patch, HDFS-4548.patch, HDFS-4548.patch, > HDFS-4548.patch, HDFS-4548.patch > > > When the webhdfs SPNEGO token expires, the fs doesn't attempt to renegotiate > a new SPNEGO token. This renders webhdfs unusable for daemons that are > logged in via a keytab which would allow a new SPNEGO token to be generated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4548) Webhdfs doesn't renegotiate SPNEGO token
[ https://issues.apache.org/jira/browse/HDFS-4548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-4548: -- Target Version/s: 3.0.0, 0.23.7, 2.0.5-beta (was: 3.0.0, 2.0.5-beta, 0.23.8) > Webhdfs doesn't renegotiate SPNEGO token > > > Key: HDFS-4548 > URL: https://issues.apache.org/jira/browse/HDFS-4548 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha, 3.0.0, 0.23.7 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HDFS-4548.branch-23.patch, HDFS-4548.branch-23.patch, > HDFS-4548.branch-23.patch, HDFS-4548.branch-23.patch, > HDFS-4548.branch-23.patch, HDFS-4548.patch, HDFS-4548.patch, HDFS-4548.patch, > HDFS-4548.patch, HDFS-4548.patch > > > When the webhdfs SPNEGO token expires, the fs doesn't attempt to renegotiate > a new SPNEGO token. This renders webhdfs unusable for daemons that are > logged in via a keytab which would allow a new SPNEGO token to be generated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4548) Webhdfs doesn't renegotiate SPNEGO token
[ https://issues.apache.org/jira/browse/HDFS-4548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-4548: -- Priority: Blocker (was: Critical) This really is a blocker because webhdfs becomes unusable by daemons after 10h. > Webhdfs doesn't renegotiate SPNEGO token > > > Key: HDFS-4548 > URL: https://issues.apache.org/jira/browse/HDFS-4548 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha, 3.0.0, 0.23.7 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HDFS-4548.branch-23.patch, HDFS-4548.branch-23.patch, > HDFS-4548.branch-23.patch, HDFS-4548.branch-23.patch, > HDFS-4548.branch-23.patch, HDFS-4548.patch, HDFS-4548.patch, HDFS-4548.patch, > HDFS-4548.patch, HDFS-4548.patch > > > When the webhdfs SPNEGO token expires, the fs doesn't attempt to renegotiate > a new SPNEGO token. This renders webhdfs unusable for daemons that are > logged in via a keytab which would allow a new SPNEGO token to be generated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4548) Webhdfs doesn't renegotiate SPNEGO token
[ https://issues.apache.org/jira/browse/HDFS-4548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-4548: -- Attachment: HDFS-4548.patch HDFS-4548.branch-23.patch Same as previous patches, just made from the top level instead of 1 dir deep. Tests aren't feasible due to kerberos being required to activate code paths, but fixes have been verified on internal clusters blocked by this issue. > Webhdfs doesn't renegotiate SPNEGO token > > > Key: HDFS-4548 > URL: https://issues.apache.org/jira/browse/HDFS-4548 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha, 3.0.0, 0.23.7 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HDFS-4548.branch-23.patch, HDFS-4548.branch-23.patch, > HDFS-4548.branch-23.patch, HDFS-4548.branch-23.patch, > HDFS-4548.branch-23.patch, HDFS-4548.patch, HDFS-4548.patch, HDFS-4548.patch, > HDFS-4548.patch, HDFS-4548.patch > > > When the webhdfs SPNEGO token expires, the fs doesn't attempt to renegotiate > a new SPNEGO token. This renders webhdfs unusable for daemons that are > logged in via a keytab which would allow a new SPNEGO token to be generated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4548) Webhdfs doesn't renegotiate SPNEGO token
[ https://issues.apache.org/jira/browse/HDFS-4548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-4548: -- Attachment: HDFS-4548.patch HDFS-4548.branch-23.patch Updated patches that will negotiate a SPNEGO token as needed for secure auth connections (token operations). > Webhdfs doesn't renegotiate SPNEGO token > > > Key: HDFS-4548 > URL: https://issues.apache.org/jira/browse/HDFS-4548 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha, 3.0.0, 0.23.7 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HDFS-4548.branch-23.patch, HDFS-4548.branch-23.patch, > HDFS-4548.branch-23.patch, HDFS-4548.branch-23.patch, HDFS-4548.patch, > HDFS-4548.patch, HDFS-4548.patch, HDFS-4548.patch > > > When the webhdfs SPNEGO token expires, the fs doesn't attempt to renegotiate > a new SPNEGO token. This renders webhdfs unusable for daemons that are > logged in via a keytab which would allow a new SPNEGO token to be generated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4548) Webhdfs doesn't renegotiate SPNEGO token
[ https://issues.apache.org/jira/browse/HDFS-4548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-4548: -- Status: Patch Available (was: Open) > Webhdfs doesn't renegotiate SPNEGO token > > > Key: HDFS-4548 > URL: https://issues.apache.org/jira/browse/HDFS-4548 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha, 3.0.0, 0.23.7 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HDFS-4548.branch-23.patch, HDFS-4548.branch-23.patch, > HDFS-4548.branch-23.patch, HDFS-4548.branch-23.patch, HDFS-4548.patch, > HDFS-4548.patch, HDFS-4548.patch, HDFS-4548.patch > > > When the webhdfs SPNEGO token expires, the fs doesn't attempt to renegotiate > a new SPNEGO token. This renders webhdfs unusable for daemons that are > logged in via a keytab which would allow a new SPNEGO token to be generated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4548) Webhdfs doesn't renegotiate SPNEGO token
[ https://issues.apache.org/jira/browse/HDFS-4548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-4548: -- Attachment: HDFS-4548.patch HDFS-4548.branch-23.patch Webhdfs is only using an authenticated url connection for token operations which relatively speaking don't occur very often esp. from a client - once for "hadoop fs" and once for job submission. The simplest solution is to SPNEGO negotiate the token connections. This patch will intersect with HDFS-3367 so I am not submitting since I will need to amend it. > Webhdfs doesn't renegotiate SPNEGO token > > > Key: HDFS-4548 > URL: https://issues.apache.org/jira/browse/HDFS-4548 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha, 3.0.0, 0.23.7 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HDFS-4548.branch-23.patch, HDFS-4548.branch-23.patch, > HDFS-4548.branch-23.patch, HDFS-4548.patch, HDFS-4548.patch, HDFS-4548.patch > > > When the webhdfs SPNEGO token expires, the fs doesn't attempt to renegotiate > a new SPNEGO token. This renders webhdfs unusable for daemons that are > logged in via a keytab which would allow a new SPNEGO token to be generated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4548) Webhdfs doesn't renegotiate SPNEGO token
[ https://issues.apache.org/jira/browse/HDFS-4548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-4548: -- Issue Type: Sub-task (was: Bug) Parent: HDFS-4576 > Webhdfs doesn't renegotiate SPNEGO token > > > Key: HDFS-4548 > URL: https://issues.apache.org/jira/browse/HDFS-4548 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha, 3.0.0, 0.23.7 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HDFS-4548.branch-23.patch, HDFS-4548.branch-23.patch, > HDFS-4548.patch, HDFS-4548.patch > > > When the webhdfs SPNEGO token expires, the fs doesn't attempt to renegotiate > a new SPNEGO token. This renders webhdfs unusable for daemons that are > logged in via a keytab which would allow a new SPNEGO token to be generated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4548) Webhdfs doesn't renegotiate SPNEGO token
[
https://issues.apache.org/jira/browse/HDFS-4548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daryn Sharp updated HDFS-4548:
--
Attachment: HDFS-4548.branch-23.patch
Patch for branch 23 to retry at a higher level when an
{{AuthenticationException}} occurs. This will handle authentication exceptions
either during connect or in response to the operations.
It's not very pretty because {{AuthenticationException}} can originate from
multiple levels sometimes wrapped in an {{IOException}}. I'd rather not change
method signatures for 23, but it should be cleaner in trunk since
{{AuthenticationException}} appears to bubble up higher.
Since the code is directly tied to kerberos, it's not possible to write tests.
I have verified existing webhdfs tests pass, that webhdfs still works on a
secure cluster, and that the re-attempt works on a secure cluster by
instrumenting the code to throw an exception on the first connect or first
validation of a response.
Trunk patch forthcoming.
> Webhdfs doesn't renegotiate SPNEGO token
>
>
> Key: HDFS-4548
> URL: https://issues.apache.org/jira/browse/HDFS-4548
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha, 3.0.0, 0.23.7
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HDFS-4548.branch-23.patch, HDFS-4548.branch-23.patch,
> HDFS-4548.patch, HDFS-4548.patch
>
>
> When the webhdfs SPNEGO token expires, the fs doesn't attempt to renegotiate
> a new SPNEGO token. This renders webhdfs unusable for daemons that are
> logged in via a keytab which would allow a new SPNEGO token to be generated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4548) Webhdfs doesn't renegotiate SPNEGO token
[ https://issues.apache.org/jira/browse/HDFS-4548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-4548: -- Attachment: HDFS-4548.patch HDFS-4548.branch-23.patch Patch is a bit smaller, and 23 is a bit different due to lack of formal retry policies. Trying to write tests. > Webhdfs doesn't renegotiate SPNEGO token > > > Key: HDFS-4548 > URL: https://issues.apache.org/jira/browse/HDFS-4548 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.0.0-alpha, 3.0.0, 0.23.7 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HDFS-4548.branch-23.patch, HDFS-4548.patch, > HDFS-4548.patch > > > When the webhdfs SPNEGO token expires, the fs doesn't attempt to renegotiate > a new SPNEGO token. This renders webhdfs unusable for daemons that are > logged in via a keytab which would allow a new SPNEGO token to be generated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4548) Webhdfs doesn't renegotiate SPNEGO token
[
https://issues.apache.org/jira/browse/HDFS-4548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daryn Sharp updated HDFS-4548:
--
Attachment: HDFS-4548.patch
Basic approach is if an {{AuthenticationException}} is encountered while
holding a SPNEGO token, blank it out and try again. If the second attempt
fails, error out.
I need to figure out how to write tests for this change, but please provide
feedback on whether this is a viable approach.
> Webhdfs doesn't renegotiate SPNEGO token
>
>
> Key: HDFS-4548
> URL: https://issues.apache.org/jira/browse/HDFS-4548
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha, 3.0.0, 0.23.7
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HDFS-4548.patch
>
>
> When the webhdfs SPNEGO token expires, the fs doesn't attempt to renegotiate
> a new SPNEGO token. This renders webhdfs unusable for daemons that are
> logged in via a keytab which would allow a new SPNEGO token to be generated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
