[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387306#comment-14387306 ] Allen Wittenauer commented on HDFS-5796: Why are we rushing such an important issue? If 2.7 is late (whatever that means), so be it. That said, was this actually tested with an AltKerberos implementation that passes custom settings through the initializer? At least from what we saw prior to this flurry of patches getting pushed in, there was more to this than just the auth cookie in the complex cases. Or should we just live with the custom fix we put in place for our 2.4.1 deploy and make a new JIRA if/when this breaks when we move to 3.x? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387335#comment-14387335 ] Haohui Mai commented on HDFS-5796: -- bq. At least from what we saw prior to this flurry of patches getting pushed in, there was more to this than just the auth cookie in the complex cases. What are the complex cases? Can you contribute some unit tests? bq. Or should we just live with the custom fix we put in place for our 2.4.1 deploy and make a new JIRA if/when this breaks when we move to 3.x? I fail to see the merits of using AltKerberos for WebHDFS (yet). After HADOOP-11754 the expected behavior is that if the authentication of the UI works correctly, then browsing through WebHDFS will work. I'm yet to be convinced that such a workaround is required (and it does not seem to work in out-of-the-box configuration). The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387378#comment-14387378 ] Allen Wittenauer commented on HDFS-5796: bq. What are the complex cases? and bq. I fail to see the merits of using AltKerberos for WebHDFS (yet). ... are directly related. SPNEGO only works if a trust can be established between any/all relevant realms. What if that trust can't be used (e.g., copying data between two Hadoop systems owned by different companies)? What if Kerberos isn't being used at all for user-side authentication? See also HDFS-7983 and HDFS-7984. This is a very real problem. We hit it every day. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382381#comment-14382381 ] Allen Wittenauer commented on HDFS-5796: bq. Revert the changes in HDFS-5716 That's an incompatible change, I think, isn't it? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380429#comment-14380429 ] Ryan Sasson commented on HDFS-5796: --- This sounds reasonable, my only question is why not allow AuthFilter to use an alt-kerberos auth handler? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378550#comment-14378550
]
Haohui Mai commented on HDFS-5796:
--
I played around with it a little bit. It looks like as long as the webhdfs
filter recognizes the auth cookie (which the browser will get from the server
when accessing the UI), the request can go through. Therefore I propose the
following solution:
* Revert the changes in HDFS-5716.
* Share the same signer / secret across the filter over the filter on UI and
{{AuthFilter}}
* {{AuthFilter}} continues to support SPNEGO to maintain backward compatibility.
Does it sound reasonable?
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Ryan Sasson
Priority: Blocker
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch,
HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376324#comment-14376324 ] Haohui Mai commented on HDFS-5796: -- As HADOOP-10703 is committed, this jira should not be a blocker once we're able to unify the signer across the httpserver. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372969#comment-14372969 ] Vinod Kumar Vavilapalli commented on HDFS-5796: --- This ticket has stalled again, blocking the 2.7 release. [~wheat9], despite your summary, I don't follow the TODO items. Can you help summarize what needs to be done to unblock 2.7? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372977#comment-14372977 ] Allen Wittenauer commented on HDFS-5796: bq. This ticket has stalled again, blocking the 2.7 release. Sort of. Everyone here went to HADOOP-10703 with the hope that it would fix the problem. However, it did not. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369437#comment-14369437 ] Kihwal Lee commented on HDFS-5796: -- bq. This was just confusion in this JIRA as folks mixed two different issues. The whole Dr. Who thing is appears to be specific to Yahoo!'s security implementation of their hadoop-auth filter. Understanding what's going on makes much more sense if you push that as a completely separate issue. For the record, I have not seen this at Yahoo! with the filter being used for the past n years. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358907#comment-14358907 ] Allen Wittenauer commented on HDFS-5796: Hmm. I'm actually not certain we can use the SignerSecretProvider without breaking backward compatibility since it uses a different configuration property. :( This is just a mess. :( The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358930#comment-14358930
]
Arun Suresh commented on HDFS-5796:
---
[~aw], I just updated HADOOP-11702 to remove the file reading logic from
{{AuthenticationFilterInitializer}}. This is not required as the
{{AuthenticationFilter}} that it adds already instantiates the
{{StringSignerSecretProvider}} that will read the file.
bq. ..we can use the SignerSecretProvider without breaking backward
compatibility since it uses a different configuration property.
Think im missing something.. why do you say it uses a different property ?
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Ryan Sasson
Priority: Blocker
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch,
HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358946#comment-14358946 ] Allen Wittenauer commented on HDFS-5796: I thought SignerSecretProvider injected 'signer' into the config property names? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358978#comment-14358978
]
Arun Suresh commented on HDFS-5796:
---
Correct me if im wrong, but, I dont think the SignerSecretProvider injects
anything.. the filter (or rather what initializes the filter) adds signer
specific properties (after stripping away the app specific config prefix). If
there is a property like app.signer.secret.provider, the specific provider
(or signer implementation) will be initialized and used. All that code is here
: {{AuthenticationFilter#initializeSecretProvider()}}
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Ryan Sasson
Priority: Blocker
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch,
HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358876#comment-14358876 ] Allen Wittenauer commented on HDFS-5796: How about: a) commit HADOOP-11702 b) update [~rsasson]'s patch to use that code plus [~asuresh]'s other comments, and commit that to 2.7 c) post-pone the filter merger This would unblock 2.7 with a less risky and (at least for us!) working fix while the IMO riskier set of fixes is still on the table. We won't feel the pressure to get it in now because of the blocked release. Thoughts? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359286#comment-14359286 ] Haohui Mai commented on HDFS-5796: -- I'm going to spend some time looking into the last proposed solution and to see whether it works. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359277#comment-14359277
]
Haohui Mai commented on HDFS-5796:
--
I think there are multiple issues being discussed here and it makes it
difficult to follow. Let me try to recap and make sure everybody is on the same
page. Then we can discuss what needs to be done to unblock 2.7.
Problems:
* The UI and WebHDFS have different sets of authentication filters.
* The UI and WebHDFS uses different signers. Therefore UI auth filters does not
recognize the auth cookie generated WebHDFS auth filters, and vice versa.
* In secure set up, the old UI allows an anonymous user to be authenticated as
dr.who, while WebHDFS never allows authentications like this.
* The new UI accesses the HDFS directories using WebHDFS, which does not allow
anonymous users to be authenticated as dr.who. Thus anonymous users can no
longer browser HDFS.
Proposed solutions so far:
* Allow configurable WebHDFS authentication filter (in HDFS-5716). The users
can work around the problem using a customizable filter but it won't work out
of the box.
* Merging authentication filters -- proposed in HADOOP-10703. Users can
configure to use {{AltKerberosAuthenticationHandler}} for WebHDFS, so that the
anonymous users can be authenticated as dr. who. The issue is that the user
can no longer be authenticated as itself.
* Getting a delegation token in the UI before issuing WebHDFS requests --
proposed in this jira. It unifies the security model for both UI and WebHDFS,
but it requires the auth filter for WebHdfs to be able to authenticate users as
dr.who and it requires changes in the UI.
* Unify the signer for both the UI and the WebHDFS filter -- proposed in this
jira. The UI can authenticate the user as dr.who, the WebHDFS auth filter can
authenticate the auth cookie and get the corresponding UGI. It requires minimal
changes but it needs confirmation whether it actually works.
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Ryan Sasson
Priority: Blocker
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch,
HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359628#comment-14359628 ] Allen Wittenauer commented on HDFS-5796: bq. In secure set up, the old UI allows an anonymous user to be authenticated as dr.who, while WebHDFS never allows authentications like this. FWIW, I've never seen this in practice with the Kerberos authenticator. I have a suspicion this might be a Y! special with backyard. bq. Allow configurable WebHDFS authentication filter (in HDFS-5716). The users can work around the problem using a customizable filter but it won't work out of the box. Yes and no. Yes you can make it work if you decide to build an *entire alternative* to the normal hadoop-auth mechanisms that are in place or if your auth is VERY VERY VERY VERY simple. But in reality the answer is no because frankly that's just a stupid thing to do after we've beat into users heads for the past 3 years, across the entirety of the Hadoop ecosystem, and across multiple major versions of Hadoop that set of methods X is how we're handling auth. bq. Merging authentication filters – proposed in HADOOP-10703. Users can configure to use AltKerberosAuthenticationHandler for WebHDFS, so that the anonymous users can be authenticated as dr. who. The issue is that the user can no longer be authenticated as itself. Those are three independent threads. One can merge the filters, but unless it is done correctly, this is all still potentially broken. AltKerberos allows for SPNEGO + some other auth. If that other auth is implemented such that anon gets Dr. Who, so be it, but it is usually coded such that users do auth as themselves. bq. Getting a delegation token in the UI before issuing WebHDFS requests – proposed in this jira. It unifies the security model for both UI and WebHDFS, but it requires the auth filter for WebHdfs to be able to authenticate users as dr.who and it requires changes in the UI. This was just confusion in this JIRA as folks mixed two different issues. The whole Dr. Who thing is appears to be specific to Yahoo!'s security implementation of their hadoop-auth filter. Understanding what's going on makes much more sense if you push that as a completely separate issue. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357561#comment-14357561 ] Ryan Sasson commented on HDFS-5796: --- [~asuresh], your ticket brings up an important point about the last patch. The way hadoop authentication filters consume signature secrets is revamped in 2.6+ with support for reading secrets from zookeeper. Because of this the last patch would be not be fully compatible, as it does not consume signature secrets the same way that hadoop authentication filters do. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356374#comment-14356374
]
Arun Suresh commented on HDFS-5796:
---
Also, please consider using HADOOP-11702, which modifies the standard
{{StringSignerSecretProvider}} to read secret from file if present.
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Ryan Sasson
Priority: Blocker
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch,
HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356370#comment-14356370
]
Arun Suresh commented on HDFS-5796:
---
bq. I'm actually inclined to say no, since the other web elements are almost
all strictly interactive. In other words, if I'm using something like SAML for
my normal web auth and only have Kerberos deployed for internal hadoop stuff,
there's no need to put a Kerberos filter in front of those other UIs.
Currently, if you configure a different Auth filter via AuthFilterInitializer
and a different one (Kerb) for dfs.web.authentication, The user still has to go
thru the Kerberos authentication.. Basically, the user has to pass thru the
stricter scheme anyway.. So why not use a single AuthenticationFilter as
[~wheat9] suggested ?
Please Also note, as I mentioned in an earlier comment, the there is a THIRD
filter involved here which is initialized by {{HttpServer2#initSpnego()}}. This
ends up being the same filter as dfs.web.authentication, but a filter is still
initialized none the less .. I feel this should be removed.. either this JIRA
or another.
w.r.t to the patch
{noformat}
+Reader reader = new InputStreamReader(new FileInputStream(
+signatureSecretFile), Charsets.UTF_8);
+int c = reader.read();
+while (c -1) {
+ secret.append((char)c);
+ c = reader.read();
+}
+reader.close();
+p.setProperty(AuthenticationFilter.SIGNATURE_SECRET,
secret.toString());
{noformat}
could be better written as
{noformat}
secret = Files.readAllBytes(new File(secretFile).toPath())
{noformat}
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Ryan Sasson
Priority: Blocker
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch,
HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356048#comment-14356048 ] Allen Wittenauer commented on HDFS-5796: bq. I don't quite follow – I wonder, why setting the secret provider can solve this problem? It's more than that. The current code passes in zero properties into the configured handler, which means that if they need other configuration bits they can't get them. In our case, we are passing on other information that allows connection to our centralized web-based auth solution. The signature matters because the cookie in the Kerberos (alt or otherwise) filter matters: these are done on a per-host/domain+per auth filter basis, so if the same host+filter combo are in play, the secret file needs to be the same across all services that use that auth filter. It likely works in 2.6 with the generic Kerberos filter because it is continually re-auth'ing against Kerberos rather than relying upon the cookie. bq. I starts to wonder whether we should use the same signer for the authentication filters of both UI and WebHDFS in the NN (like the one proposed in HADOOP-10670) since both filters handle URLs coming from the same origin. This is an interesting question. I'm actually inclined to say no, since the other web elements are almost all strictly interactive. In other words, if I'm using something like SAML for my normal web auth and only have Kerberos deployed for internal hadoop stuff, there's no need to put a Kerberos filter in front of those other UIs. There's also the issue of what to do about the 2NN-NN connection if that is configured to use the same auth method The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356253#comment-14356253
]
Hadoop QA commented on HDFS-5796:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703777/HDFS-5796.4.patch
against trunk revision a5cf985.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9829//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9829//console
This message is automatically generated.
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Ryan Sasson
Priority: Blocker
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch,
HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14355950#comment-14355950 ] Allen Wittenauer commented on HDFS-5796: I'm (obviously) +1. This fixes the issues that we saw when using AltKerberos and suspect it will fix other people who are seeing failures when using either AltKerberos-based or an alternative handler. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14355992#comment-14355992 ] Haohui Mai commented on HDFS-5796: -- I don't quite follow -- I wonder, why setting the secret provider can solve this problem? I starts to wonder whether we should use the same signer for the authentication filters of both UI and WebHDFS in the NN (like the one proposed in HADOOP-10670) since both filters handle URLs coming from the same origin. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Ryan Sasson Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch, HDFS-5796.4.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353889#comment-14353889 ] Ryan Sasson commented on HDFS-5796: --- It appears that HADOOP-10709 will not fix this problem, more details in my comment on the ticket. I am currently working on a patch that will allow WebHDFS to initialize with a custom AltKerberosAuthenticationHandler subclass that can see custom filter parameters and read the cookie signature secret file. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14351307#comment-14351307 ] Allen Wittenauer commented on HDFS-5796: bq. when security is enabled, WebHDFS by default picks up SPNEGO + KerberosAuthFilter. So the UI works, but only when the browser is launched after a kinit. If I don't do a kinit, I cannot browse files through the UI - this is the loss of functionality that is being discussed here? No. The key point in that summary is by default. If you need something that isn't the default, the whole system falls apart. The fundamental problem is that if you use something like the AltKerberos filter, it flat out doesn't work. There two key problems we've noticed: a) filter parameters don't get passed down to either AltK's SPNEGO filter or a user's custom one b) after we did some custom hacking, we noticed that cookie secret handling is broken. Thus, using a browser to peruse HDFS is completely broken in 2.6 and up due to the removal of the old UI. bq. with HDFS-5716, you can turn the KerberosAuthFilter off and replace it with PseudoAuthFilter, but then the UI as well as applications always thinks you are dr.who. So, I guess this is not acceptable? No. HDFS-5716 just flat doesn't work in practice due to the above issues. It isn't reflective of real world usage at all. (.. and, believe me, we've tried to make it work without completely rewriting the built-in AltKerberos filter.) There's a very high chance that HADOOP-10709 might actually fix our issues, but the person who was testing for me today went home ill. :( So hopefully we'll try to verify on Monday. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14351156#comment-14351156 ] Vinod Kumar Vavilapalli commented on HDFS-5796: --- Hey everyone, I've been trying to understand the problem here, but it is a big wall of text. It'll be great if someone can help me. It seems like # when security is enabled, WebHDFS by default picks up SPNEGO + KerberosAuthFilter. So the UI works, but only when the browser is launched after a kinit. If I don't do a kinit, I cannot browse files through the UI - this is the loss of functionality that is being discussed here? # with HDFS-5716, you can turn the KerberosAuthFilter off and replace it with PseudoAuthFilter, but then the UI as well as applications always thinks you are dr.who. So, I guess this is not acceptable? # Is the patch trying to add (back) in a way to use KerberosAuthFilter for regular applications but use Dr.Who for browsers? And that is a security concern, so we don't want to put it back? Going back to the title, The file system browser in the namenode UI requires SPNEGO.. Seems like with HDFS-5716, you can set your own filter and so the discussion is really about the defaults? Trying to gauge its priority for 2.7. Thanks. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14351319#comment-14351319 ] Allen Wittenauer commented on HDFS-5796: (And, just to make clear the impact, this issue prevents us from upgrading Hadoop.) The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345589#comment-14345589 ] Allen Wittenauer commented on HDFS-5796: bq. issue 1 If this fixes the fact that we can't pass configuration parameters to filters, then go for it. We've got a patch we're playing with as well, but no unit tests written for it. bq. What do we do about Client browsers that cannot handle SPNEGO (or if the users browser is outside the security infrastructure of the Cluster) ? This is exactly the purpose of the AltKerberos filter and the one we're using. It flips between SPNENGO and non-SPNEGO auth based upon the browser string. bq. I still feel that (if configured), requests from browsers should be handled differently (via the use of the AltKerberosAuthFilter), possibly by allowing those requests to be authenticated as a special, configured proxy user. That's basically the same thing as Sure, I live in a glass house, but I have security and privacy because there is a lock on the door. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345808#comment-14345808
]
Arun Suresh commented on HDFS-5796:
---
[~aw],
bq. .. One configures it in core-site.xml to enable it.
Aah.. apologize for the misunderstanding.. Am aware of us being able to
configure it via *hadoop.http.authentication.type* parameter in core-site.xml.
But like you mentioned, the user has to provide a jar containing the subclass
of {{AltKerberosAuthenticationHandler}}. I was just attempting to provider a
default implementation.
If you feel we should allow the user to configure a subclass.. then yes...
maybe we should close this JIRA and fix the actual problem, which I believe is
the multitude of filters
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Arun Suresh
Priority: Blocker
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch,
HDFS-5796.3.patch, HDFS-5796.3.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345818#comment-14345818 ] Allen Wittenauer commented on HDFS-5796: There's still a problem though. dfs.web.authentication doesn't pass any configuration information into the filter for it to use, unlike the other http-auth filters. So if you need to give other information, it gets nuked and the filter fails. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345703#comment-14345703 ] Allen Wittenauer commented on HDFS-5796: bq. please correct me if Im wrong, but looking a the code in hadoop trunk, I don't think AltKerberos is currently used. It's existed for a very long time. We're using it in 2.4.1 on our secure clusters now. One configures it in core-site.xml to enable it. bq. So that we are on the same page, we agree using AltKerberos Handler is the right approach.. but I think we should agree on what exactly should be the alternate mechanism... This stuff is (as typical) poorly documented, but that's the point of AltKerberos. Users can build their own filter mechanism to work alongside the SPNEGO one. So if someone wants to use (for example) OAuth, they just need to push that Implementation into their own jar and configure it in core-site.xml. So if you wanted to, you could do the necessary Implementation of the AltKerberos methods that said we auth via SAML and anyone that fails gets Dr. Who. This we we don't have to dictate anything. It probably would be useful, however, to have a working AltKerberos example that does something real... but that's a different issue. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345746#comment-14345746 ] Haohui Mai commented on HDFS-5796: -- bq. Sure, I live in a glass house, but I have security and privacy because there is a lock on the door. Can't agree more. I have a strong preferences on not going back to the dr.who approach from a security point of view. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345676#comment-14345676 ] Arun Suresh commented on HDFS-5796: --- [~aw], thanks for the comments, I think we are talking about similar things, but : bq. This is exactly the purpose of the AltKerberos filter and the one we're using... please correct me if Im wrong, but looking a the code in hadoop trunk, I don't think AltKerberos is currently used. The patches I have posted are an attempt at introducing the AltKerberos auth handler for Browser access. So that we are on the same page, we agree using AltKerberos Handler is the right approach.. but I think we should agree on what exactly should be the alternate mechanism... I vote, the default case should be to bring back the old dr.who user (unfortunately, I feel this is closer to the glass house situation you mentioned), or a scheme like what I proposed in my patches, where the proxy user has to have a proper kerberos principal and keytab. And allow the specific alternate mechanism to be configured. Thoughts ? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344890#comment-14344890
]
Arun Suresh commented on HDFS-5796:
---
[~aw], [~wheat9],
I do agree that we need a rethink of the auth-filter.
But there are actually 2 issues here :
*Issue 1* :
From my analysis of the code, there are actually 3 filters that come into play
for the Namenode UI currently
# If the user follows this link for
[web-console|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/HttpAuthentication.html],
the {{AuthenticationFilterInitializer}} class will initialize a third
{{AuthenticationFilter}} for root (*/*) resource.
# If security is enabled, {{HttpServer2#initSpnego()}} will, in addition to the
above... initilize an {{AuthenticationFilter}} for all urls
# The {{NamenodeHttpServer}} which actually uses the {{HttpServer2}}, will add
an {{AuthFilter}} (which incidentally is a subclass of the above
{{AutthenticationFilter}}) for all */webhdfs/v1* urls
I propose that the http-auth be initialized and configured only by the
{{AuthenticationFilterInitializer}}.. and sub component may add other filters
but should not be Authentication related.
If you guys are ok with the above, I am happy to put together a patch for this.
*Issue 2* :
What do we do about Client browsers that cannot handle SPNEGO (or if the users
browser is outside the security infrastructure of the Cluster) ?
I still feel that (if configured), requests from browsers should be handled
differently (via the use of the {{AltKerberosAuthFilter}}), possibly by
allowing those requests to be authenticated as a special, configured proxy
user.
Again am happy to work on this, if you guys are are ok with this approach.
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Arun Suresh
Priority: Blocker
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch,
HDFS-5796.3.patch, HDFS-5796.3.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342536#comment-14342536 ] Allen Wittenauer commented on HDFS-5796: All: I have set this as a blocker for 2.7.0 on the basis that http-auth-based plug-ins (especially those that require additional configuration) don't work. This is a pretty big regression from previous versions of Hadoop, made fatal now that the old HDFS browse code has been removed. If we feel that this particular JIRA has forked from that issue, then let's create a new one. Thanks. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Priority: Blocker Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326287#comment-14326287 ] Haohui Mai commented on HDFS-5796: -- bq. Forgive me if I am wrong, but I see that the the request actually goes thru multiple Filters before it gets to WebHDFS It can be done through specifying custom filters, just like what needs to be done to enable SPNEGO for the web UI today. It requires some configuration but it can be done. The mechanisms of filters are overdue for some redesigns. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14321291#comment-14321291 ] Arun Suresh commented on HDFS-5796: --- [~wheat9], bq. .. Does it make sense to just modify the UI to issue a `GET_DELEGATION_TOKEN` call to get a token before browsing the filesystem ? .. Do you mean to call the 'getDelegationToken()' on the WebHdfsFilesystem ? Forgive me if I am wrong, but I see that the the request actually goes thru multiple Filters before it gets to WebHDFS, and in a secure cluster, these are all SPNEGO filters... and will not solve this particular issue. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318856#comment-14318856 ] Allen Wittenauer commented on HDFS-5796: The irony is that our handler is using the AltKerberos code (which means it does support SPNEGO). However WebHDFS still requires us mucking about with dfs.web.authentication.filter rather than inherit from the global setting. HDFS-5716 seems like a bug rather than a feature at this point. :( (I also found HDFS-7033 while researching why our auth bits aren't working. So now I feel stupid as well as perplexed trying to get this feature to work.) The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318847#comment-14318847
]
Arun Suresh commented on HDFS-5796:
---
[~aw], Yup, you are correct..
HDFS-5716 does work as expected but this issue is a bit different. (Do correct
me if i am wrong) The current Web UI delegates to WebHDFS. If we use the
{{dfs.web.authentication.filter}} exposed by HDFS-5716, and set a filter that
does not SPNEGO authenticate, then ALL access to WebHDFS will be
un-authenticated. This is probably un-desirable.
What the current patch does is let WebHDFS use the default filter but the
AuthHandler detects Browser access via user-agent and forwards as a different
user. I guess the debate is whether to use a static user like the old
{{dr.who}} or maybe another user.
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Arun Suresh
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch,
HDFS-5796.3.patch, HDFS-5796.3.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318866#comment-14318866 ] Arun Suresh commented on HDFS-5796: --- bq. The irony is that our handler is using the AltKerberos code (which means it does support SPNEGO) Hmm.. technically, it does.. you can just configure it to bypass SPNEGO for browser based access... The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318883#comment-14318883 ] Arun Suresh commented on HDFS-5796: --- apologize for the spam.. thought you meant doesn't The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319269#comment-14319269 ] Haohui Mai commented on HDFS-5796: -- bq. That is incorrect. Permissions are checked as dr.who username. What version are you observing this in specifically? That has never been the behaviour in any Apache release I've known since 0.20.2. Thanks for the clarification. I just checked the code and you're right. Now I understand what you're coming from. However, I think it is a bad idea to add it into the filter. Does it make sense to just modify the UI to issue a `GET_DELEGATION_TOKEN` call to get a token before browsing the filesystem? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318561#comment-14318561 ] Allen Wittenauer commented on HDFS-5796: So, a point of clarification: * We have a custom auth plugin * We are trying to use the new WebHDFS-based UI * We get an authorization failure * Switch to old DFS RPC-based UI * It works Is this patch/JIRA supposed to fix/be this issue and the discussion has just veered off or are we looking at HDFS-5716 just not actually working? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14315076#comment-14315076 ] Allen Wittenauer commented on HDFS-5796: Can we split out some of the debug and logging changes into a separate patch so we can get those committed while y'all are still discussing the merits of the approach? Those are useful no matter what kind of auth you are doing... :D The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14315183#comment-14315183 ] Harsh J commented on HDFS-5796: --- [~wheat9] bq. I don't quite follow. Does the user need to able to read all files in the HDFS cluster in order for the UI to work? What kinds of access controls do you plan to apply on the particular user? The goal, again, is to have what we had before. The user dr.who functioned like the others user w.r.t. permissions, and did not have access to all files. Likewise is expected with the approach here. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14315208#comment-14315208 ] Haohui Mai commented on HDFS-5796: -- bq. The goal, again, is to have what we had before. The user dr.who functioned like the others user w.r.t. permissions, and did not have access to all files. Likewise is expected with the approach here. To clarify, in the old UI, dr.who does have read access of all files. bq. The configuration is explicit. Can you clarify on what vulnerability adding a static, non-existent user to the viewer brings? Is the following true -- an attacker who does not have any read access of the cluster is able to read some files through the UI, but not through the HDFS RPC. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14315187#comment-14315187 ] Harsh J commented on HDFS-5796: --- bq. From a security prospective, I think that it is a no-go if users that are using the browser and users that are using standard RPC interfaces are treated differently – it can easily lead to misconfiguration and security vulnerabilities. The configuration is explicit. Can you clarify on what vulnerability adding a static, non-existent user to the viewer brings? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14315333#comment-14315333
]
Harsh J commented on HDFS-5796:
---
bq. To clarify, in the old UI, dr.who does have read access of all files.
That is incorrect. Permissions are checked as dr.who username. What version are
you observing this in specifically? That has never been the behaviour in any
Apache release I've known since 0.20.2.
I clearly do get this error displayed on the old UI, with or without strong
authentication:
bq. Permission denied: user=dr.who, access=READ_EXECUTE,
inode=/tmp/foo:root:supergroup:drwx--
Steps were:
* {{hadoop fs -mkdir /tmp/foo}}
* {{hadoop fs -chmod 700 /tmp/foo}}
* {{hadoop fs -put pig_1423045014803.log /tmp/foo/}}
Then visit /tmp/foo on NN UI. This above test was done on 2.0.5 for example.
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Arun Suresh
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch,
HDFS-5796.3.patch, HDFS-5796.3.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311816#comment-14311816
]
Arun Suresh commented on HDFS-5796:
---
[~wheat9],
bq. ... Does the user need to able to read all files in the HDFS cluster in
order for the UI to work? What kinds of access controls do you plan to apply on
the particular user?
So what I meant was, unlike before where *dr.who*, who not only is an
un-authenticated user, but is also not a real HDFS recognized user (dr.who is
not associated to any groups and thus cannot be ACL restricted / permission
restricted on any folder) is able to access any file in HDFS... what I propose
is a scheme where browser access is auto-authenticated (when turned-on
explicitly) as an (explicitly configured) HDFS user associated with a group and
thus can be ACL / permission restricted from viewing certain files / folders by
the cluster admin.
bq. From a security prospective, I think that it is a no-go if users that are
using the browser and users that are using standard RPC interfaces are treated
differently – it can easily lead to misconfiguration and security
vulnerabilities.
Wrt. Misconfiguration, I agree that it would be a security issue.. but I am
infact reusing the existing {{AltKerberosAuthenticationHandler}} which does
browser check based on user agent.. I would be happy to take a shot at fixing
that up if you find any vulnerabilities in it.
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Arun Suresh
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch,
HDFS-5796.3.patch, HDFS-5796.3.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14302047#comment-14302047 ] Arun Suresh commented on HDFS-5796: --- ping [~wheat9] .. does the above suggestion work ? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14302490#comment-14302490 ] Haohui Mai commented on HDFS-5796: -- bq. Since this would be a real valid user, hdfs admin can apply normal access grants / restrictions on this user.. I don't quite follow. Does the user need to able to read all files in the HDFS cluster in order for the UI to work? What kinds of access controls do you plan to apply on the particular user? From a security prospective, I think that it is a no-go if users that are using the browser and users that are using standard RPC interfaces are treated differently -- it can easily lead to misconfiguration and security vulnerabilities. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14293957#comment-14293957 ] Haohui Mai commented on HDFS-5796: -- bq. Would that be OK to place back as a feature (turned off by default if needed), as the new file browser has regressed? Is giving every files the world-readable permission a possible workaround? It looks like the workaround is fully equivalent to using dr. who here? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14294023#comment-14294023 ] Arun Suresh commented on HDFS-5796: --- [~wheat9], [~qwertymaniac], Thank you both for chiming in.. bq. Is giving every files the world-readable permission a possible workaround? It looks like the workaround is fully equivalent to using dr. who here? The latest patch allows the administrator to configure a real user as the browser proxy (not static dr.who). Since this would be a real valid user, hdfs admin can apply normal access grants / restrictions on this user... and thus wont be world readable like dr.who The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14293261#comment-14293261
]
Hadoop QA commented on HDFS-5796:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12694722/HDFS-5796.3.patch
against trunk revision 6f9fe76.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common
hadoop-hdfs-project/hadoop-hdfs.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9339//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9339//console
This message is automatically generated.
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Arun Suresh
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch,
HDFS-5796.3.patch, HDFS-5796.3.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14293396#comment-14293396 ] Harsh J commented on HDFS-5796: --- [~wheat9], bq. This has been called out a security vulnerability. The user has to authenticate himself / herself before accessing any data in the cluster. The goal of this JIRA is to allow flexibility like it existed in pre-bootstrap UI, where not having web console authentication turned on also applied to the provided file browser. With that in mind, I don't see how the static user concept proves itself as a vulnerability, cause the user is already aware their web console is not authenticating anyone for anything, including the web browser. We have customers who need generic user (dr.who, etc. - this is configurable) file browsing on the NN UI without authentication just as it had existed prior to the WebHDFS file browser introduction, even though their kerberos authentication is turned on in the cluster. Would that be OK to place back as a feature (turned off by default if needed), as the new file browser has regressed? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch, HDFS-5796.3.patch, HDFS-5796.3.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292904#comment-14292904
]
Hadoop QA commented on HDFS-5796:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12694648/HDFS-5796.3.patch
against trunk revision 1f2b695.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.ipc.TestRPCCallBenchmark
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9333//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9333//console
This message is automatically generated.
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Arun Suresh
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch,
HDFS-5796.3.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291045#comment-14291045
]
Hadoop QA commented on HDFS-5796:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12694417/HDFS-5796.2.patch
against trunk revision 3703965.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-auth hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.fs.TestHdfsNativeCodeLoader
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9325//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9325//console
This message is automatically generated.
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Arun Suresh
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch, HDFS-5796.2.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14220063#comment-14220063 ] Arun Suresh commented on HDFS-5796: --- [~benoyantony], Thanks for the pointer to HADOOP-10709 From what I understand after going thru the patch, it looks like you have a _TokenAuthFIlter_ that SKIPs authentication if it finds a DelegationToken in the request attribute. But for the delegation token to be there in the first place, it would require the user to be get authenticated atleast once (the first time).. which is the problem I was trying to solve... Basically, I was trying to bring the current NN Web UI to user experience parity prior to HDFS-5382. Correct me if I am wrong, but prior to that, even on a secure cluster, Web UI access was basically un-authenticated (as _dr.who_ always).. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14220080#comment-14220080 ] Haohui Mai commented on HDFS-5796: -- bq. Correct me if I am wrong, but prior to that, even on a secure cluster, Web UI access was basically un-authenticated (as dr.who always).. This has been called out a security vulnerability. The user has to authenticate himself / herself before accessing any data in the cluster. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14218745#comment-14218745 ] Benoy Antony commented on HDFS-5796: Got it, Thanks [~asuresh]. The proposed solution has the following drawback: Users from browsers cannot access the cluster as themselves. One approach would be to modify AuthFilter so that one can configure an _AuthenticationHandler_ of choice. In our clusters, I have taken a slightly different approach which is outlined in HADOOP-10709. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14217381#comment-14217381
]
Hadoop QA commented on HDFS-5796:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12682075/HDFS-5796.1.patch
against trunk revision 79301e8.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/8777//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8777//console
This message is automatically generated.
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Arun Suresh
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14217431#comment-14217431
]
Benoy Antony commented on HDFS-5796:
{quote}
So now, when the handler detects requests are coming from a browser (based on
user agent), it will not initiate SPNEGO handshake and will return an
AuthenticationToken for the configured HTTP kerberos principal
{quote}
Does that mean for any browser request to webhdfs, the authenticated user will
be configured HTTP kerberos principal ? That doesn't sound right since file
system has to be accessed as the user itself and not as the configured HTTP
kerberos principal. And the real user has to be authenticated properly.
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Arun Suresh
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14217472#comment-14217472 ] Arun Suresh commented on HDFS-5796: --- [~benoyantony], The old Web UI used to allow browser based access as a _dr.who_ user that could see/read world readable files irrespective of whether security was turned on or off. After HDFS-5382, this was not possible, since the the browser request was routed thru WebHDFS.. and on a secure cluster, WebHDFS required the client to be SPNEGO authenticated. This cannot be expected of a user's browser that is outside the cluster's security infrastructure and has no access to a KDC. Now HDFS-5716 allows one to configure a user specified filter for WebHDFS that can side step SPNEGO, but unfortunately this mean all requests from inside the secure cluster will also forgo SPNEGO authentication. My patch was for a (IMO) middle ground where experience of users switching from the old Web UI is not severely degraded.. by allowing unauthenticated browser based access as the configured _HTTP_ kerberos principal. In any case, many browsers do not even support SPNEGO authentications, so authentication might not even be possible.. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14215502#comment-14215502 ] Arun Suresh commented on HDFS-5796: --- Hey [~wheat9], I've been working on a patch for this. Do you mind if I assign this issue to myself, since it looks like you haven't worked on it in a while? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.3.0 Reporter: Kihwal Lee Assignee: Haohui Mai After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14215505#comment-14215505 ] Haohui Mai commented on HDFS-5796: -- No problem. Please feel free to go ahead assign it to yourself. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.3.0 Reporter: Kihwal Lee Assignee: Haohui Mai After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14215524#comment-14215524 ] Haohui Mai commented on HDFS-5796: -- bq. It basically uses AltKerberosAuthenticationHandler to no do SPNEGO for browser based requests.. For the basic secure cluster set up. The user opens the browser, and the browsers authenticate the user to the NN through SPNEGO. How does this patch change this use case? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.3.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14215533#comment-14215533
]
Arun Suresh commented on HDFS-5796:
---
So now, when the handler detects requests are coming from a browser (based on
user agent), it will not initiate SPNEGO handshake and will return an
{{AuthenticationToken}} for the configured HTTP kerberos principal
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.3.0
Reporter: Kihwal Lee
Assignee: Arun Suresh
Attachments: HDFS-5796.1.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14215550#comment-14215550
]
Hadoop QA commented on HDFS-5796:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12682039/HDFS-5796.1.patch
against trunk revision 2fce6d6.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8769//console
This message is automatically generated.
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.3.0
Reporter: Kihwal Lee
Assignee: Arun Suresh
Attachments: HDFS-5796.1.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[
https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14215770#comment-14215770
]
Hadoop QA commented on HDFS-5796:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12682075/HDFS-5796.1.patch
against trunk revision 9dd5d67.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.fs.loadGenerator.TestLoadGenerator
org.apache.hadoop.hdfs.TestReservedRawPaths
org.apache.hadoop.hdfs.TestFileCreationDelete
org.apache.hadoop.hdfs.TestEncryptionZones
org.apache.hadoop.hdfs.TestEncryptionZonesWithHA
org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS
The following test timeouts occurred in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestDatanodeBlockScanner
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/8771//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8771//console
This message is automatically generated.
The file system browser in the namenode UI requires SPNEGO.
---
Key: HDFS-5796
URL: https://issues.apache.org/jira/browse/HDFS-5796
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee
Assignee: Arun Suresh
Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch
After HDFS-5382, the browser makes webhdfs REST calls directly, requiring
SPNEGO to work between user's browser and namenode. This won't work if the
cluster's security infrastructure is isolated from the regular network.
Moreover, SPNEGO is not supposed to be required for user-facing web pages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14215779#comment-14215779 ] Arun Suresh commented on HDFS-5796: --- The testcase failures are un-related to this patch... The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Kihwal Lee Assignee: Arun Suresh Attachments: HDFS-5796.1.patch, HDFS-5796.1.patch After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13922624#comment-13922624 ] Kihwal Lee commented on HDFS-5796: -- The existing use case was to use a custom auth filter for web ui accesses and SPNEGO for webhdfs. HDFS-5716 cannot be used to support this use case. In any case, I no longer think this is a blocker. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.3.0 Reporter: Kihwal Lee Assignee: Haohui Mai Priority: Blocker After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13921237#comment-13921237 ] Suresh Srinivas commented on HDFS-5796: --- [~kihwal], this is marked as a blocker. Can you please respond to [~wheat9]'s comment above and confirm the pluggable authentication mechanism addresses the need? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.3.0 Reporter: Kihwal Lee Assignee: Haohui Mai Priority: Blocker After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904784#comment-13904784 ] Haohui Mai commented on HDFS-5796: -- HDFS-5716 allows pluggable authentication mechanism in WebHDFS which provides a solution to this problem. Is it okay to mark this bug as a duplicate of HDFS-5716? The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.3.0 Reporter: Kihwal Lee Assignee: Haohui Mai Priority: Blocker After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HDFS-5796) The file system browser in the namenode UI requires SPNEGO.
[ https://issues.apache.org/jira/browse/HDFS-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13875192#comment-13875192 ] Haohui Mai commented on HDFS-5796: -- WebHDFS is designed to be a gateway for all third-party services where SPNEGO might not be available. In my opinion WebHDFS should support the same authentication mechanisms as the one the web UI supports. The file system browser in the namenode UI requires SPNEGO. --- Key: HDFS-5796 URL: https://issues.apache.org/jira/browse/HDFS-5796 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.4.0 Reporter: Kihwal Lee Priority: Critical After HDFS-5382, the browser makes webhdfs REST calls directly, requiring SPNEGO to work between user's browser and namenode. This won't work if the cluster's security infrastructure is isolated from the regular network. Moreover, SPNEGO is not supposed to be required for user-facing web pages. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
