[jira] [Commented] (HDFS-5661) Browsing FileSystem via web ui, should use datanode's hostname instead of ip address
[ https://issues.apache.org/jira/browse/HDFS-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13850720#comment-13850720 ] Benoy Antony commented on HDFS-5661: The attached patch is tested on a 6 node cluster with SSL and security enabled. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address Key: HDFS-5661 URL: https://issues.apache.org/jira/browse/HDFS-5661 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.2.0 Reporter: Benoy Antony Assignee: Benoy Antony Attachments: HDFS-5661.patch, HDFS-5661.patch If authentication is enabled on the web ui, then a cookie is used to keep track of the authentication information. There is normally a domain associated with the cookie. Since ip address doesn't have any domain , the cookie will not be sent by the browser while making http calls with ip address as the destination server. This will break browsing files system via web ui , if authentication is enabled. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HDFS-5661) Browsing FileSystem via web ui, should use datanode's hostname instead of ip address
[ https://issues.apache.org/jira/browse/HDFS-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13850718#comment-13850718 ] Benoy Antony commented on HDFS-5661: New patch is attached. The patch uses canonicalized fqdn instead of ip address or hostname passed in by DN. The canonicalize method is moved from DatanodeJspHelper.java to JspHelper.java Browsing FileSystem via web ui, should use datanode's hostname instead of ip address Key: HDFS-5661 URL: https://issues.apache.org/jira/browse/HDFS-5661 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.2.0 Reporter: Benoy Antony Assignee: Benoy Antony Attachments: HDFS-5661.patch, HDFS-5661.patch If authentication is enabled on the web ui, then a cookie is used to keep track of the authentication information. There is normally a domain associated with the cookie. Since ip address doesn't have any domain , the cookie will not be sent by the browser while making http calls with ip address as the destination server. This will break browsing files system via web ui , if authentication is enabled. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HDFS-5661) Browsing FileSystem via web ui, should use datanode's hostname instead of ip address
[ https://issues.apache.org/jira/browse/HDFS-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849381#comment-13849381 ] Benoy Antony commented on HDFS-5661: DelegationToken is used to access Namenode. Here is the sequence: # NN generates DT and put the DT into the redirect URL # DN receives the redirect request. The AuthenticationFIlter authenticates using hadoop.auth cookie if available. # JSPs on the datadode server (i.e., tail / browseBlock / browseDirectory) access the NN using DelegationToken obtained as a URL parameter. For step 2 to work, the uri should have FQDN and the FQDN should be suffixed with hadoop.http.authentication.cookie.domain . One can verify the above by reviewing the code and testing it. Usage of IP address (introduced in HDFS-5307) broke the file browsing when security is turned on. What's the argument against using FQDN instead of ip address ? A hostname is always ensured during DN registration and the attached patch uses it. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address Key: HDFS-5661 URL: https://issues.apache.org/jira/browse/HDFS-5661 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.2.0 Reporter: Benoy Antony Assignee: Benoy Antony Attachments: HDFS-5661.patch If authentication is enabled on the web ui, then a cookie is used to keep track of the authentication information. There is normally a domain associated with the cookie. Since ip address doesn't have any domain , the cookie will not be sent by the browser while making http calls with ip address as the destination server. This will break browsing files system via web ui , if authentication is enabled. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HDFS-5661) Browsing FileSystem via web ui, should use datanode's hostname instead of ip address
[ https://issues.apache.org/jira/browse/HDFS-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848410#comment-13848410 ] Benoy Antony commented on HDFS-5661: AuthFilter.java is used only for webhdfs. While accessing JSP files, AuthenticationFilter is used and AuthenticationFilter does not use delegationToken. Note that the use of IP address while generating redirectURL was introduced with HDFS-5307. It used to be fqdn before. From the HDFS-5307 patch , -String fqdn = canonicalize(chosenNode.getIpAddr()); -String tailUrl = /// + fqdn + : + chosenNode.getInfoPort() + +String tailUrl = /// + JspHelper.Url.authority(req.getScheme(), chosenNode) Browsing FileSystem via web ui, should use datanode's hostname instead of ip address Key: HDFS-5661 URL: https://issues.apache.org/jira/browse/HDFS-5661 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.2.0 Reporter: Benoy Antony Assignee: Benoy Antony Attachments: HDFS-5661.patch If authentication is enabled on the web ui, then a cookie is used to keep track of the authentication information. There is normally a domain associated with the cookie. Since ip address doesn't have any domain , the cookie will not be sent by the browser while making http calls with ip address as the destination server. This will break browsing files system via web ui , if authentication is enabled. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HDFS-5661) Browsing FileSystem via web ui, should use datanode's hostname instead of ip address
[
https://issues.apache.org/jira/browse/HDFS-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848440#comment-13848440
]
Haohui Mai commented on HDFS-5661:
--
bq. AuthFilter.java is used only for webhdfs. While accessing JSP files,
AuthenticationFilter is used and AuthenticationFilter does not use
delegationToken.
All meaningful JSP on the datadode server (i.e., tail / browseBlock /
browseDirectory) accesses the HDFS. You cannot proceed without a delegation
token.
If you are able to access it without a DT, this is a security vulnerability and
please file a jira to report it.
bq. Note that the use of IP address while generating redirectURL was introduced
with HDFS-5307. It used to be fqdn before.
It calls {{InetSocketAddress#getCanonicalHostName()}} internally. It is broken
when the machine have multiple DNS names.
Popping up one level, can you please restate what you are trying to achieve?
The old UI is no longer under active development, it may be deprecated or
removed at some point. It may be worthwhile to spend the time of migrating to
the new UI.
Browsing FileSystem via web ui, should use datanode's hostname instead of ip
address
Key: HDFS-5661
URL: https://issues.apache.org/jira/browse/HDFS-5661
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.2.0
Reporter: Benoy Antony
Assignee: Benoy Antony
Attachments: HDFS-5661.patch
If authentication is enabled on the web ui, then a cookie is used to keep
track of the authentication information. There is normally a domain
associated with the cookie. Since ip address doesn't have any domain , the
cookie will not be sent by the browser while making http calls with ip
address as the destination server.
This will break browsing files system via web ui , if authentication is
enabled.
Browsing FileSystem via web ui, should use datanode's hostname instead of ip
address.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-5661) Browsing FileSystem via web ui, should use datanode's hostname instead of ip address
[ https://issues.apache.org/jira/browse/HDFS-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1384#comment-1384 ] Haohui Mai commented on HDFS-5661: -- The namenode and the datanode have different origins therefore the browser will not attach the cookies when making a request to the datanode. Redirecting using the domain name will not help. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address Key: HDFS-5661 URL: https://issues.apache.org/jira/browse/HDFS-5661 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.2.0 Reporter: Benoy Antony Assignee: Benoy Antony Attachments: HDFS-5661.patch If authentication is enabled on the web ui, then a cookie is used to keep track of the authentication information. There is normally a domain associated with the cookie. Since ip address doesn't have any domain , the cookie will not be sent by the browser while making http calls with ip address as the destination server. This will break browsing files system via web ui , if authentication is enabled. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HDFS-5661) Browsing FileSystem via web ui, should use datanode's hostname instead of ip address
[ https://issues.apache.org/jira/browse/HDFS-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13847789#comment-13847789 ] Benoy Antony commented on HDFS-5661: Browser visits namenode.domainname.com:50070 . hadoopauth cookie is dropped with domain as domainname.com. User clicks browse File system on the webui and browser gets redirected to ADataNode.domainname.com:1006 . Then the browser will send cookies to ADataNode.domainname.com . Without this patch, when user clicks browse File system on the webui and gets redirected to datanodeipaddress:1006 and hence no cookies will be sent. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address Key: HDFS-5661 URL: https://issues.apache.org/jira/browse/HDFS-5661 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.2.0 Reporter: Benoy Antony Assignee: Benoy Antony Attachments: HDFS-5661.patch If authentication is enabled on the web ui, then a cookie is used to keep track of the authentication information. There is normally a domain associated with the cookie. Since ip address doesn't have any domain , the cookie will not be sent by the browser while making http calls with ip address as the destination server. This will break browsing files system via web ui , if authentication is enabled. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HDFS-5661) Browsing FileSystem via web ui, should use datanode's hostname instead of ip address
[ https://issues.apache.org/jira/browse/HDFS-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13847826#comment-13847826 ] Haohui Mai commented on HDFS-5661: -- I'm curious how you manage to pass the cookie to the datanode. Even with your patch, the cookies should not be passed from the namenode and the datanode. The browser is not supposed to shared the cookies. An origin is defined by the scheme, host, and port of a URL[1], where two hosts with the same hostname but different ports are considered different origins. The browsers implement the same-origin policy, where the cookies are isolated in different origins [2]. [1] http://www.w3.org/Security/wiki/Same_Origin_Policy [2] https://code.google.com/p/browsersec/wiki/Part2#Same-origin_policy Browsing FileSystem via web ui, should use datanode's hostname instead of ip address Key: HDFS-5661 URL: https://issues.apache.org/jira/browse/HDFS-5661 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.2.0 Reporter: Benoy Antony Assignee: Benoy Antony Attachments: HDFS-5661.patch If authentication is enabled on the web ui, then a cookie is used to keep track of the authentication information. There is normally a domain associated with the cookie. Since ip address doesn't have any domain , the cookie will not be sent by the browser while making http calls with ip address as the destination server. This will break browsing files system via web ui , if authentication is enabled. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HDFS-5661) Browsing FileSystem via web ui, should use datanode's hostname instead of ip address
[ https://issues.apache.org/jira/browse/HDFS-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13847852#comment-13847852 ] Jing Zhao commented on HDFS-5661: - For the redirect, I think we are using DelegationToken which is included in the redirect URL? Thus we do not need to worry about hostname/ip address here I guess. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address Key: HDFS-5661 URL: https://issues.apache.org/jira/browse/HDFS-5661 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.2.0 Reporter: Benoy Antony Assignee: Benoy Antony Attachments: HDFS-5661.patch If authentication is enabled on the web ui, then a cookie is used to keep track of the authentication information. There is normally a domain associated with the cookie. Since ip address doesn't have any domain , the cookie will not be sent by the browser while making http calls with ip address as the destination server. This will break browsing files system via web ui , if authentication is enabled. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HDFS-5661) Browsing FileSystem via web ui, should use datanode's hostname instead of ip address
[ https://issues.apache.org/jira/browse/HDFS-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848011#comment-13848011 ] Benoy Antony commented on HDFS-5661: Only Namenode will authenticate a client using HDFS Delegation Token ( which is issued by the Namenode.) The client still needs to authenticate to the datanode and cannot use the Delegation Token for this purpose. The configuration related to the cookie domain is hadoop.http.authentication.cookie.domain and more details on the authentication of http consoles is here : https://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-common/HttpAuthentication.html When browsing the filesystem via original webui, the DelegationToken is used by the DataNode to contact the Namenode . Browsing FileSystem via web ui, should use datanode's hostname instead of ip address Key: HDFS-5661 URL: https://issues.apache.org/jira/browse/HDFS-5661 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.2.0 Reporter: Benoy Antony Assignee: Benoy Antony Attachments: HDFS-5661.patch If authentication is enabled on the web ui, then a cookie is used to keep track of the authentication information. There is normally a domain associated with the cookie. Since ip address doesn't have any domain , the cookie will not be sent by the browser while making http calls with ip address as the destination server. This will break browsing files system via web ui , if authentication is enabled. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HDFS-5661) Browsing FileSystem via web ui, should use datanode's hostname instead of ip address
[ https://issues.apache.org/jira/browse/HDFS-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848048#comment-13848048 ] Haohui Mai commented on HDFS-5661: -- The only way to access the data on a secure DN is to present a valid delegation token. The HTTP auth tokens do not contain the DT, presenting the HTTP auth tokens to the DN does not grant you the access, thus it makes no sense to pass them around. Regardless of what UI you're using, the NN fetches the DT on the behalf of the client, and the client presents this DT to authenticate with DN. This should the only way you can access the data. If you happen to get the data in your approach, this is a security hole and please file a jira to track it. Again, I'll encourage you to check out the new web UI. It accesses the data through WebHDFS which is much more robust. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address Key: HDFS-5661 URL: https://issues.apache.org/jira/browse/HDFS-5661 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.2.0 Reporter: Benoy Antony Assignee: Benoy Antony Attachments: HDFS-5661.patch If authentication is enabled on the web ui, then a cookie is used to keep track of the authentication information. There is normally a domain associated with the cookie. Since ip address doesn't have any domain , the cookie will not be sent by the browser while making http calls with ip address as the destination server. This will break browsing files system via web ui , if authentication is enabled. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HDFS-5661) Browsing FileSystem via web ui, should use datanode's hostname instead of ip address
[ https://issues.apache.org/jira/browse/HDFS-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848076#comment-13848076 ] Benoy Antony commented on HDFS-5661: --the client presents this DT to authenticate with DN. This s not correct. How can DN validate and read DT ? The DT is issued by NN. client/agent can authenticate using DT only with NN. Both NN/DN use https://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-common/HttpAuthentication.html to authenticate a user's access to their http interfaces. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address Key: HDFS-5661 URL: https://issues.apache.org/jira/browse/HDFS-5661 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.2.0 Reporter: Benoy Antony Assignee: Benoy Antony Attachments: HDFS-5661.patch If authentication is enabled on the web ui, then a cookie is used to keep track of the authentication information. There is normally a domain associated with the cookie. Since ip address doesn't have any domain , the cookie will not be sent by the browser while making http calls with ip address as the destination server. This will break browsing files system via web ui , if authentication is enabled. Browsing FileSystem via web ui, should use datanode's hostname instead of ip address. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HDFS-5661) Browsing FileSystem via web ui, should use datanode's hostname instead of ip address
[
https://issues.apache.org/jira/browse/HDFS-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848085#comment-13848085
]
Jing Zhao commented on HDFS-5661:
-
In AuthFilter#doFilter, we have the following code:
{code}
public void doFilter(ServletRequest request, ServletResponse response,
FilterChain filterChain) throws IOException, ServletException {
final HttpServletRequest httpRequest =
toLowerCase((HttpServletRequest)request);
final String tokenString = httpRequest.getParameter(DelegationParam.NAME);
if (tokenString != null) {
//Token is present in the url, therefore token will be used for
//authentication, bypass kerberos authentication.
filterChain.doFilter(httpRequest, response);
return;
}
super.doFilter(httpRequest, response, filterChain);
}
{code}
In DatanodeJspHelper#generateDirectoryStructure, we have
{code}
String tokenString = req.getParameter(JspHelper.DELEGATION_PARAMETER_NAME);
UserGroupInformation ugi = JspHelper.getUGI(req, conf);
.
DFSClient dfs = getDFSClient(ugi, nnAddr, conf);
{code}
So I think here the whole process is:
1. NN generates DT and put the DT into the redirect URL
2. DN receives the redirect request, finds that there is DT in the request,
thus the corresponding SPNEGO filter will bypass the auth check
3. DN uses the DT and files a getFileInfo RPC call to NN
4. DN shows the result to web ui
Browsing FileSystem via web ui, should use datanode's hostname instead of ip
address
Key: HDFS-5661
URL: https://issues.apache.org/jira/browse/HDFS-5661
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.2.0
Reporter: Benoy Antony
Assignee: Benoy Antony
Attachments: HDFS-5661.patch
If authentication is enabled on the web ui, then a cookie is used to keep
track of the authentication information. There is normally a domain
associated with the cookie. Since ip address doesn't have any domain , the
cookie will not be sent by the browser while making http calls with ip
address as the destination server.
This will break browsing files system via web ui , if authentication is
enabled.
Browsing FileSystem via web ui, should use datanode's hostname instead of ip
address.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
