[jira] [Commented] (HDFS-4783) TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684207#comment-13684207 ] Hudson commented on HDFS-4783: -- Integrated in Hadoop-Mapreduce-trunk #1458 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1458/]) HDFS-4783. TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows. Contributed by Chris Nauroth. (Revision 1493149) Result = FAILURE cnauroth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493149 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDelegationTokensWithHA.java > TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows > > > Key: HDFS-4783 > URL: https://issues.apache.org/jira/browse/HDFS-4783 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.1.0-beta >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Fix For: 3.0.0, 2.1.0-beta > > Attachments: HDFS-4783.1.patch > > > This test asserts that delegation tokens previously associated to a host with > a resolved IP address no longer match for selection when > hadoop.security.token.service.use_ip is set false. The test assumes that > 127.0.0.1 resolves to host name "localhost". On Windows, this is not the > case, and instead it resolves to "127.0.0.1". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4783) TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684182#comment-13684182 ] Hudson commented on HDFS-4783: -- Integrated in Hadoop-Hdfs-trunk #1431 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1431/]) HDFS-4783. TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows. Contributed by Chris Nauroth. (Revision 1493149) Result = FAILURE cnauroth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493149 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDelegationTokensWithHA.java > TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows > > > Key: HDFS-4783 > URL: https://issues.apache.org/jira/browse/HDFS-4783 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.1.0-beta >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Fix For: 3.0.0, 2.1.0-beta > > Attachments: HDFS-4783.1.patch > > > This test asserts that delegation tokens previously associated to a host with > a resolved IP address no longer match for selection when > hadoop.security.token.service.use_ip is set false. The test assumes that > 127.0.0.1 resolves to host name "localhost". On Windows, this is not the > case, and instead it resolves to "127.0.0.1". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4783) TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684144#comment-13684144 ] Hudson commented on HDFS-4783: -- Integrated in Hadoop-Yarn-trunk #241 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/241/]) HDFS-4783. TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows. Contributed by Chris Nauroth. (Revision 1493149) Result = SUCCESS cnauroth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493149 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDelegationTokensWithHA.java > TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows > > > Key: HDFS-4783 > URL: https://issues.apache.org/jira/browse/HDFS-4783 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.1.0-beta >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Fix For: 3.0.0, 2.1.0-beta > > Attachments: HDFS-4783.1.patch > > > This test asserts that delegation tokens previously associated to a host with > a resolved IP address no longer match for selection when > hadoop.security.token.service.use_ip is set false. The test assumes that > 127.0.0.1 resolves to host name "localhost". On Windows, this is not the > case, and instead it resolves to "127.0.0.1". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4783) TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13683520#comment-13683520 ] Hudson commented on HDFS-4783: -- Integrated in Hadoop-trunk-Commit #3922 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3922/]) HDFS-4783. TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows. Contributed by Chris Nauroth. (Revision 1493149) Result = SUCCESS cnauroth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493149 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDelegationTokensWithHA.java > TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows > > > Key: HDFS-4783 > URL: https://issues.apache.org/jira/browse/HDFS-4783 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.1.0-beta >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Attachments: HDFS-4783.1.patch > > > This test asserts that delegation tokens previously associated to a host with > a resolved IP address no longer match for selection when > hadoop.security.token.service.use_ip is set false. The test assumes that > 127.0.0.1 resolves to host name "localhost". On Windows, this is not the > case, and instead it resolves to "127.0.0.1". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4783) TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13682431#comment-13682431 ] Chris Nauroth commented on HDFS-4783: - Daryn, is there an issue for that underlying bug, or should we file a new one? I'm planning on committing this, but I'll wait until end of day in case there are any other objections. > TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows > > > Key: HDFS-4783 > URL: https://issues.apache.org/jira/browse/HDFS-4783 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.1.0-beta >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Attachments: HDFS-4783.1.patch > > > This test asserts that delegation tokens previously associated to a host with > a resolved IP address no longer match for selection when > hadoop.security.token.service.use_ip is set false. The test assumes that > 127.0.0.1 resolves to host name "localhost". On Windows, this is not the > case, and instead it resolves to "127.0.0.1". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4783) TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13682288#comment-13682288 ] Daryn Sharp commented on HDFS-4783: --- Sigh, this is technically masking an underlying bug of a double fwd/rev resolve which can return a different hostname and foul the use of CNAMES. Fixing it is probably more complex than it seems, so +0.5. > TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows > > > Key: HDFS-4783 > URL: https://issues.apache.org/jira/browse/HDFS-4783 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.1.0-beta >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Attachments: HDFS-4783.1.patch > > > This test asserts that delegation tokens previously associated to a host with > a resolved IP address no longer match for selection when > hadoop.security.token.service.use_ip is set false. The test assumes that > 127.0.0.1 resolves to host name "localhost". On Windows, this is not the > case, and instead it resolves to "127.0.0.1". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4783) TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
[
https://issues.apache.org/jira/browse/HDFS-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681695#comment-13681695
]
Chuan Liu commented on HDFS-4783:
-
+1
I think changing the test case to use "localhost" is acceptable here.
I checked {{HAUtil.cloneDelegationTokenForLogicalUri()}} usage in
ConfiguredFailoverProxyProvider.
The hostname passed in was from a configuration, i.e. dfs.namenode.rpc-address;
so we don't have the issue that the host was get from {{getNameNodeAddress()}}.
If there are similar problems elsewhere, they should be separate problems, and
we don't need to address the host name difference here in this JIRA.
The above analysis is really useful and should help us understanding future
problems if there is any.
> TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
>
>
> Key: HDFS-4783
> URL: https://issues.apache.org/jira/browse/HDFS-4783
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.1.0-beta
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HDFS-4783.1.patch
>
>
> This test asserts that delegation tokens previously associated to a host with
> a resolved IP address no longer match for selection when
> hadoop.security.token.service.use_ip is set false. The test assumes that
> 127.0.0.1 resolves to host name "localhost". On Windows, this is not the
> case, and instead it resolves to "127.0.0.1".
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4783) TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
[
https://issues.apache.org/jira/browse/HDFS-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647034#comment-13647034
]
Chris Nauroth commented on HDFS-4783:
-
I tried static resolution, but unfortunately, it didn't work. Eventually, we
get to a point inside the RPC {{Server#Listener}} constructor where we pass
through a {{ServerSocketChannel}}. This seems to always drop the host name
from the address and re-resolve, so on Windows, this drops "localhost" in favor
of "127.0.0.1".
{code}
public Listener() throws IOException {
address = new InetSocketAddress(bindAddress, port);
// Create a new server socket and set to non blocking mode
acceptChannel = ServerSocketChannel.open();
acceptChannel.configureBlocking(false);
// Bind the server socket to the local host and port
bind(acceptChannel.socket(), address, backlogLength, conf,
portRangeConfig);
{code}
In the above code, even if {{bindAddress}} is "localhost" (thanks to static
resolution), the end result is still an address with host name "127.0.0.1".
Here is a very simplified version of what happens inside the bind method:
{code}
InetSocketAddress localhost = new InetSocketAddress("localhost", 12345);
System.out.println("localhost = " + localhost + ", w/hostname = " +
localhost.getHostName());
ServerSocketChannel ssc = ServerSocketChannel.open();
ssc.socket().bind(localhost);
InetSocketAddress addr =
(InetSocketAddress)ssc.socket().getLocalSocketAddress();
System.out.println("addr = " + addr + ", w/hostname = " +
addr.getHostName());
{code}
Running this code on non-Windows prints:
{code}
localhost = localhost/127.0.0.1:12345, w/hostname = localhost
addr = /127.0.0.1:12345, w/hostname = localhost
{code}
And on Windows:
{code}
localhost = localhost/127.0.0.1:12345, w/hostname = localhost
addr = /127.0.0.1:12345, w/hostname = 127.0.0.1
{code}
> TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
>
>
> Key: HDFS-4783
> URL: https://issues.apache.org/jira/browse/HDFS-4783
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.0.5-beta
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HDFS-4783.1.patch
>
>
> This test asserts that delegation tokens previously associated to a host with
> a resolved IP address no longer match for selection when
> hadoop.security.token.service.use_ip is set false. The test assumes that
> 127.0.0.1 resolves to host name "localhost". On Windows, this is not the
> case, and instead it resolves to "127.0.0.1".
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4783) TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
[
https://issues.apache.org/jira/browse/HDFS-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13646621#comment-13646621
]
Daryn Sharp commented on HDFS-4783:
---
Might want to test if {{NetUtils.addStaticResolution("127.0.0.1",
"localhost")}} will resolve this recurring issue.
> TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
>
>
> Key: HDFS-4783
> URL: https://issues.apache.org/jira/browse/HDFS-4783
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.0.5-beta
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HDFS-4783.1.patch
>
>
> This test asserts that delegation tokens previously associated to a host with
> a resolved IP address no longer match for selection when
> hadoop.security.token.service.use_ip is set false. The test assumes that
> 127.0.0.1 resolves to host name "localhost". On Windows, this is not the
> case, and instead it resolves to "127.0.0.1".
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4783) TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
[
https://issues.apache.org/jira/browse/HDFS-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13646277#comment-13646277
]
Hadoop QA commented on HDFS-4783:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12581275/HDFS-4783.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/4348//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4348//console
This message is automatically generated.
> TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
>
>
> Key: HDFS-4783
> URL: https://issues.apache.org/jira/browse/HDFS-4783
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.0.5-beta
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HDFS-4783.1.patch
>
>
> This test asserts that delegation tokens previously associated to a host with
> a resolved IP address no longer match for selection when
> hadoop.security.token.service.use_ip is set false. The test assumes that
> 127.0.0.1 resolves to host name "localhost". On Windows, this is not the
> case, and instead it resolves to "127.0.0.1".
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4783) TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13646170#comment-13646170 ] Chris Nauroth commented on HDFS-4783: - This patch can go to both trunk and branch-2. There are no dependencies on other Windows-specific code that only resides in trunk. > TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows > > > Key: HDFS-4783 > URL: https://issues.apache.org/jira/browse/HDFS-4783 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.0.5-beta >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Attachments: HDFS-4783.1.patch > > > This test asserts that delegation tokens previously associated to a host with > a resolved IP address no longer match for selection when > hadoop.security.token.service.use_ip is set false. The test assumes that > 127.0.0.1 resolves to host name "localhost". On Windows, this is not the > case, and instead it resolves to "127.0.0.1". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
