[jira] [Updated] (YARN-1078) TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater fail on Windows
[
https://issues.apache.org/jira/browse/YARN-1078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated YARN-1078:
Target Version/s: 3.0.0, 2.1.1-beta
Affects Version/s: (was: 2.3.0)
2.1.1-beta
> TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater
> fail on Windows
> -
>
> Key: YARN-1078
> URL: https://issues.apache.org/jira/browse/YARN-1078
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.1.1-beta
>Reporter: Chuan Liu
>Assignee: Chuan Liu
>Priority: Minor
> Attachments: YARN-1078.2.patch, YARN-1078.3.patch,
> YARN-1078.branch-2.patch, YARN-1078.patch
>
>
> The three unit tests fail on Windows due to host name resolution differences
> on Windows, i.e. 127.0.0.1 does not resolve to host name "localhost".
> {noformat}
> org.apache.hadoop.security.token.SecretManager$InvalidToken: Given Container
> container_0__01_00 identifier is not valid for current Node manager.
> Expected : 127.0.0.1:12345 Found : localhost:12345
> {noformat}
> {noformat}
> testNMConnectionToRM(org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater)
> Time elapsed: 8343 sec <<< FAILURE!
> org.junit.ComparisonFailure: expected:<[localhost]:12345> but
> was:<[127.0.0.1]:12345>
> at org.junit.Assert.assertEquals(Assert.java:125)
> at org.junit.Assert.assertEquals(Assert.java:147)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater$MyResourceTracker6.registerNodeManager(TestNodeStatusUpdater.java:712)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:101)
> at $Proxy26.registerNodeManager(Unknown Source)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:212)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:149)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater$MyNodeStatusUpdater4.serviceStart(TestNodeStatusUpdater.java:369)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:213)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater.testNMConnectionToRM(TestNodeStatusUpdater.java:985)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-1078) TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater fail on Windows
[
https://issues.apache.org/jira/browse/YARN-1078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chuan Liu updated YARN-1078:
Attachment: YARN-1078.3.patch
Attach a branch-2 patch; and remove one empty line from the old patch.
> TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater
> fail on Windows
> -
>
> Key: YARN-1078
> URL: https://issues.apache.org/jira/browse/YARN-1078
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Chuan Liu
>Assignee: Chuan Liu
>Priority: Minor
> Attachments: YARN-1078.2.patch, YARN-1078.3.patch,
> YARN-1078.branch-2.patch, YARN-1078.patch
>
>
> The three unit tests fail on Windows due to host name resolution differences
> on Windows, i.e. 127.0.0.1 does not resolve to host name "localhost".
> {noformat}
> org.apache.hadoop.security.token.SecretManager$InvalidToken: Given Container
> container_0__01_00 identifier is not valid for current Node manager.
> Expected : 127.0.0.1:12345 Found : localhost:12345
> {noformat}
> {noformat}
> testNMConnectionToRM(org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater)
> Time elapsed: 8343 sec <<< FAILURE!
> org.junit.ComparisonFailure: expected:<[localhost]:12345> but
> was:<[127.0.0.1]:12345>
> at org.junit.Assert.assertEquals(Assert.java:125)
> at org.junit.Assert.assertEquals(Assert.java:147)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater$MyResourceTracker6.registerNodeManager(TestNodeStatusUpdater.java:712)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:101)
> at $Proxy26.registerNodeManager(Unknown Source)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:212)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:149)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater$MyNodeStatusUpdater4.serviceStart(TestNodeStatusUpdater.java:369)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:213)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater.testNMConnectionToRM(TestNodeStatusUpdater.java:985)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-1078) TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater fail on Windows
[
https://issues.apache.org/jira/browse/YARN-1078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chuan Liu updated YARN-1078:
Attachment: YARN-1078.branch-2.patch
> TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater
> fail on Windows
> -
>
> Key: YARN-1078
> URL: https://issues.apache.org/jira/browse/YARN-1078
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Chuan Liu
>Assignee: Chuan Liu
>Priority: Minor
> Attachments: YARN-1078.2.patch, YARN-1078.3.patch,
> YARN-1078.branch-2.patch, YARN-1078.patch
>
>
> The three unit tests fail on Windows due to host name resolution differences
> on Windows, i.e. 127.0.0.1 does not resolve to host name "localhost".
> {noformat}
> org.apache.hadoop.security.token.SecretManager$InvalidToken: Given Container
> container_0__01_00 identifier is not valid for current Node manager.
> Expected : 127.0.0.1:12345 Found : localhost:12345
> {noformat}
> {noformat}
> testNMConnectionToRM(org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater)
> Time elapsed: 8343 sec <<< FAILURE!
> org.junit.ComparisonFailure: expected:<[localhost]:12345> but
> was:<[127.0.0.1]:12345>
> at org.junit.Assert.assertEquals(Assert.java:125)
> at org.junit.Assert.assertEquals(Assert.java:147)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater$MyResourceTracker6.registerNodeManager(TestNodeStatusUpdater.java:712)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:101)
> at $Proxy26.registerNodeManager(Unknown Source)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:212)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:149)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater$MyNodeStatusUpdater4.serviceStart(TestNodeStatusUpdater.java:369)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:213)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater.testNMConnectionToRM(TestNodeStatusUpdater.java:985)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-1078) TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater fail on Windows
[
https://issues.apache.org/jira/browse/YARN-1078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated YARN-1078:
Hadoop Flags: Reviewed
+1 for the patch. I verified that tests pass on Mac and Windows.
Chuan, the current patch applies cleanly to trunk, but not branch-2 or
branch-2.1-beta. Can you please investigate and provide a branch-2 patch?
Thanks!
> TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater
> fail on Windows
> -
>
> Key: YARN-1078
> URL: https://issues.apache.org/jira/browse/YARN-1078
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Chuan Liu
>Assignee: Chuan Liu
>Priority: Minor
> Attachments: YARN-1078.2.patch, YARN-1078.patch
>
>
> The three unit tests fail on Windows due to host name resolution differences
> on Windows, i.e. 127.0.0.1 does not resolve to host name "localhost".
> {noformat}
> org.apache.hadoop.security.token.SecretManager$InvalidToken: Given Container
> container_0__01_00 identifier is not valid for current Node manager.
> Expected : 127.0.0.1:12345 Found : localhost:12345
> {noformat}
> {noformat}
> testNMConnectionToRM(org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater)
> Time elapsed: 8343 sec <<< FAILURE!
> org.junit.ComparisonFailure: expected:<[localhost]:12345> but
> was:<[127.0.0.1]:12345>
> at org.junit.Assert.assertEquals(Assert.java:125)
> at org.junit.Assert.assertEquals(Assert.java:147)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater$MyResourceTracker6.registerNodeManager(TestNodeStatusUpdater.java:712)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:101)
> at $Proxy26.registerNodeManager(Unknown Source)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:212)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:149)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater$MyNodeStatusUpdater4.serviceStart(TestNodeStatusUpdater.java:369)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:213)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater.testNMConnectionToRM(TestNodeStatusUpdater.java:985)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-1078) TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater fail on Windows
[
https://issues.apache.org/jira/browse/YARN-1078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chuan Liu updated YARN-1078:
Attachment: YARN-1078.2.patch
I looked into the failure. It turns out we use
InetAddress.getCanonicalHostName() to construct nodeId in
"ContainerManagerImpl". In the test, we assume this will always be "localhost"
for a local loop back address, i.e. 127.0.0.1. However, this is not the case on
Windows. As the method could return "127.0.0.1" on Windows instead of
"localhost". In the old patch, I switch from "localhost" to "127.0.0.1", and
regressed Linux. Attach a new patch that uses getCanonicalHostName() to obtain
the name for nodeId constructed in the tests.
> TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater
> fail on Windows
> -
>
> Key: YARN-1078
> URL: https://issues.apache.org/jira/browse/YARN-1078
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Chuan Liu
>Assignee: Chuan Liu
>Priority: Minor
> Attachments: YARN-1078.2.patch, YARN-1078.patch
>
>
> The three unit tests fail on Windows due to host name resolution differences
> on Windows, i.e. 127.0.0.1 does not resolve to host name "localhost".
> {noformat}
> org.apache.hadoop.security.token.SecretManager$InvalidToken: Given Container
> container_0__01_00 identifier is not valid for current Node manager.
> Expected : 127.0.0.1:12345 Found : localhost:12345
> {noformat}
> {noformat}
> testNMConnectionToRM(org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater)
> Time elapsed: 8343 sec <<< FAILURE!
> org.junit.ComparisonFailure: expected:<[localhost]:12345> but
> was:<[127.0.0.1]:12345>
> at org.junit.Assert.assertEquals(Assert.java:125)
> at org.junit.Assert.assertEquals(Assert.java:147)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater$MyResourceTracker6.registerNodeManager(TestNodeStatusUpdater.java:712)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:101)
> at $Proxy26.registerNodeManager(Unknown Source)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:212)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:149)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater$MyNodeStatusUpdater4.serviceStart(TestNodeStatusUpdater.java:369)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:213)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater.testNMConnectionToRM(TestNodeStatusUpdater.java:985)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-1078) TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater fail on Windows
[
https://issues.apache.org/jira/browse/YARN-1078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chuan Liu updated YARN-1078:
Attachment: YARN-1078.patch
Attach a patch. The fixes are quite straight forward.
> TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater
> fail on Windows
> -
>
> Key: YARN-1078
> URL: https://issues.apache.org/jira/browse/YARN-1078
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Chuan Liu
>Assignee: Chuan Liu
>Priority: Minor
> Attachments: YARN-1078.patch
>
>
> The three unit tests fail on Windows due to host name resolution differences
> on Windows, i.e. 127.0.0.1 does not resolve to host name "localhost".
> {noformat}
> org.apache.hadoop.security.token.SecretManager$InvalidToken: Given Container
> container_0__01_00 identifier is not valid for current Node manager.
> Expected : 127.0.0.1:12345 Found : localhost:12345
> {noformat}
> {noformat}
> testNMConnectionToRM(org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater)
> Time elapsed: 8343 sec <<< FAILURE!
> org.junit.ComparisonFailure: expected:<[localhost]:12345> but
> was:<[127.0.0.1]:12345>
> at org.junit.Assert.assertEquals(Assert.java:125)
> at org.junit.Assert.assertEquals(Assert.java:147)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater$MyResourceTracker6.registerNodeManager(TestNodeStatusUpdater.java:712)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:101)
> at $Proxy26.registerNodeManager(Unknown Source)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:212)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:149)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater$MyNodeStatusUpdater4.serviceStart(TestNodeStatusUpdater.java:369)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:213)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater.testNMConnectionToRM(TestNodeStatusUpdater.java:985)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
