[jira] [Assigned] (YARN-32) TestApplicationTokens fails intermintently on jdk7
[ https://issues.apache.org/jira/browse/YARN-32?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker reassigned YARN-32: - Assignee: Robert Parker (was: Thomas Graves) > TestApplicationTokens fails intermintently on jdk7 > -- > > Key: YARN-32 > URL: https://issues.apache.org/jira/browse/YARN-32 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 0.23.3 >Reporter: Thomas Graves >Assignee: Robert Parker > Labels: java7 > > TestApplicationsTokens fails intermintently on jdk7. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-32) TestApplicationTokens fails intermintently on jdk7
[ https://issues.apache.org/jira/browse/YARN-32?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-32: -- Attachment: YARN32.patch > TestApplicationTokens fails intermintently on jdk7 > -- > > Key: YARN-32 > URL: https://issues.apache.org/jira/browse/YARN-32 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 0.23.3 >Reporter: Thomas Graves >Assignee: Robert Parker > Labels: java7 > Fix For: 0.23.3 > > Attachments: YARN32.patch > > > TestApplicationsTokens fails intermintently on jdk7. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (YARN-144) MiniMRYarnCluster launches RM and JHS on default ports
Robert Parker created YARN-144: -- Summary: MiniMRYarnCluster launches RM and JHS on default ports Key: YARN-144 URL: https://issues.apache.org/jira/browse/YARN-144 Project: Hadoop YARN Issue Type: Bug Affects Versions: 0.23.4 Reporter: Robert Parker Assignee: Robert Parker Fix For: 0.23.5 MAPREDUCE-3867, MAPREDUCE-3869, and MAPREDUCE-4406 need to be combined and applied to branch-0.23 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-144) MiniMRYarnCluster launches RM and JHS on default ports
[ https://issues.apache.org/jira/browse/YARN-144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-144: --- Attachment: YARN-144-branch_0.23.patch > MiniMRYarnCluster launches RM and JHS on default ports > -- > > Key: YARN-144 > URL: https://issues.apache.org/jira/browse/YARN-144 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 0.23.4 >Reporter: Robert Parker >Assignee: Robert Parker > Fix For: 0.23.5 > > Attachments: YARN-144-branch_0.23.patch > > > MAPREDUCE-3867, MAPREDUCE-3869, and MAPREDUCE-4406 need to be combined and > applied to branch-0.23 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-144) MiniMRYarnCluster launches RM and JHS on default ports
[ https://issues.apache.org/jira/browse/YARN-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13471621#comment-13471621 ] Robert Parker commented on YARN-144: Verified that all affected unit tests completed correctly. > MiniMRYarnCluster launches RM and JHS on default ports > -- > > Key: YARN-144 > URL: https://issues.apache.org/jira/browse/YARN-144 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 0.23.4 >Reporter: Robert Parker >Assignee: Robert Parker > Fix For: 0.23.5 > > Attachments: YARN-144-branch_0.23.patch > > > MAPREDUCE-3867, MAPREDUCE-3869, and MAPREDUCE-4406 need to be combined and > applied to branch-0.23 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-32) TestApplicationTokens fails intermintently on jdk7
[ https://issues.apache.org/jira/browse/YARN-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473301#comment-13473301 ] Robert Parker commented on YARN-32: --- Vinod I could reproduce this issue on jdk6 when I reversed the order of the tests. The default ipc.client.connection.maxidletime is 10 seconds' so the connections hang around after shutdown and get used by the next test. > TestApplicationTokens fails intermintently on jdk7 > -- > > Key: YARN-32 > URL: https://issues.apache.org/jira/browse/YARN-32 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 0.23.3 >Reporter: Thomas Graves >Assignee: Robert Parker > Labels: java7 > Fix For: 0.23.4 > > Attachments: YARN32.patch > > > TestApplicationsTokens fails intermintently on jdk7. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-32) TestApplicationTokens fails intermintently on jdk7
[ https://issues.apache.org/jira/browse/YARN-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477901#comment-13477901 ] Robert Parker commented on YARN-32: --- Both tests explicitly stop the proxy currently. From my investigation, the current implementation does not kill the IPC threads until the max idle time has expired. > TestApplicationTokens fails intermintently on jdk7 > -- > > Key: YARN-32 > URL: https://issues.apache.org/jira/browse/YARN-32 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 0.23.3 >Reporter: Thomas Graves >Assignee: Robert Parker > Labels: java7 > Fix For: 0.23.5 > > Attachments: YARN32.patch > > > TestApplicationsTokens fails intermintently on jdk7. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-32) TestApplicationTokens fails intermintently on jdk7
[ https://issues.apache.org/jira/browse/YARN-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478319#comment-13478319 ] Robert Parker commented on YARN-32: --- Vinod, Thanks for the patch. It is much cleaner and the tests pass for JDK6 and JDK7. > TestApplicationTokens fails intermintently on jdk7 > -- > > Key: YARN-32 > URL: https://issues.apache.org/jira/browse/YARN-32 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 0.23.3 >Reporter: Thomas Graves >Assignee: Robert Parker > Labels: java7 > Fix For: 0.23.5 > > Attachments: YARN-32-20121017.txt, YARN32.patch > > > TestApplicationsTokens fails intermintently on jdk7. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-165) RM should point tracking URL to RM web page for app when AM fails
[ https://issues.apache.org/jira/browse/YARN-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487892#comment-13487892 ] Robert Parker commented on YARN-165: I am getting an NPE on the test added by YARN-165-branch23.patch > RM should point tracking URL to RM web page for app when AM fails > - > > Key: YARN-165 > URL: https://issues.apache.org/jira/browse/YARN-165 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.0.3-alpha, 0.23.5 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Attachments: YARN-165-branch23.patch, YARN-165.patch > > > Currently when an ApplicationMaster fails the ResourceManager is updating the > tracking URL to an empty string, see > RMAppAttemptImpl.ContainerFinishedTransition. Unfortunately when the client > attempts to follow the proxy URL it results in a web page showing an HTTP 500 > error and an ugly backtrace because "http://"; isn't a very helpful tracking > URL. > It would be much more helpful if the proxy URL redirected to the RM webapp > page for the specific application. That page shows the various AM attempts > and pointers to their logs which will be useful for debugging the problems > that caused the AM attempts to fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-144) MiniMRYarnCluster launches RM and JHS on default ports
[ https://issues.apache.org/jira/browse/YARN-144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-144: --- Attachment: YARN-144-branch_0.23-2.patch Jason nice catch! AdminService, ApplicationMasterService, ClientRMService, and ResourceTrackerService did not need any changes when I compared to trunk. Removed the CHANGES.txt. Retested and all passed. > MiniMRYarnCluster launches RM and JHS on default ports > -- > > Key: YARN-144 > URL: https://issues.apache.org/jira/browse/YARN-144 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 0.23.4 >Reporter: Robert Parker >Assignee: Robert Parker > Fix For: 0.23.5 > > Attachments: YARN-144-branch_0.23-2.patch, YARN-144-branch_0.23.patch > > > MAPREDUCE-3867, MAPREDUCE-3869, and MAPREDUCE-4406 need to be combined and > applied to branch-0.23 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-144) MiniMRYarnCluster launches RM and JHS on default ports
[ https://issues.apache.org/jira/browse/YARN-144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-144: --- Attachment: YARN-144-branch_0.23-3.patch > MiniMRYarnCluster launches RM and JHS on default ports > -- > > Key: YARN-144 > URL: https://issues.apache.org/jira/browse/YARN-144 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 0.23.4 >Reporter: Robert Parker >Assignee: Robert Parker > Fix For: 0.23.5 > > Attachments: YARN-144-branch_0.23-2.patch, > YARN-144-branch_0.23-3.patch, YARN-144-branch_0.23.patch > > > MAPREDUCE-3867, MAPREDUCE-3869, and MAPREDUCE-4406 need to be combined and > applied to branch-0.23 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Moved] (YARN-263) Modify Security Conditional that check for KERBEROS
[ https://issues.apache.org/jira/browse/YARN-263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker moved MAPREDUCE-4855 to YARN-263: --- Component/s: (was: security) Key: YARN-263 (was: MAPREDUCE-4855) Project: Hadoop YARN (was: Hadoop Map/Reduce) > Modify Security Conditional that check for KERBEROS > --- > > Key: YARN-263 > URL: https://issues.apache.org/jira/browse/YARN-263 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Robert Parker >Assignee: Robert Parker > > To support PLAIN authentication, checks should disallow certain types (TOKEN > for token delegation) instead of allowing only KERBEROS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-345) Many InvalidStateTransitonException errors for ApplicationImpl in Node Manager
[
https://issues.apache.org/jira/browse/YARN-345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Parker updated YARN-345:
---
Attachment: YARN-345.patch
> Many InvalidStateTransitonException errors for ApplicationImpl in Node Manager
> --
>
> Key: YARN-345
> URL: https://issues.apache.org/jira/browse/YARN-345
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.0.2-alpha, 2.0.1-alpha, 0.23.5
>Reporter: Devaraj K
>Assignee: Robert Parker
>Priority: Critical
> Attachments: YARN-345.patch
>
>
> {code:xml}
> org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event:
> FINISH_APPLICATION at FINISHED
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:301)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl.handle(ApplicationImpl.java:398)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl.handle(ApplicationImpl.java:58)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher.handle(ContainerManagerImpl.java:520)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher.handle(ContainerManagerImpl.java:512)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:126)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:75)
> at java.lang.Thread.run(Thread.java:662)
> {code}
> {code:xml}
> 2013-01-17 04:03:46,726 WARN
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application:
> Can't handle this event at current state
> org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event:
> FINISH_APPLICATION at APPLICATION_RESOURCES_CLEANINGUP
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:301)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl.handle(ApplicationImpl.java:398)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl.handle(ApplicationImpl.java:58)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher.handle(ContainerManagerImpl.java:520)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher.handle(ContainerManagerImpl.java:512)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:126)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:75)
> at java.lang.Thread.run(Thread.java:662)
> {code}
> {code:xml}
> 2013-01-17 00:01:11,006 WARN
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application:
> Can't handle this event at current state
> org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event:
> FINISH_APPLICATION at FINISHING_CONTAINERS_WAIT
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:301)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl.handle(ApplicationImpl.java:398)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl.handle(ApplicationImpl.java:58)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher.handle(ContainerManagerImpl.java:520)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher.handle(ContainerManagerImpl.java:512)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:126)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:75)
> at java.lang.Thread.run(Thread
[jira] [Updated] (YARN-345) Many InvalidStateTransitonException errors for ApplicationImpl in Node Manager
[
https://issues.apache.org/jira/browse/YARN-345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Parker updated YARN-345:
---
Attachment: YARN-354v2.patch
> Many InvalidStateTransitonException errors for ApplicationImpl in Node Manager
> --
>
> Key: YARN-345
> URL: https://issues.apache.org/jira/browse/YARN-345
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.0.2-alpha, 2.0.1-alpha, 0.23.5
>Reporter: Devaraj K
>Assignee: Robert Parker
>Priority: Critical
> Attachments: YARN-345.patch, YARN-354v2.patch
>
>
> {code:xml}
> org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event:
> FINISH_APPLICATION at FINISHED
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:301)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl.handle(ApplicationImpl.java:398)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl.handle(ApplicationImpl.java:58)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher.handle(ContainerManagerImpl.java:520)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher.handle(ContainerManagerImpl.java:512)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:126)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:75)
> at java.lang.Thread.run(Thread.java:662)
> {code}
> {code:xml}
> 2013-01-17 04:03:46,726 WARN
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application:
> Can't handle this event at current state
> org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event:
> FINISH_APPLICATION at APPLICATION_RESOURCES_CLEANINGUP
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:301)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl.handle(ApplicationImpl.java:398)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl.handle(ApplicationImpl.java:58)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher.handle(ContainerManagerImpl.java:520)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher.handle(ContainerManagerImpl.java:512)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:126)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:75)
> at java.lang.Thread.run(Thread.java:662)
> {code}
> {code:xml}
> 2013-01-17 00:01:11,006 WARN
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application:
> Can't handle this event at current state
> org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event:
> FINISH_APPLICATION at FINISHING_CONTAINERS_WAIT
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:301)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl.handle(ApplicationImpl.java:398)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl.handle(ApplicationImpl.java:58)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher.handle(ContainerManagerImpl.java:520)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher.handle(ContainerManagerImpl.java:512)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:126)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:75)
> at java.la
[jira] [Created] (YARN-819) ResourceManager and NodeManager mismatched version should create an error
Robert Parker created YARN-819: -- Summary: ResourceManager and NodeManager mismatched version should create an error Key: YARN-819 URL: https://issues.apache.org/jira/browse/YARN-819 Project: Hadoop YARN Issue Type: Improvement Reporter: Robert Parker Assignee: Robert Parker Our use case is during upgrade on a large cluster several NodeManagers may not restart with the new version. Once the RM comes back up the NodeManager will re-register without issue to the RM. The NM should report the version the RM. The RM should have a configuration to disallow the check (default), equal to the RM (to prevent config change for each release), equal to or greater than RM (to allow NM upgrades), and finally an explicit version or version range. The RM should also have an configuration on how to treat the mismatch: REJECT, or REBOOT the NM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-819) ResourceManager and NodeManager mismatched version should create an error
[ https://issues.apache.org/jira/browse/YARN-819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-819: --- Affects Version/s: 0.23.8 > ResourceManager and NodeManager mismatched version should create an error > - > > Key: YARN-819 > URL: https://issues.apache.org/jira/browse/YARN-819 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 0.23.8 >Reporter: Robert Parker >Assignee: Robert Parker > > Our use case is during upgrade on a large cluster several NodeManagers may > not restart with the new version. Once the RM comes back up the NodeManager > will re-register without issue to the RM. > The NM should report the version the RM. The RM should have a configuration > to disallow the check (default), equal to the RM (to prevent config change > for each release), equal to or greater than RM (to allow NM upgrades), and > finally an explicit version or version range. > The RM should also have an configuration on how to treat the mismatch: > REJECT, or REBOOT the NM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-819) ResourceManager and NodeManager mismatched version should create an error
[ https://issues.apache.org/jira/browse/YARN-819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-819: --- Issue Type: Sub-task (was: Improvement) Parent: YARN-666 > ResourceManager and NodeManager mismatched version should create an error > - > > Key: YARN-819 > URL: https://issues.apache.org/jira/browse/YARN-819 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 0.23.8 >Reporter: Robert Parker >Assignee: Robert Parker > > Our use case is during upgrade on a large cluster several NodeManagers may > not restart with the new version. Once the RM comes back up the NodeManager > will re-register without issue to the RM. > The NM should report the version the RM. The RM should have a configuration > to disallow the check (default), equal to the RM (to prevent config change > for each release), equal to or greater than RM (to allow NM upgrades), and > finally an explicit version or version range. > The RM should also have an configuration on how to treat the mismatch: > REJECT, or REBOOT the NM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-819) ResourceManager and NodeManager mismatched version should create an error
[ https://issues.apache.org/jira/browse/YARN-819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-819: --- Component/s: resourcemanager nodemanager > ResourceManager and NodeManager mismatched version should create an error > - > > Key: YARN-819 > URL: https://issues.apache.org/jira/browse/YARN-819 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Affects Versions: 2.0.4-alpha, 0.23.8 >Reporter: Robert Parker >Assignee: Robert Parker > > Our use case is during upgrade on a large cluster several NodeManagers may > not restart with the new version. Once the RM comes back up the NodeManager > will re-register without issue to the RM. > The NM should report the version the RM. The RM should have a configuration > to disallow the check (default), equal to the RM (to prevent config change > for each release), equal to or greater than RM (to allow NM upgrades), and > finally an explicit version or version range. > The RM should also have an configuration on how to treat the mismatch: > REJECT, or REBOOT the NM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-819) ResourceManager and NodeManager mismatched version should create an error
[ https://issues.apache.org/jira/browse/YARN-819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-819: --- Target Version/s: 3.0.0, 2.0.5-alpha (was: 3.0.0, 2.0.5-alpha, 0.23.9) > ResourceManager and NodeManager mismatched version should create an error > - > > Key: YARN-819 > URL: https://issues.apache.org/jira/browse/YARN-819 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Affects Versions: 2.0.4-alpha, 0.23.8 >Reporter: Robert Parker >Assignee: Robert Parker > > Our use case is during upgrade on a large cluster several NodeManagers may > not restart with the new version. Once the RM comes back up the NodeManager > will re-register without issue to the RM. > The NM should report the version the RM. The RM should have a configuration > to disallow the check (default), equal to the RM (to prevent config change > for each release), equal to or greater than RM (to allow NM upgrades), and > finally an explicit version or version range. > The RM should also have an configuration on how to treat the mismatch: > REJECT, or REBOOT the NM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-819) ResourceManager and NodeManager mismatched version should create an error
[ https://issues.apache.org/jira/browse/YARN-819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-819: --- Affects Version/s: 2.0.4-alpha > ResourceManager and NodeManager mismatched version should create an error > - > > Key: YARN-819 > URL: https://issues.apache.org/jira/browse/YARN-819 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.0.4-alpha, 0.23.8 >Reporter: Robert Parker >Assignee: Robert Parker > > Our use case is during upgrade on a large cluster several NodeManagers may > not restart with the new version. Once the RM comes back up the NodeManager > will re-register without issue to the RM. > The NM should report the version the RM. The RM should have a configuration > to disallow the check (default), equal to the RM (to prevent config change > for each release), equal to or greater than RM (to allow NM upgrades), and > finally an explicit version or version range. > The RM should also have an configuration on how to treat the mismatch: > REJECT, or REBOOT the NM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-819) ResourceManager and NodeManager should check for a minimum allowed version
[ https://issues.apache.org/jira/browse/YARN-819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-819: --- Summary: ResourceManager and NodeManager should check for a minimum allowed version (was: ResourceManager and NodeManager mismatched version should create an error) > ResourceManager and NodeManager should check for a minimum allowed version > -- > > Key: YARN-819 > URL: https://issues.apache.org/jira/browse/YARN-819 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Affects Versions: 2.0.4-alpha, 0.23.8 >Reporter: Robert Parker >Assignee: Robert Parker > > Our use case is during upgrade on a large cluster several NodeManagers may > not restart with the new version. Once the RM comes back up the NodeManager > will re-register without issue to the RM. > The NM should report the version the RM. The RM should have a configuration > to disallow the check (default), equal to the RM (to prevent config change > for each release), equal to or greater than RM (to allow NM upgrades), and > finally an explicit version or version range. > The RM should also have an configuration on how to treat the mismatch: > REJECT, or REBOOT the NM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-862) ResourceManager and NodeManager versions should match on node registration or error out
[ https://issues.apache.org/jira/browse/YARN-862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-862: --- Summary: ResourceManager and NodeManager versions should match on node registration or error out (was: ResourceManager and NodeManager versions should on node registration or error out) > ResourceManager and NodeManager versions should match on node registration or > error out > --- > > Key: YARN-862 > URL: https://issues.apache.org/jira/browse/YARN-862 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager, resourcemanager >Affects Versions: 0.23.8 >Reporter: Robert Parker >Assignee: Robert Parker > > For branch-0.23 the versions of the node manager and the resource manager > should match to complete a successful registration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (YARN-862) ResourceManager and NodeManager versions should on node registration or error out
Robert Parker created YARN-862: -- Summary: ResourceManager and NodeManager versions should on node registration or error out Key: YARN-862 URL: https://issues.apache.org/jira/browse/YARN-862 Project: Hadoop YARN Issue Type: Bug Components: nodemanager, resourcemanager Affects Versions: 0.23.8 Reporter: Robert Parker Assignee: Robert Parker For branch-0.23 the versions of the node manager and the resource manager should match to complete a successful registration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-862) ResourceManager and NodeManager versions should match on node registration or error out
[ https://issues.apache.org/jira/browse/YARN-862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-862: --- Attachment: YARN-862-b0.23-v1.patch > ResourceManager and NodeManager versions should match on node registration or > error out > --- > > Key: YARN-862 > URL: https://issues.apache.org/jira/browse/YARN-862 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager, resourcemanager >Affects Versions: 0.23.8 >Reporter: Robert Parker >Assignee: Robert Parker > Attachments: YARN-862-b0.23-v1.patch > > > For branch-0.23 the versions of the node manager and the resource manager > should match to complete a successful registration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-862) ResourceManager and NodeManager versions should match on node registration or error out
[ https://issues.apache.org/jira/browse/YARN-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689515#comment-13689515 ] Robert Parker commented on YARN-862: [~ojoshi] yes this change is not compatible with rolling upgrades, that work will be in YARN-819 which will allow version checking that will be compatible with rolling upgrades. This change may not be valid outside of the Yahoo! use case for current 0.23 were old version NM connects to the new RM without note during cluster upgrades. > ResourceManager and NodeManager versions should match on node registration or > error out > --- > > Key: YARN-862 > URL: https://issues.apache.org/jira/browse/YARN-862 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager, resourcemanager >Affects Versions: 0.23.8 >Reporter: Robert Parker >Assignee: Robert Parker > Attachments: YARN-862-b0.23-v1.patch > > > For branch-0.23 the versions of the node manager and the resource manager > should match to complete a successful registration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-862) ResourceManager and NodeManager versions should match on node registration or error out
[ https://issues.apache.org/jira/browse/YARN-862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-862: --- Attachment: YARN-862-b0.23-v2.patch Correct unit test failures > ResourceManager and NodeManager versions should match on node registration or > error out > --- > > Key: YARN-862 > URL: https://issues.apache.org/jira/browse/YARN-862 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager, resourcemanager >Affects Versions: 0.23.8 >Reporter: Robert Parker >Assignee: Robert Parker > Attachments: YARN-862-b0.23-v1.patch, YARN-862-b0.23-v2.patch > > > For branch-0.23 the versions of the node manager and the resource manager > should match to complete a successful registration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-819) ResourceManager and NodeManager should check for a minimum allowed version
[ https://issues.apache.org/jira/browse/YARN-819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-819: --- Attachment: YARN-819-1.patch > ResourceManager and NodeManager should check for a minimum allowed version > -- > > Key: YARN-819 > URL: https://issues.apache.org/jira/browse/YARN-819 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Affects Versions: 2.0.4-alpha, 0.23.8 >Reporter: Robert Parker >Assignee: Robert Parker > Attachments: YARN-819-1.patch > > > Our use case is during upgrade on a large cluster several NodeManagers may > not restart with the new version. Once the RM comes back up the NodeManager > will re-register without issue to the RM. > The NM should report the version the RM. The RM should have a configuration > to disallow the check (default), equal to the RM (to prevent config change > for each release), equal to or greater than RM (to allow NM upgrades), and > finally an explicit version or version range. > The RM should also have an configuration on how to treat the mismatch: > REJECT, or REBOOT the NM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (YARN-1101) Active nodes can be decremented below 0
Robert Parker created YARN-1101: --- Summary: Active nodes can be decremented below 0 Key: YARN-1101 URL: https://issues.apache.org/jira/browse/YARN-1101 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.0.6-alpha, 0.23.9, 3.0.0 Reporter: Robert Parker Assignee: Robert Parker The issue is in RMNodeImpl where both RUNNING and UNHEALTHY states that transition to a deactive state (LOST, DECOMMISSIONED, REBOOTED) use the same DeactivateNodeTransition class. The DeactivateNodeTransition class naturally decrements the active node, however the in cases where the node has transition to UNHEALTHY the active count has already been decremented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-1101) Active nodes can be decremented below 0
[ https://issues.apache.org/jira/browse/YARN-1101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-1101: Attachment: YARN-1101_v1.patch > Active nodes can be decremented below 0 > --- > > Key: YARN-1101 > URL: https://issues.apache.org/jira/browse/YARN-1101 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 3.0.0, 0.23.9, 2.0.6-alpha >Reporter: Robert Parker >Assignee: Robert Parker > Attachments: YARN-1101_b0.23_v1.patch, YARN-1101_v1.patch > > > The issue is in RMNodeImpl where both RUNNING and UNHEALTHY states that > transition to a deactive state (LOST, DECOMMISSIONED, REBOOTED) use the same > DeactivateNodeTransition class. The DeactivateNodeTransition class naturally > decrements the active node, however the in cases where the node has > transition to UNHEALTHY the active count has already been decremented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-1101) Active nodes can be decremented below 0
[ https://issues.apache.org/jira/browse/YARN-1101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-1101: Attachment: YARN-1101_b0.23_v1.patch > Active nodes can be decremented below 0 > --- > > Key: YARN-1101 > URL: https://issues.apache.org/jira/browse/YARN-1101 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 3.0.0, 0.23.9, 2.0.6-alpha >Reporter: Robert Parker >Assignee: Robert Parker > Attachments: YARN-1101_b0.23_v1.patch, YARN-1101_v1.patch > > > The issue is in RMNodeImpl where both RUNNING and UNHEALTHY states that > transition to a deactive state (LOST, DECOMMISSIONED, REBOOTED) use the same > DeactivateNodeTransition class. The DeactivateNodeTransition class naturally > decrements the active node, however the in cases where the node has > transition to UNHEALTHY the active count has already been decremented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (YARN-1119) Add ClusterMetrics checks to tho TestRMNodeTransitions tests
Robert Parker created YARN-1119: --- Summary: Add ClusterMetrics checks to tho TestRMNodeTransitions tests Key: YARN-1119 URL: https://issues.apache.org/jira/browse/YARN-1119 Project: Hadoop YARN Issue Type: Test Components: resourcemanager Affects Versions: 2.0.6-alpha, 0.23.9, 3.0.0 Reporter: Robert Parker Assignee: Robert Parker YARN-1101 identified an issue where UNHEALTHY nodes could double decrement the active nodes. We should add checks for RUNNING node transitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1101) Active nodes can be decremented below 0
[ https://issues.apache.org/jira/browse/YARN-1101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13752852#comment-13752852 ] Robert Parker commented on YARN-1101: - Tom, Thanks for the review. I have added YARN-1119 to address improving the other tests in this Test Class. > Active nodes can be decremented below 0 > --- > > Key: YARN-1101 > URL: https://issues.apache.org/jira/browse/YARN-1101 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 3.0.0, 0.23.9, 2.0.6-alpha >Reporter: Robert Parker >Assignee: Robert Parker > Attachments: YARN-1101_b0.23_v1.patch, YARN-1101_v1.patch > > > The issue is in RMNodeImpl where both RUNNING and UNHEALTHY states that > transition to a deactive state (LOST, DECOMMISSIONED, REBOOTED) use the same > DeactivateNodeTransition class. The DeactivateNodeTransition class naturally > decrements the active node, however the in cases where the node has > transition to UNHEALTHY the active count has already been decremented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-1119) Add ClusterMetrics checks to tho TestRMNodeTransitions tests
[ https://issues.apache.org/jira/browse/YARN-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-1119: Assignee: (was: Robert Parker) > Add ClusterMetrics checks to tho TestRMNodeTransitions tests > > > Key: YARN-1119 > URL: https://issues.apache.org/jira/browse/YARN-1119 > Project: Hadoop YARN > Issue Type: Test > Components: resourcemanager >Affects Versions: 3.0.0, 0.23.9, 2.0.6-alpha >Reporter: Robert Parker > > YARN-1101 identified an issue where UNHEALTHY nodes could double decrement > the active nodes. We should add checks for RUNNING node transitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-819) ResourceManager and NodeManager should check for a minimum allowed version
[ https://issues.apache.org/jira/browse/YARN-819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-819: --- Attachment: YARN-819-2.patch > ResourceManager and NodeManager should check for a minimum allowed version > -- > > Key: YARN-819 > URL: https://issues.apache.org/jira/browse/YARN-819 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Affects Versions: 2.0.4-alpha >Reporter: Robert Parker >Assignee: Robert Parker > Attachments: YARN-819-1.patch, YARN-819-2.patch > > > Our use case is during upgrade on a large cluster several NodeManagers may > not restart with the new version. Once the RM comes back up the NodeManager > will re-register without issue to the RM. > The NM should report the version the RM. The RM should have a configuration > to disallow the check (default), equal to the RM (to prevent config change > for each release), equal to or greater than RM (to allow NM upgrades), and > finally an explicit version or version range. > The RM should also have an configuration on how to treat the mismatch: > REJECT, or REBOOT the NM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1199) Make NM/RM Versions Available
[ https://issues.apache.org/jira/browse/YARN-1199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13771056#comment-13771056 ] Robert Parker commented on YARN-1199: - +1 lgtm (non-binding) > Make NM/RM Versions Available > - > > Key: YARN-1199 > URL: https://issues.apache.org/jira/browse/YARN-1199 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Mit Desai >Assignee: Mit Desai > Attachments: YARN-1199.patch, YARN-1199.patch > > > Now as we have the NM and RM Versions available, we can display the YARN > version of nodes running in the cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1199) Make NM/RM Versions Available
[
https://issues.apache.org/jira/browse/YARN-1199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13771311#comment-13771311
]
Robert Parker commented on YARN-1199:
-
Mit, when running the tests I am getting:
{noformat}
testNodesQueryNew(org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes)
Time elapsed: 0.242 sec <<< FAILURE!
java.lang.AssertionError: incorrect number of elements expected:<10> but
was:<11>
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes.verifyNodeInfo(TestRMWebServicesNodes.java:663)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes.testNodesQueryNew(TestRMWebServicesNodes.java:189)
{noformat}
Need to account for the extra version field.
> Make NM/RM Versions Available
> -
>
> Key: YARN-1199
> URL: https://issues.apache.org/jira/browse/YARN-1199
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Mit Desai
>Assignee: Mit Desai
> Attachments: YARN-1199.patch, YARN-1199.patch
>
>
> Now as we have the NM and RM Versions available, we can display the YARN
> version of nodes running in the cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-819) ResourceManager and NodeManager should check for a minimum allowed version
[ https://issues.apache.org/jira/browse/YARN-819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13772993#comment-13772993 ] Robert Parker commented on YARN-819: Here are the primary use cases that this change is trying to cover Scenario 1) Upgrade the Node Managers (NMs) first in a rolling upgrade, then upgrade the Resoure Manager(RM) # Config required: #* yarn.resourcemanager.nodemanager.minimum.version => EqualToRM #* yarn.nodemanager.resourcemanager.minimum.version => NONE # The RM will ensure all NM have at least the same version as the RM, therefore the NMs can be upgraded in a incremental manner. # In this scenario the RM could precede the NMs for an emergency bug fix by changing the yarn.resourcemanager.nodemanager.minimum.version config to the Original RM version Scenario 2) Upgrade of the RM first then perform a rolling upgrade on the NMs. # Config required: #* yarn.resourcemanager.nodemanager.minimum.version => NONE #* yarn.nodemanager.resourcemanager.minimum.version => EqualToNM # The NMs will ensure that the RM version at least the same version as the current NM's version. Scenario 3) Ensure the RM version is equal the NM version. # Config required: #* yarn.resourcemanager.nodemanager.minimum.version => EqualToRM #* yarn.nodemanager.resourcemanager.minimum.version => EqualToNM # Since the check is done on both the RM and the NM the versions will have to be equal. > ResourceManager and NodeManager should check for a minimum allowed version > -- > > Key: YARN-819 > URL: https://issues.apache.org/jira/browse/YARN-819 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Affects Versions: 2.0.4-alpha >Reporter: Robert Parker >Assignee: Robert Parker > Attachments: YARN-819-1.patch, YARN-819-2.patch > > > Our use case is during upgrade on a large cluster several NodeManagers may > not restart with the new version. Once the RM comes back up the NodeManager > will re-register without issue to the RM. > The NM should report the version the RM. The RM should have a configuration > to disallow the check (default), equal to the RM (to prevent config change > for each release), equal to or greater than RM (to allow NM upgrades), and > finally an explicit version or version range. > The RM should also have an configuration on how to treat the mismatch: > REJECT, or REBOOT the NM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-819) ResourceManager and NodeManager should check for a minimum allowed version
[ https://issues.apache.org/jira/browse/YARN-819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated YARN-819: --- Attachment: YARN-819-3.patch > ResourceManager and NodeManager should check for a minimum allowed version > -- > > Key: YARN-819 > URL: https://issues.apache.org/jira/browse/YARN-819 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Affects Versions: 2.0.4-alpha >Reporter: Robert Parker >Assignee: Robert Parker > Attachments: YARN-819-1.patch, YARN-819-2.patch, YARN-819-3.patch > > > Our use case is during upgrade on a large cluster several NodeManagers may > not restart with the new version. Once the RM comes back up the NodeManager > will re-register without issue to the RM. > The NM should report the version the RM. The RM should have a configuration > to disallow the check (default), equal to the RM (to prevent config change > for each release), equal to or greater than RM (to allow NM upgrades), and > finally an explicit version or version range. > The RM should also have an configuration on how to treat the mismatch: > REJECT, or REBOOT the NM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-819) ResourceManager and NodeManager should check for a minimum allowed version
[ https://issues.apache.org/jira/browse/YARN-819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13775697#comment-13775697 ] Robert Parker commented on YARN-819: Jon, Thanks for the review and nice catch on branch-2. * Changed the TestResourceTrackerService#testNodeRegistrationVersionLessThanRM test case to run on branch-2 and greater. * The jira mentions reboot as on option. A reboot would cover the case where new software is deployed but the NM process is not restarted. There is no guarantee that the new version can talk to the older version so rejection of the connection will satisfy the requirement and is much less complicated. * Corrected the other three code issues. > ResourceManager and NodeManager should check for a minimum allowed version > -- > > Key: YARN-819 > URL: https://issues.apache.org/jira/browse/YARN-819 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Affects Versions: 2.0.4-alpha >Reporter: Robert Parker >Assignee: Robert Parker > Attachments: YARN-819-1.patch, YARN-819-2.patch, YARN-819-3.patch > > > Our use case is during upgrade on a large cluster several NodeManagers may > not restart with the new version. Once the RM comes back up the NodeManager > will re-register without issue to the RM. > The NM should report the version the RM. The RM should have a configuration > to disallow the check (default), equal to the RM (to prevent config change > for each release), equal to or greater than RM (to allow NM upgrades), and > finally an explicit version or version range. > The RM should also have an configuration on how to treat the mismatch: > REJECT, or REBOOT the NM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1199) Make NM/RM Versions Available
[ https://issues.apache.org/jira/browse/YARN-1199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13775724#comment-13775724 ] Robert Parker commented on YARN-1199: - +1 non-binding > Make NM/RM Versions Available > - > > Key: YARN-1199 > URL: https://issues.apache.org/jira/browse/YARN-1199 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Mit Desai >Assignee: Mit Desai > Attachments: YARN-1199.patch, YARN-1199.patch, YARN-1199.patch > > > Now as we have the NM and RM Versions available, we can display the YARN > version of nodes running in the cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-658) Command to kill a YARN application does not work with newer Ubuntu versions
[ https://issues.apache.org/jira/browse/YARN-658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13784250#comment-13784250 ] Robert Parker commented on YARN-658: The issue is with the kill command in later Ubuntu versions. > Command to kill a YARN application does not work with newer Ubuntu versions > --- > > Key: YARN-658 > URL: https://issues.apache.org/jira/browse/YARN-658 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.0.3-alpha, 2.0.4-alpha >Reporter: David Yan > Attachments: AppMaster.stderr, > yarn-david-nodemanager-david-ubuntu.out, > yarn-david-resourcemanager-david-ubuntu.out > > > After issuing a KillApplicationRequest, the application keeps running on the > system even though the state is changed to KILLED. It happens on both Ubuntu > 12.10 and 13.04, but works fine on Ubuntu 12.04. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-658) Command to kill a YARN application does not work with newer Ubuntu versions
[ https://issues.apache.org/jira/browse/YARN-658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13784282#comment-13784282 ] Robert Parker commented on YARN-658: looking at https://launchpad.net/ubuntu/+source/procps it appears that 13.10 will still deploy procps-3.3.3 and the /bin/kill parameter parsing bug is not fixed until 3.3.4. > Command to kill a YARN application does not work with newer Ubuntu versions > --- > > Key: YARN-658 > URL: https://issues.apache.org/jira/browse/YARN-658 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.0.3-alpha, 2.0.4-alpha >Reporter: David Yan > Attachments: AppMaster.stderr, > yarn-david-nodemanager-david-ubuntu.out, > yarn-david-resourcemanager-david-ubuntu.out > > > After issuing a KillApplicationRequest, the application keeps running on the > system even though the state is changed to KILLED. It happens on both Ubuntu > 12.10 and 13.04, but works fine on Ubuntu 12.04. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-658) Command to kill a YARN application does not work with newer Ubuntu versions
[ https://issues.apache.org/jira/browse/YARN-658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13784550#comment-13784550 ] Robert Parker commented on YARN-658: The problem is reproducible in mvn test -Dtest=org.apache.hadoop.yarn.server.nodemanager.TestNodeManagerShutdown I have confirmed that kill version 3.3.8 does not have this problem by installing it ahead of /bin/kill and running the above test(no code change required). I had a half baked patch that I will dig up, clean up and post. The patch precedes the - with a ' -- ' . I had hoped Ubuntu would fix this sooner but that sadly does not appear to be the case. > Command to kill a YARN application does not work with newer Ubuntu versions > --- > > Key: YARN-658 > URL: https://issues.apache.org/jira/browse/YARN-658 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.0.3-alpha, 2.0.4-alpha >Reporter: David Yan > Attachments: AppMaster.stderr, > yarn-david-nodemanager-david-ubuntu.out, > yarn-david-resourcemanager-david-ubuntu.out > > > After issuing a KillApplicationRequest, the application keeps running on the > system even though the state is changed to KILLED. It happens on both Ubuntu > 12.10 and 13.04, but works fine on Ubuntu 12.04. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (YARN-658) Command to kill a YARN application does not work with newer Ubuntu versions
[ https://issues.apache.org/jira/browse/YARN-658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker reassigned YARN-658: -- Assignee: Robert Parker > Command to kill a YARN application does not work with newer Ubuntu versions > --- > > Key: YARN-658 > URL: https://issues.apache.org/jira/browse/YARN-658 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.0.3-alpha, 2.0.4-alpha >Reporter: David Yan >Assignee: Robert Parker > Attachments: AppMaster.stderr, > yarn-david-nodemanager-david-ubuntu.out, > yarn-david-resourcemanager-david-ubuntu.out > > > After issuing a KillApplicationRequest, the application keeps running on the > system even though the state is changed to KILLED. It happens on both Ubuntu > 12.10 and 13.04, but works fine on Ubuntu 12.04. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (YARN-658) Command to kill a YARN application does not work with newer Ubuntu versions
[ https://issues.apache.org/jira/browse/YARN-658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker resolved YARN-658. Resolution: Duplicate > Command to kill a YARN application does not work with newer Ubuntu versions > --- > > Key: YARN-658 > URL: https://issues.apache.org/jira/browse/YARN-658 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.0.3-alpha, 2.0.4-alpha >Reporter: David Yan >Assignee: Robert Parker > Attachments: AppMaster.stderr, > yarn-david-nodemanager-david-ubuntu.out, > yarn-david-resourcemanager-david-ubuntu.out > > > After issuing a KillApplicationRequest, the application keeps running on the > system even though the state is changed to KILLED. It happens on both Ubuntu > 12.10 and 13.04, but works fine on Ubuntu 12.04. -- This message was sent by Atlassian JIRA (v6.1#6144)
