[jira] [Commented] (YARN-5611) Provide an API to update lifetime of an application.

2016-09-23 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516508#comment-15516508 ] Jian He commented on YARN-5611: --- - I thought the lifetime passed in the updateLifetime API is an absolute

[jira] [Commented] (YARN-5610) Initial code for native services REST API

2016-09-23 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15515597#comment-15515597 ] Jian He commented on YARN-5610: --- bq. The reason these attributes are at the Application level as well is

[jira] [Updated] (YARN-5662) Provide an option to enable ContainerMonitor

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5662: -- Attachment: YARN-5662.2.patch > Provide an option to enable ContainerMonitor >

[jira] [Comment Edited] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513480#comment-15513480 ] Jian He edited comment on YARN-5609 at 9/22/16 2:44 PM: looks good to me overall,

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513480#comment-15513480 ] Jian He commented on YARN-5609: --- looks good to me overall, I think wherever {{setIsReInitializing(false)}}

[jira] [Commented] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513402#comment-15513402 ] Jian He commented on YARN-3139: --- There are also similar comments like this... I just couldn't remember why

[jira] [Issue Comment Deleted] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-3139: -- Comment: was deleted (was: May be it's related to some consistency issues with respect to queue stats?) >

[jira] [Comment Edited] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513402#comment-15513402 ] Jian He edited comment on YARN-3139 at 9/22/16 2:20 PM: There are also similar

[jira] [Commented] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513416#comment-15513416 ] Jian He commented on YARN-3139: --- May be it's related to some consistency issues with respect to queue stats?

[jira] [Commented] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513407#comment-15513407 ] Jian He commented on YARN-3139: --- [~templedf], would you like to check the FairScheduler changes? > Improve

[jira] [Updated] (YARN-5662) Provide an option to enable ContainerMonitor

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5662: -- Attachment: YARN-5662.1.patch > Provide an option to enable ContainerMonitor >

[jira] [Created] (YARN-5662) Provide an option to enable ContainerMonitor

2016-09-22 Thread Jian He (JIRA)
Jian He created YARN-5662: - Summary: Provide an option to enable ContainerMonitor Key: YARN-5662 URL: https://issues.apache.org/jira/browse/YARN-5662 Project: Hadoop YARN Issue Type: Bug

[jira] [Updated] (YARN-5662) Provide an option to enable ContainerMonitor

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5662: -- Description: Currently, if vmem/pmem check is not enabled, ContainerMonitor would not run. In certain cases,

[jira] [Updated] (YARN-5662) Provide an option to enable ContainerMonitor

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5662: -- Release Note: (was: Currently, if vmem/pmem check is not enabled, ContainerMonitor would not run. In certain

[jira] [Issue Comment Deleted] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5609: -- Comment: was deleted (was: IIUC, even if it's a merged operation, the current will be a superset of previous,

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512205#comment-15512205 ] Jian He commented on YARN-5609: --- IIUC, even if it's a merged operation, the current will be a superset of

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512203#comment-15512203 ] Jian He commented on YARN-5609: --- IIUC, even if it's a merged operation, the current will be a superset of

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512108#comment-15512108 ] Jian He commented on YARN-5609: --- bq. during rollback, we should not merge the current resourceSet with new

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510458#comment-15510458 ] Jian He commented on YARN-5609: --- Yep, what I meant is the second last patch is fine, as the issue I raised

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510111#comment-15510111 ] Jian He commented on YARN-5609: --- I had re-checked the code... actually, even the old resources will also not

[jira] [Commented] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler

2016-09-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509412#comment-15509412 ] Jian He commented on YARN-3139: --- - Is removing these locks ok? {code} // It is crucial to acquire leaf

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-20 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15508770#comment-15508770 ] Jian He commented on YARN-5609: --- bq. since it is always called in conjunctions with a

[jira] [Comment Edited] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-20 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15508741#comment-15508741 ] Jian He edited comment on YARN-5609 at 9/21/16 4:58 AM: bq. I had intentionally

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-20 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15508741#comment-15508741 ] Jian He commented on YARN-5609: --- bq. I had intentionally kept it that way (my thinking was that the Tracker

[jira] [Comment Edited] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-20 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15508689#comment-15508689 ] Jian He edited comment on YARN-5621 at 9/21/16 4:40 AM: Thanks for your

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-20 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15508689#comment-15508689 ] Jian He commented on YARN-5621: --- Thanks for your explanation, some parts I'm missing: bq. Both will start

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-20 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506031#comment-15506031 ] Jian He commented on YARN-5609: --- - I think this will cause the resources to be re-requested on restart. Even

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505601#comment-15505601 ] Jian He commented on YARN-5621: --- My original thought is to create the symlinks right after localization

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505585#comment-15505585 ] Jian He commented on YARN-5609: --- Thanks Arun, some more comments: - IIUC, when restarting the container, the

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503725#comment-15503725 ] Jian He commented on YARN-5609: --- - Can you add comments in commitLastReInitialization that once committed,

[jira] [Commented] (YARN-5611) Provide an API to update lifetime of an application.

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502879#comment-15502879 ] Jian He commented on YARN-5611: --- sounds good to me. we can make sure code reused at server side. > Provide

[jira] [Issue Comment Deleted] (YARN-5611) Provide an API to update lifetime of an application.

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5611: -- Comment: was deleted (was: sounds good to me. we can make sure code reused at server side.) > Provide an API to

[jira] [Commented] (YARN-5611) Provide an API to update lifetime of an application.

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502878#comment-15502878 ] Jian He commented on YARN-5611: --- sounds good to me. we can make sure code reused at server side. > Provide

[jira] [Commented] (YARN-3141) Improve locks in SchedulerApplicationAttempt/FSAppAttempt/FiCaSchedulerApp

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502803#comment-15502803 ] Jian He commented on YARN-3141: --- Thanks [~templedf] for reviewing the patch ! > Improve locks in

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502785#comment-15502785 ] Jian He commented on YARN-5609: --- Yes, it'll be sever-side changes and a bit of client. > Expose upgrade and

[jira] [Comment Edited] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502625#comment-15502625 ] Jian He edited comment on YARN-5609 at 9/19/16 7:51 AM: Also, I plan to use

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502625#comment-15502625 ] Jian He commented on YARN-5609: --- Also, I plan to use HADOOP-11552 for the relocalize API so that AM does not

[jira] [Commented] (YARN-3140) Improve locks in AbstractCSQueue/LeafQueue/ParentQueue

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502599#comment-15502599 ] Jian He commented on YARN-3140: --- The conflict is due to log statement reformatted into a bad format, we

[jira] [Comment Edited] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-18 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502377#comment-15502377 ] Jian He edited comment on YARN-5621 at 9/19/16 5:49 AM: Hi [~chris.douglas], I had

[jira] [Comment Edited] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-18 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502377#comment-15502377 ] Jian He edited comment on YARN-5621 at 9/19/16 5:47 AM: Hi [~chris.douglas], I had

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-18 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502377#comment-15502377 ] Jian He commented on YARN-5621: --- Hi [~chris.douglas], I had tried to implement with your approach. However,

[jira] [Commented] (YARN-3140) Improve locks in AbstractCSQueue/LeafQueue/ParentQueue

2016-09-18 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502282#comment-15502282 ] Jian He commented on YARN-3140: --- [~leftnoteasy], the patch doesn't apply any more, could you rebase? >

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-18 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502259#comment-15502259 ] Jian He commented on YARN-5609: --- one question, what is the difference between restartContainer and

[jira] [Commented] (YARN-5637) Changes in NodeManager to support Container rollback and commit

2016-09-17 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15498281#comment-15498281 ] Jian He commented on YARN-5637: --- lgtm, only comment is This {code} if (oldLaunchContext != null) {

[jira] [Commented] (YARN-3140) Improve locks in AbstractCSQueue/LeafQueue/ParentQueue

2016-09-15 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15495424#comment-15495424 ] Jian He commented on YARN-3140: --- lgtm, committing in a day if no comments from others. > Improve locks in

[jira] [Commented] (YARN-3140) Improve locks in AbstractCSQueue/LeafQueue/ParentQueue

2016-09-15 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15493012#comment-15493012 ] Jian He commented on YARN-3140: --- - Is this method not used ? If so, labelManager no need to be volatile, and

[jira] [Commented] (YARN-5611) Provide an API to update lifetime of an application.

2016-09-14 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15489888#comment-15489888 ] Jian He commented on YARN-5611: --- Things like priority, timeout, node-label are attributes of application,

[jira] [Commented] (YARN-4205) Add a service for monitoring application life time out

2016-09-14 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15489769#comment-15489769 ] Jian He commented on YARN-4205: --- latest patch looks good to me. > Add a service for monitoring application

[jira] [Commented] (YARN-5637) Changes in NodeManager to support Container upgrade and rollback/commit

2016-09-14 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15489728#comment-15489728 ] Jian He commented on YARN-5637: --- Thanks Arun, some more comments: - Here, we could make

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support re-initialization of Containers with new launchContext

2016-09-14 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15489593#comment-15489593 ] Jian He commented on YARN-5620: --- bq. KilledExternallyForReInitTransition we use this function to merge the

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support re-initialization of Containers with new launchContext

2016-09-13 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15489523#comment-15489523 ] Jian He commented on YARN-5620: --- Thanks Arun for updating, The merged variable is not initialized with the

[jira] [Commented] (YARN-5637) Changes in NodeManager to support Container upgrade and rollback/commit

2016-09-13 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15487659#comment-15487659 ] Jian He commented on YARN-5637: --- Thanks Arun, some comments: - Do we need to do something for this condition

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support re-initialization of Containers with new launchContext

2016-09-13 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15486721#comment-15486721 ] Jian He commented on YARN-5620: --- [~asuresh], found a couple more issues while looking at the other patch: -

[jira] [Commented] (YARN-5602) Utils for Federation State and Policy Store

2016-09-12 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15486347#comment-15486347 ] Jian He commented on YARN-5602: --- few minor comments: - why this ? {{// Capability and HeartBeat are not

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-12 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484308#comment-15484308 ] Jian He commented on YARN-5620: --- Arun, thank you very much for the prompt response.. I think you forgot to

[jira] [Comment Edited] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-12 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483565#comment-15483565 ] Jian He edited comment on YARN-5620 at 9/12/16 9:18 AM: bq. It is also possible

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-12 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483565#comment-15483565 ] Jian He commented on YARN-5620: --- bq. It is also possible that the an admin logs into the NM and does a 'kill

[jira] [Comment Edited] (YARN-5610) Initial code for native services REST API

2016-09-12 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483469#comment-15483469 ] Jian He edited comment on YARN-5610 at 9/12/16 8:26 AM: Thanks, Gour. bq. So

[jira] [Commented] (YARN-5610) Initial code for native services REST API

2016-09-12 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483469#comment-15483469 ] Jian He commented on YARN-5610: --- Thanks, Gour. bq. So according to an app-owner, it is STARTED but not

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-12 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483313#comment-15483313 ] Jian He commented on YARN-5620: --- One more thing about the test.. In testContainerUpgradeSuccess, could you

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-12 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483274#comment-15483274 ] Jian He commented on YARN-5620: --- bq. the container should be killable explicitly via an external signal.

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-11 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483135#comment-15483135 ] Jian He commented on YARN-5620: --- Arun, thanks for updating ! looks good to me overall, few more comments: -

[jira] [Updated] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-11 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5621: -- Attachment: YARN-5621.5.patch fixed some warnings > Support LinuxContainerExecutor to create symlinks for

[jira] [Updated] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-10 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5621: -- Attachment: YARN-5621.4.patch Thanks for the comments, bq. I'm asking why this is an absolute path ah, IIRC,

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-09 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477474#comment-15477474 ] Jian He commented on YARN-5620: --- Thanks Arun, some more comments and questions: - The reInitContext is

[jira] [Comment Edited] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475973#comment-15475973 ] Jian He edited comment on YARN-5621 at 9/9/16 5:32 AM: --- bq. the container launch

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475973#comment-15475973 ] Jian He commented on YARN-5621: --- bq. the container launch script is pretty self contained, is mostly

[jira] [Comment Edited] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475838#comment-15475838 ] Jian He edited comment on YARN-5621 at 9/9/16 4:14 AM: --- Hi Chris, thanks for chime

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475838#comment-15475838 ] Jian He commented on YARN-5621: --- Hi Chris, thanks for chime in. bq. Is the patch intended for another JIRA

[jira] [Updated] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5621: -- Attachment: YARN-5621.3.patch > Support LinuxContainerExecutor to create symlinks >

[jira] [Updated] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5621: -- Attachment: (was: YARN-5621.3.patch) > Support LinuxContainerExecutor to create symlinks >

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15474018#comment-15474018 ] Jian He commented on YARN-5621: --- Thanks Varun for the review, addressed all comments. bq. Can you explain

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15473997#comment-15473997 ] Jian He commented on YARN-5620: --- Thanks Arun, few comments on the upgrade flow: - The addResource better be

[jira] [Updated] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5621: -- Attachment: YARN-5621.3.patch > Support LinuxContainerExecutor to create symlinks >

[jira] [Comment Edited] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-07 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472777#comment-15472777 ] Jian He edited comment on YARN-5620 at 9/8/16 5:06 AM: --- [~asuresh], thanks for the

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-07 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472777#comment-15472777 ] Jian He commented on YARN-5620: --- [~asuresh], thanks for the explanation bq. Only the AM knows if the upgrade

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-07 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15471166#comment-15471166 ] Jian He commented on YARN-5620: --- Thanks Arun, some questions on the API: - The COMMIT_UPGRADE API: I don’t

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-07 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470494#comment-15470494 ] Jian He commented on YARN-5621: --- Because it is a actually a script file and can also be used for other

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-07 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15469843#comment-15469843 ] Jian He commented on YARN-5621: --- The warnings are not related. > Support LinuxContainerExecutor to create

[jira] [Updated] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-07 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5621: -- Attachment: YARN-5621.2.patch > Support LinuxContainerExecutor to create symlinks >

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-07 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15469687#comment-15469687 ] Jian He commented on YARN-5621: --- Thanks for the comments, That was just to follow what it does in

[jira] [Updated] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-06 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5621: -- Description: When new resources are localized, new symlink needs to be created for the localized resource. This

[jira] [Updated] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-06 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5621: -- Attachment: YARN-5621.1.patch upload the patch. > Support LinuxContainerExecutor to create symlinks >

[jira] [Created] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-06 Thread Jian He (JIRA)
Jian He created YARN-5621: - Summary: Support LinuxContainerExecutor to create symlinks Key: YARN-5621 URL: https://issues.apache.org/jira/browse/YARN-5621 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Commented] (YARN-4205) Add a service for monitoring application life time out

2016-09-05 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466519#comment-15466519 ] Jian He commented on YARN-4205: --- Thanks Rohith, looks good overall, few comments: - can you clarify the

[jira] [Commented] (YARN-5576) Allow resource localization while container is running

2016-09-05 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464221#comment-15464221 ] Jian He commented on YARN-5576: --- The failed tests are passing locally for me. > Allow resource localization

[jira] [Updated] (YARN-5576) Allow resource localization while container is running

2016-09-03 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5576: -- Attachment: YARN-5576.4.branch-2.patch > Allow resource localization while container is running >

[jira] [Commented] (YARN-5615) onContainerStarted/Stopped callback handler in NMClientAsync does not necessarily mean container started/stopped

2016-09-02 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15457917#comment-15457917 ] Jian He commented on YARN-5615: --- HADOOP-11552 is one solution to this. NMClient waits until the container is

[jira] [Updated] (YARN-5615) onContainerStarted/Stopped callback handler in NMClientAsync does not necessarily mean container started/stopped

2016-09-02 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5615: -- Summary: onContainerStarted/Stopped callback handler in NMClientAsync does not necessarily mean container

[jira] [Created] (YARN-5615) NMClienAsynct#CallbackHandler#onContainerStarted/Stopped does not necessarily mean container started/stopped

2016-09-02 Thread Jian He (JIRA)
Jian He created YARN-5615: - Summary: NMClienAsynct#CallbackHandler#onContainerStarted/Stopped does not necessarily mean container started/stopped Key: YARN-5615 URL: https://issues.apache.org/jira/browse/YARN-5615

[jira] [Commented] (YARN-5614) NMClient APIs do not need to take NodeId

2016-09-02 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15457793#comment-15457793 ] Jian He commented on YARN-5614: --- Actually, this may not be that straightfoward.. if the cache in NMClient is

[jira] [Updated] (YARN-5614) NMClient APIs do not need to take NodeId

2016-09-02 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5614: -- Description: Currently, all the NMClient APIs, e.g. startContainer, stopContainer, getContainerStatus force

[jira] [Created] (YARN-5614) NMClient APIs do not need to take NodeId

2016-09-02 Thread Jian He (JIRA)
Jian He created YARN-5614: - Summary: NMClient APIs do not need to take NodeId Key: YARN-5614 URL: https://issues.apache.org/jira/browse/YARN-5614 Project: Hadoop YARN Issue Type: Bug

[jira] [Comment Edited] (YARN-5576) Core change to localize resource while container is running

2016-09-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15457446#comment-15457446 ] Jian He edited comment on YARN-5576 at 9/2/16 4:08 AM: --- Thanks for the review, Arun !

[jira] [Updated] (YARN-5576) Core change to localize resource while container is running

2016-09-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5576: -- Attachment: YARN-5576.4.patch > Core change to localize resource while container is running >

[jira] [Comment Edited] (YARN-5576) Core change to localize resource while container is running

2016-09-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15457446#comment-15457446 ] Jian He edited comment on YARN-5576 at 9/2/16 4:06 AM: --- bq. We need to override the

[jira] [Commented] (YARN-5576) Core change to localize resource while container is running

2016-09-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15457446#comment-15457446 ] Jian He commented on YARN-5576: --- bq. We need to override the ContainerManagerImpl::localize() method in the

[jira] [Commented] (YARN-5610) Initial code for native services REST API

2016-09-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456061#comment-15456061 ] Jian He commented on YARN-5610: --- Some more comments: - Application#get/setNumberOfContainers, this API is a

[jira] [Commented] (YARN-5610) Initial code for native services REST API

2016-09-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456016#comment-15456016 ] Jian He commented on YARN-5610: --- Copy over comments from parent jira. *API Models* - {artifact, resource,

[jira] [Commented] (YARN-5576) Core change to localize resource while container is running

2016-09-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455286#comment-15455286 ] Jian He commented on YARN-5576: --- Latest patch fixed the styles that I think make sense. > Core change to

<    7   8   9   10   11   12   13   14   15   16   >