[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-23 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15518350#comment-15518350 ] Jian He commented on YARN-5609: --- I see, I'm ok to not use the same method. The main thing I'm

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-23 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15518166#comment-15518166 ] Jian He commented on YARN-5609: --- I saw that the latest patch removed a couple occurrence of

[jira] [Updated] (YARN-5662) Provide an option to enable ContainerMonitor

2016-09-23 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5662: -- Attachment: YARN-5662.3.patch [~varun_saxena], Thank you a lot for the comments ! Addressed them. > Provide an

[jira] [Commented] (YARN-5611) Provide an API to update lifetime of an application.

2016-09-23 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15516508#comment-15516508 ] Jian He commented on YARN-5611: --- - I thought the lifetime passed in the updateLifetime API is

[jira] [Commented] (YARN-5610) Initial code for native services REST API

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15515597#comment-15515597 ] Jian He commented on YARN-5610: --- bq. The reason these attributes are at the Application level

[jira] [Updated] (YARN-5662) Provide an option to enable ContainerMonitor

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5662: -- Attachment: YARN-5662.2.patch > Provide an option to enable ContainerMonitor > -

[jira] [Comment Edited] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15513480#comment-15513480 ] Jian He edited comment on YARN-5609 at 9/22/16 2:44 PM: looks good

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15513480#comment-15513480 ] Jian He commented on YARN-5609: --- looks good to me overall, I think wherever {{setIsReInitial

[jira] [Commented] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15513402#comment-15513402 ] Jian He commented on YARN-3139: --- There are also similar comments like this... I just couldn't

[jira] [Issue Comment Deleted] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-3139: -- Comment: was deleted (was: May be it's related to some consistency issues with respect to queue stats?) > Impro

[jira] [Comment Edited] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15513402#comment-15513402 ] Jian He edited comment on YARN-3139 at 9/22/16 2:20 PM: There are a

[jira] [Commented] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15513416#comment-15513416 ] Jian He commented on YARN-3139: --- May be it's related to some consistency issues with respect

[jira] [Commented] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15513407#comment-15513407 ] Jian He commented on YARN-3139: --- [~templedf], would you like to check the FairScheduler chang

[jira] [Updated] (YARN-5662) Provide an option to enable ContainerMonitor

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5662: -- Attachment: YARN-5662.1.patch > Provide an option to enable ContainerMonitor > -

[jira] [Created] (YARN-5662) Provide an option to enable ContainerMonitor

2016-09-22 Thread Jian He (JIRA)
Jian He created YARN-5662: - Summary: Provide an option to enable ContainerMonitor Key: YARN-5662 URL: https://issues.apache.org/jira/browse/YARN-5662 Project: Hadoop YARN Issue Type: Bug

[jira] [Updated] (YARN-5662) Provide an option to enable ContainerMonitor

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5662: -- Description: Currently, if vmem/pmem check is not enabled, ContainerMonitor would not run. In certain cases, Con

[jira] [Updated] (YARN-5662) Provide an option to enable ContainerMonitor

2016-09-22 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5662: -- Release Note: (was: Currently, if vmem/pmem check is not enabled, ContainerMonitor would not run. In certain c

[jira] [Issue Comment Deleted] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5609: -- Comment: was deleted (was: IIUC, even if it's a merged operation, the current will be a superset of previous, so

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15512205#comment-15512205 ] Jian He commented on YARN-5609: --- IIUC, even if it's a merged operation, the current will be a

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15512203#comment-15512203 ] Jian He commented on YARN-5609: --- IIUC, even if it's a merged operation, the current will be a

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15512108#comment-15512108 ] Jian He commented on YARN-5609: --- bq. during rollback, we should not merge the current resourc

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510458#comment-15510458 ] Jian He commented on YARN-5609: --- Yep, what I meant is the second last patch is fine, as the i

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510111#comment-15510111 ] Jian He commented on YARN-5609: --- I had re-checked the code... actually, even the old resource

[jira] [Commented] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler

2016-09-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509412#comment-15509412 ] Jian He commented on YARN-3139: --- - Is removing these locks ok? {code} // It is crucial to

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-20 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15508770#comment-15508770 ] Jian He commented on YARN-5609: --- bq. since it is always called in conjunctions with a contai

[jira] [Comment Edited] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-20 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15508741#comment-15508741 ] Jian He edited comment on YARN-5609 at 9/21/16 4:58 AM: bq. I had i

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-20 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15508741#comment-15508741 ] Jian He commented on YARN-5609: --- bq. I had intentionally kept it that way (my thinking was th

[jira] [Comment Edited] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-20 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15508689#comment-15508689 ] Jian He edited comment on YARN-5621 at 9/21/16 4:40 AM: Thanks for

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-20 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15508689#comment-15508689 ] Jian He commented on YARN-5621: --- Thanks for your explanation, some parts I'm missing: bq. B

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-20 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15506031#comment-15506031 ] Jian He commented on YARN-5609: --- - I think this will cause the resources to be re-requested o

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505601#comment-15505601 ] Jian He commented on YARN-5621: --- My original thought is to create the symlinks right after lo

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505585#comment-15505585 ] Jian He commented on YARN-5609: --- Thanks Arun, some more comments: - IIUC, when restarting the

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15503725#comment-15503725 ] Jian He commented on YARN-5609: --- - Can you add comments in commitLastReInitialization that on

[jira] [Commented] (YARN-5611) Provide an API to update lifetime of an application.

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502879#comment-15502879 ] Jian He commented on YARN-5611: --- sounds good to me. we can make sure code reused at server si

[jira] [Issue Comment Deleted] (YARN-5611) Provide an API to update lifetime of an application.

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5611: -- Comment: was deleted (was: sounds good to me. we can make sure code reused at server side.) > Provide an API to

[jira] [Commented] (YARN-5611) Provide an API to update lifetime of an application.

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502878#comment-15502878 ] Jian He commented on YARN-5611: --- sounds good to me. we can make sure code reused at server si

[jira] [Commented] (YARN-3141) Improve locks in SchedulerApplicationAttempt/FSAppAttempt/FiCaSchedulerApp

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502803#comment-15502803 ] Jian He commented on YARN-3141: --- Thanks [~templedf] for reviewing the patch ! > Improve lock

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502785#comment-15502785 ] Jian He commented on YARN-5609: --- Yes, it'll be sever-side changes and a bit of client. > Exp

[jira] [Comment Edited] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502625#comment-15502625 ] Jian He edited comment on YARN-5609 at 9/19/16 7:51 AM: Also, I pla

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502625#comment-15502625 ] Jian He commented on YARN-5609: --- Also, I plan to use HADOOP-11552 for the relocalize API so t

[jira] [Commented] (YARN-3140) Improve locks in AbstractCSQueue/LeafQueue/ParentQueue

2016-09-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502599#comment-15502599 ] Jian He commented on YARN-3140: --- The conflict is due to log statement reformatted into a bad

[jira] [Comment Edited] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-18 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502377#comment-15502377 ] Jian He edited comment on YARN-5621 at 9/19/16 5:49 AM: Hi [~chris.

[jira] [Comment Edited] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-18 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502377#comment-15502377 ] Jian He edited comment on YARN-5621 at 9/19/16 5:47 AM: Hi [~chris.

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-18 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502377#comment-15502377 ] Jian He commented on YARN-5621: --- Hi [~chris.douglas], I had tried to implement with your appr

[jira] [Commented] (YARN-3140) Improve locks in AbstractCSQueue/LeafQueue/ParentQueue

2016-09-18 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502282#comment-15502282 ] Jian He commented on YARN-3140: --- [~leftnoteasy], the patch doesn't apply any more, could you

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-18 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502259#comment-15502259 ] Jian He commented on YARN-5609: --- one question, what is the difference between restartContaine

[jira] [Commented] (YARN-5637) Changes in NodeManager to support Container rollback and commit

2016-09-16 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15498281#comment-15498281 ] Jian He commented on YARN-5637: --- lgtm, only comment is This {code} if (oldLaunchContext

[jira] [Commented] (YARN-3140) Improve locks in AbstractCSQueue/LeafQueue/ParentQueue

2016-09-15 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15495424#comment-15495424 ] Jian He commented on YARN-3140: --- lgtm, committing in a day if no comments from others. > Imp

[jira] [Commented] (YARN-3140) Improve locks in AbstractCSQueue/LeafQueue/ParentQueue

2016-09-15 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15493012#comment-15493012 ] Jian He commented on YARN-3140: --- - Is this method not used ? If so, labelManager no need to b

[jira] [Commented] (YARN-5611) Provide an API to update lifetime of an application.

2016-09-14 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15489888#comment-15489888 ] Jian He commented on YARN-5611: --- Things like priority, timeout, node-label are attributes of

[jira] [Commented] (YARN-4205) Add a service for monitoring application life time out

2016-09-14 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15489769#comment-15489769 ] Jian He commented on YARN-4205: --- latest patch looks good to me. > Add a service for monitori

[jira] [Commented] (YARN-5637) Changes in NodeManager to support Container upgrade and rollback/commit

2016-09-14 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15489728#comment-15489728 ] Jian He commented on YARN-5637: --- Thanks Arun, some more comments: - Here, we could make reIni

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support re-initialization of Containers with new launchContext

2016-09-13 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15489593#comment-15489593 ] Jian He commented on YARN-5620: --- bq. KilledExternallyForReInitTransition we use this function

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support re-initialization of Containers with new launchContext

2016-09-13 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15489523#comment-15489523 ] Jian He commented on YARN-5620: --- Thanks Arun for updating, The merged variable is not initia

[jira] [Commented] (YARN-5637) Changes in NodeManager to support Container upgrade and rollback/commit

2016-09-13 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15487659#comment-15487659 ] Jian He commented on YARN-5637: --- Thanks Arun, some comments: - Do we need to do something for

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support re-initialization of Containers with new launchContext

2016-09-13 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15486721#comment-15486721 ] Jian He commented on YARN-5620: --- [~asuresh], found a couple more issues while looking at the

[jira] [Commented] (YARN-5602) Utils for Federation State and Policy Store

2016-09-12 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15486347#comment-15486347 ] Jian He commented on YARN-5602: --- few minor comments: - why this ? {{// Capability and HeartB

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-12 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15484308#comment-15484308 ] Jian He commented on YARN-5620: --- Arun, thank you very much for the prompt response.. I think

[jira] [Comment Edited] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-12 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15483565#comment-15483565 ] Jian He edited comment on YARN-5620 at 9/12/16 9:18 AM: bq. It is a

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-12 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15483565#comment-15483565 ] Jian He commented on YARN-5620: --- bq. It is also possible that the an admin logs into the NM a

[jira] [Comment Edited] (YARN-5610) Initial code for native services REST API

2016-09-12 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15483469#comment-15483469 ] Jian He edited comment on YARN-5610 at 9/12/16 8:26 AM: Thanks, Gou

[jira] [Commented] (YARN-5610) Initial code for native services REST API

2016-09-12 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15483469#comment-15483469 ] Jian He commented on YARN-5610: --- Thanks, Gour. bq. So according to an app-owner, it is START

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-12 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15483313#comment-15483313 ] Jian He commented on YARN-5620: --- One more thing about the test.. In testContainerUpgradeSucce

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-11 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15483274#comment-15483274 ] Jian He commented on YARN-5620: --- bq. the container should be killable explicitly via an exter

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-11 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15483135#comment-15483135 ] Jian He commented on YARN-5620: --- Arun, thanks for updating ! looks good to me overall, few mo

[jira] [Updated] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-11 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5621: -- Attachment: YARN-5621.5.patch fixed some warnings > Support LinuxContainerExecutor to create symlinks for contin

[jira] [Updated] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-10 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5621: -- Attachment: YARN-5621.4.patch Thanks for the comments, bq. I'm asking why this is an absolute path ah, IIRC, this

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-09 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15477474#comment-15477474 ] Jian He commented on YARN-5620: --- Thanks Arun, some more comments and questions: - The reInit

[jira] [Comment Edited] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475973#comment-15475973 ] Jian He edited comment on YARN-5621 at 9/9/16 5:32 AM: --- bq. the conta

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475973#comment-15475973 ] Jian He commented on YARN-5621: --- bq. the container launch script is pretty self contained, is

[jira] [Comment Edited] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475838#comment-15475838 ] Jian He edited comment on YARN-5621 at 9/9/16 4:14 AM: --- Hi Chris, tha

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475838#comment-15475838 ] Jian He commented on YARN-5621: --- Hi Chris, thanks for chime in. bq. Is the patch intended for

[jira] [Updated] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5621: -- Attachment: YARN-5621.3.patch > Support LinuxContainerExecutor to create symlinks > -

[jira] [Updated] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5621: -- Attachment: (was: YARN-5621.3.patch) > Support LinuxContainerExecutor to create symlinks > --

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474018#comment-15474018 ] Jian He commented on YARN-5621: --- Thanks Varun for the review, addressed all comments. bq. Ca

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15473997#comment-15473997 ] Jian He commented on YARN-5620: --- Thanks Arun, few comments on the upgrade flow: - The addRes

[jira] [Updated] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5621: -- Attachment: YARN-5621.3.patch > Support LinuxContainerExecutor to create symlinks > -

[jira] [Comment Edited] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-07 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472777#comment-15472777 ] Jian He edited comment on YARN-5620 at 9/8/16 5:06 AM: --- [~asuresh], t

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-07 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472777#comment-15472777 ] Jian He commented on YARN-5620: --- [~asuresh], thanks for the explanation bq. Only the AM knows

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support for upgrade and rollback of Containers

2016-09-07 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15471166#comment-15471166 ] Jian He commented on YARN-5620: --- Thanks Arun, some questions on the API: - The COMMIT_UPGRADE

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-07 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15470494#comment-15470494 ] Jian He commented on YARN-5621: --- Because it is a actually a script file and can also be used

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-07 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15469843#comment-15469843 ] Jian He commented on YARN-5621: --- The warnings are not related. > Support LinuxContainerExecu

[jira] [Updated] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-06 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5621: -- Attachment: YARN-5621.2.patch > Support LinuxContainerExecutor to create symlinks > -

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-06 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15469687#comment-15469687 ] Jian He commented on YARN-5621: --- Thanks for the comments, That was just to follow what it do

[jira] [Updated] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-06 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5621: -- Description: When new resources are localized, new symlink needs to be created for the localized resource. This i

[jira] [Updated] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-06 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5621: -- Attachment: YARN-5621.1.patch upload the patch. > Support LinuxContainerExecutor to create symlinks > --

[jira] [Created] (YARN-5621) Support LinuxContainerExecutor to create symlinks

2016-09-06 Thread Jian He (JIRA)
Jian He created YARN-5621: - Summary: Support LinuxContainerExecutor to create symlinks Key: YARN-5621 URL: https://issues.apache.org/jira/browse/YARN-5621 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Commented] (YARN-4205) Add a service for monitoring application life time out

2016-09-05 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15466519#comment-15466519 ] Jian He commented on YARN-4205: --- Thanks Rohith, looks good overall, few comments: - can you

[jira] [Commented] (YARN-5576) Allow resource localization while container is running

2016-09-04 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464221#comment-15464221 ] Jian He commented on YARN-5576: --- The failed tests are passing locally for me. > Allow resour

[jira] [Updated] (YARN-5576) Allow resource localization while container is running

2016-09-03 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5576: -- Attachment: YARN-5576.4.branch-2.patch > Allow resource localization while container is running > ---

[jira] [Commented] (YARN-5615) onContainerStarted/Stopped callback handler in NMClientAsync does not necessarily mean container started/stopped

2016-09-02 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15457917#comment-15457917 ] Jian He commented on YARN-5615: --- HADOOP-11552 is one solution to this. NMClient waits until t

[jira] [Updated] (YARN-5615) onContainerStarted/Stopped callback handler in NMClientAsync does not necessarily mean container started/stopped

2016-09-02 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5615: -- Summary: onContainerStarted/Stopped callback handler in NMClientAsync does not necessarily mean container started

[jira] [Created] (YARN-5615) NMClienAsynct#CallbackHandler#onContainerStarted/Stopped does not necessarily mean container started/stopped

2016-09-02 Thread Jian He (JIRA)
Jian He created YARN-5615: - Summary: NMClienAsynct#CallbackHandler#onContainerStarted/Stopped does not necessarily mean container started/stopped Key: YARN-5615 URL: https://issues.apache.org/jira/browse/YARN-5615

[jira] [Commented] (YARN-5614) NMClient APIs do not need to take NodeId

2016-09-02 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15457793#comment-15457793 ] Jian He commented on YARN-5614: --- Actually, this may not be that straightfoward.. if the cache

[jira] [Updated] (YARN-5614) NMClient APIs do not need to take NodeId

2016-09-02 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5614: -- Description: Currently, all the NMClient APIs, e.g. startContainer, stopContainer, getContainerStatus force call

[jira] [Created] (YARN-5614) NMClient APIs do not need to take NodeId

2016-09-02 Thread Jian He (JIRA)
Jian He created YARN-5614: - Summary: NMClient APIs do not need to take NodeId Key: YARN-5614 URL: https://issues.apache.org/jira/browse/YARN-5614 Project: Hadoop YARN Issue Type: Bug Repo

[jira] [Comment Edited] (YARN-5576) Core change to localize resource while container is running

2016-09-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15457446#comment-15457446 ] Jian He edited comment on YARN-5576 at 9/2/16 4:08 AM: --- Thanks for th

[jira] [Updated] (YARN-5576) Core change to localize resource while container is running

2016-09-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5576: -- Attachment: YARN-5576.4.patch > Core change to localize resource while container is running > ---

[jira] [Comment Edited] (YARN-5576) Core change to localize resource while container is running

2016-09-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15457446#comment-15457446 ] Jian He edited comment on YARN-5576 at 9/2/16 4:06 AM: --- bq. We need t

[jira] [Commented] (YARN-5576) Core change to localize resource while container is running

2016-09-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15457446#comment-15457446 ] Jian He commented on YARN-5576: --- bq. We need to override the ContainerManagerImpl::localize()

<    7   8   9   10   11   12   13   14   15   16   >