[jira] [Created] (YARN-5909) Upgrade framework jetty version

2016-11-17 Thread Jian He (JIRA)
Jian He created YARN-5909: - Summary: Upgrade framework jetty version Key: YARN-5909 URL: https://issues.apache.org/jira/browse/YARN-5909 Project: Hadoop YARN Issue Type: Sub-task Reporte

[jira] [Commented] (YARN-4206) Add life time value in Application report and CLI

2016-11-17 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15675011#comment-15675011 ] Jian He commented on YARN-4206: --- looks good overall ,thanks Rohith, few comments - what is th

[jira] [Comment Edited] (YARN-4206) Add life time value in Application report and CLI

2016-11-17 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15675011#comment-15675011 ] Jian He edited comment on YARN-4206 at 11/17/16 10:35 PM: -- looks g

[jira] [Commented] (YARN-5870) Expose getApplications API in YarnClient with GetApplicationsRequest parameter

2016-11-16 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671379#comment-15671379 ] Jian He commented on YARN-5870: --- The javadoc complains exception no description, but all othe

[jira] [Updated] (YARN-5870) Expose getApplications API in YarnClient with GetApplicationsRequest parameter

2016-11-16 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5870: -- Attachment: YARN-5870.2.patch > Expose getApplications API in YarnClient with GetApplicationsRequest parameter >

[jira] [Updated] (YARN-5870) Expose getApplications API in YarnClient with GetApplicationsRequest parameter

2016-11-15 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5870: -- Attachment: YARN-5870.1.patch > Expose getApplications API in YarnClient with GetApplicationsRequest parameter >

[jira] [Commented] (YARN-5870) Expose getApplications API in YarnClient with GetApplicationsRequest parameter

2016-11-15 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668020#comment-15668020 ] Jian He commented on YARN-5870: --- Good catch, Rohith ! I updated the patch to fix that. > E

[jira] [Assigned] (YARN-5769) Integrate update app lifetime using feature implemented in YARN-5611

2016-11-15 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He reassigned YARN-5769: - Assignee: Jian He > Integrate update app lifetime using feature implemented in YARN-5611 > ---

[jira] [Commented] (YARN-4126) RM should not issue delegation tokens in unsecure mode

2016-11-14 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665729#comment-15665729 ] Jian He commented on YARN-4126: --- [~daryn], what's your suggestion to my last question ? Reve

[jira] [Updated] (YARN-5870) Expose getApplications API in YarnClient with GetApplicationsRequest parameter

2016-11-14 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5870: -- Attachment: YARN-5870.patch > Expose getApplications API in YarnClient with GetApplicationsRequest parameter > --

[jira] [Assigned] (YARN-5870) Expose getApplications API in YarnClient with GetApplicationsRequest parameter

2016-11-14 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He reassigned YARN-5870: - Assignee: Jian He > Expose getApplications API in YarnClient with GetApplicationsRequest parameter > -

[jira] [Commented] (YARN-5762) Summarize ApplicationNotFoundException in the RM log

2016-11-11 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15658453#comment-15658453 ] Jian He commented on YARN-5762: --- It does have an API which is getApplications, does that work

[jira] [Commented] (YARN-5825) ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of synchronized block

2016-11-11 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15658406#comment-15658406 ] Jian He commented on YARN-5825: --- lgtm , committing > ProportionalPreemptionalPolicy could us

[jira] [Commented] (YARN-5865) Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611

2016-11-11 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15658396#comment-15658396 ] Jian He commented on YARN-5865: --- thanks Sunil, one comment: we should make sure update in sto

[jira] [Commented] (YARN-5825) ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of synchronized block

2016-11-10 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15655310#comment-15655310 ] Jian He commented on YARN-5825: --- looks good overall, looks like this method added is not use

[jira] [Commented] (YARN-4206) Add life time value in Application report and web UI

2016-11-10 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15655155#comment-15655155 ] Jian He commented on YARN-4206: --- The remaining time can be inferred from the absolute timeout

[jira] [Comment Edited] (YARN-5694) ZKRMStateStore should always start its verification thread to prevent accidental state store corruption

2016-11-10 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15654697#comment-15654697 ] Jian He edited comment on YARN-5694 at 11/10/16 6:09 PM: - bq. If we

[jira] [Commented] (YARN-5694) ZKRMStateStore should always start its verification thread to prevent accidental state store corruption

2016-11-10 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15654697#comment-15654697 ] Jian He commented on YARN-5694: --- bq. If we agree that it's bad to have two RMs accidentally s

[jira] [Commented] (YARN-3955) Support for priority ACLs in CapacityScheduler

2016-11-09 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15652791#comment-15652791 ] Jian He commented on YARN-3955: --- thanks Sunil, some comments for the patch: - PriorityACLCon

[jira] [Commented] (YARN-5611) Provide an API to update lifetime of an application.

2016-11-09 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15652550#comment-15652550 ] Jian He commented on YARN-5611: --- Committed to trunk, branch-2, thanks Rohith ! Thanks Sunil f

[jira] [Commented] (YARN-4597) Add SCHEDULE to NM container lifecycle

2016-11-09 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15652237#comment-15652237 ] Jian He commented on YARN-4597: --- the patch looks good to me, is [~kkaranasos] going to take a

[jira] [Commented] (YARN-4597) Add SCHEDULE to NM container lifecycle

2016-11-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649229#comment-15649229 ] Jian He commented on YARN-4597: --- I see, sounds reasonable. bq. I feel a better name for 'sho

[jira] [Commented] (YARN-4597) Add SCHEDULE to NM container lifecycle

2016-11-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649082#comment-15649082 ] Jian He commented on YARN-4597: --- I see.. thanks for looking into it.. I guess the killedBefor

[jira] [Commented] (YARN-4597) Add SCHEDULE to NM container lifecycle

2016-11-08 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15648821#comment-15648821 ] Jian He commented on YARN-4597: --- bq. I tried to reuse shouldLaunchContainer for the killedBef

[jira] [Commented] (YARN-5694) ZKRMStateStore should always start its verification thread to prevent accidental state store corruption

2016-11-07 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646047#comment-15646047 ] Jian He commented on YARN-5694: --- One other thing is the normal state-store operation already

[jira] [Created] (YARN-5852) Consolidate CSAssignment, ContainerAllocation, ContainerAllocationContext class in CapacityScheduler

2016-11-07 Thread Jian He (JIRA)
Jian He created YARN-5852: - Summary: Consolidate CSAssignment, ContainerAllocation, ContainerAllocationContext class in CapacityScheduler Key: YARN-5852 URL: https://issues.apache.org/jira/browse/YARN-5852 Pr

[jira] [Commented] (YARN-4597) Add SCHEDULE to NM container lifecycle

2016-11-07 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15645979#comment-15645979 ] Jian He commented on YARN-4597: --- bq. scheduledToRunContainers sounds good bq. Using shouldL

[jira] [Commented] (YARN-5716) Add global scheduler interface definition and update CapacityScheduler to use it.

2016-11-05 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15640617#comment-15640617 ] Jian He commented on YARN-5716: --- we'll commit this soon, if no further comments > Add global

[jira] [Commented] (YARN-5611) Provide an API to update lifetime of an application.

2016-11-04 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15638126#comment-15638126 ] Jian He commented on YARN-5611: --- Currently, the patch is taking a lock on the applicationId w

[jira] [Commented] (YARN-5611) Provide an API to update lifetime of an application.

2016-11-02 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630958#comment-15630958 ] Jian He commented on YARN-5611: --- - can you add comments about the type of the timeout value ?

[jira] [Comment Edited] (YARN-5694) ZKRMStateStore should always start its verification thread to prevent accidental state store corruption

2016-11-02 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629737#comment-15629737 ] Jian He edited comment on YARN-5694 at 11/2/16 5:32 PM: sorry, I do

[jira] [Commented] (YARN-5694) ZKRMStateStore should always start its verification thread to prevent accidental state store corruption

2016-11-02 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629737#comment-15629737 ] Jian He commented on YARN-5694: --- sorry, I don't quite get it. why is the active thread needed

[jira] [Updated] (YARN-5813) Slider should not try to set a negative lifetime timeout value

2016-11-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5813: -- Attachment: YARN-5813-yarn-native-services.001.patch > Slider should not try to set a negative lifetime timeout v

[jira] [Updated] (YARN-5813) Slider should not try to set a negative lifetime timeout value

2016-11-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5813: -- Attachment: YARN-5813.1.patch upload a patch > Slider should not try to set a negative lifetime timeout value >

[jira] [Updated] (YARN-5813) Slider should not try to set a negative lifetime timeout value

2016-11-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5813: -- Attachment: (was: YARN-5813.1.patch) > Slider should not try to set a negative lifetime timeout value > -

[jira] [Assigned] (YARN-5813) Slider should not try to set a negative lifetime timeout value

2016-11-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He reassigned YARN-5813: - Assignee: Jian He > Slider should not try to set a negative lifetime timeout value > -

[jira] [Commented] (YARN-2009) CapacityScheduler: Add intra-queue preemption for app priority support

2016-11-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15626578#comment-15626578 ] Jian He commented on YARN-2009: --- One question, not sure if this is expected. This code synch

[jira] [Commented] (YARN-5762) Summarize ApplicationNotFoundException in the RM log

2016-11-01 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15626511#comment-15626511 ] Jian He commented on YARN-5762: --- [~raviprak], instead of ignoring the exception at server, I

[jira] [Commented] (YARN-4597) Add SCHEDULE to NM container lifecycle

2016-10-31 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15623825#comment-15623825 ] Jian He commented on YARN-4597: --- Few more comments - maybe rename ContainerScheduler#running

[jira] [Commented] (YARN-5809) AsyncDispatcher possibly invokes multiple shutdown thread when handling exception

2016-10-31 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15623807#comment-15623807 ] Jian He commented on YARN-5809: --- [~varun_saxena], thanks for the review and commit ! > Async

[jira] [Commented] (YARN-5611) Provide an API to update lifetime of an application.

2016-10-31 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15623498#comment-15623498 ] Jian He commented on YARN-5611: --- - What is the reason to involve RMAppImpl state-machine to u

[jira] [Updated] (YARN-5809) AsyncDispatcher possibly invokes multiple shutdown thread when handling exception

2016-10-31 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5809: -- Attachment: YARN-5809.1.patch > AsyncDispatcher possibly invokes multiple shutdown thread when handling > except

[jira] [Created] (YARN-5809) AsyncDispatcher possibly invokes multiple shutdown thread when handling exception

2016-10-31 Thread Jian He (JIRA)
Jian He created YARN-5809: - Summary: AsyncDispatcher possibly invokes multiple shutdown thread when handling exception Key: YARN-5809 URL: https://issues.apache.org/jira/browse/YARN-5809 Project: Hadoop YARN

[jira] [Commented] (YARN-4126) RM should not issue delegation tokens in unsecure mode

2016-10-25 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15606964#comment-15606964 ] Jian He commented on YARN-4126: --- bq. what was the motivation for removing the delegation tok

[jira] [Commented] (YARN-4126) RM should not issue delegation tokens in unsecure mode

2016-10-23 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15600515#comment-15600515 ] Jian He commented on YARN-4126: --- it is now reverted from 2.8 and branch-2 > RM should not is

[jira] [Commented] (YARN-5716) Add global scheduler interface definition and update CapacityScheduler to use it.

2016-10-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596971#comment-15596971 ] Jian He commented on YARN-5716: --- latest patch looks good to me > Add global scheduler interf

[jira] [Commented] (YARN-4597) Add SCHEDULE to NM container lifecycle

2016-10-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596564#comment-15596564 ] Jian He commented on YARN-4597: --- bq. Will rerun and see if they are required… but in anycase,

[jira] [Commented] (YARN-4597) Add SCHEDULE to NM container lifecycle

2016-10-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596462#comment-15596462 ] Jian He commented on YARN-4597: --- bq. the current default one only takes into account actual

[jira] [Commented] (YARN-5711) AM cannot reconnect to RM after failover when using RequestHedgingRMFailoverProxyProvider

2016-10-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596253#comment-15596253 ] Jian He commented on YARN-5711: --- looks good, + 1 > AM cannot reconnect to RM after failover

[jira] [Commented] (YARN-5525) Make log aggregation service class configurable

2016-10-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596232#comment-15596232 ] Jian He commented on YARN-5525: --- bq. How about I add the AppLogAggregator in Yarn conf as the

[jira] [Commented] (YARN-4126) RM should not issue delegation tokens in unsecure mode

2016-10-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596178#comment-15596178 ] Jian He commented on YARN-4126: --- ok,I'll revert it from 2.8, branch-2, and leave it in trunk

[jira] [Commented] (YARN-5658) YARN should have a hook to delete a path from HDFS when an application ends

2016-10-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596062#comment-15596062 ] Jian He commented on YARN-5658: --- Agree with Daniel here, this is a much smaller change and so

[jira] [Commented] (YARN-4126) RM should not issue delegation tokens in unsecure mode

2016-10-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596031#comment-15596031 ] Jian He commented on YARN-4126: --- The delegationToken retrieved from RM in non-secure cluster

[jira] [Commented] (YARN-5750) YARN-4126 broke Oozie on unsecure cluster

2016-10-21 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15595993#comment-15595993 ] Jian He commented on YARN-5750: --- By impersonation, do you mean the hadoop proxy user function

[jira] [Commented] (YARN-4126) RM should not issue delegation tokens in unsecure mode

2016-10-20 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15593445#comment-15593445 ] Jian He commented on YARN-4126: --- [~jlowe], I think this jira's intention is not wrong, deleti

[jira] [Commented] (YARN-5750) YARN-4126 broke Oozie on unsecure cluster

2016-10-20 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15593427#comment-15593427 ] Jian He commented on YARN-5750: --- [~gezapeti], what is the DummyRenewer used for ? I wonder wh

[jira] [Commented] (YARN-4597) Add SCHEDULE to NM container lifecycle

2016-10-20 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15593320#comment-15593320 ] Jian He commented on YARN-4597: --- [~asuresh], some more questions and comments on the patch:

[jira] [Commented] (YARN-5525) Make log aggregation service class configurable

2016-10-20 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15592878#comment-15592878 ] Jian He commented on YARN-5525: --- bq. push it down completely to AppLogAggregatorImpl I vote f

[jira] [Commented] (YARN-5716) Add global scheduler interface definition and update CapacityScheduler to use it.

2016-10-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15590262#comment-15590262 ] Jian He commented on YARN-5716: --- - remove this? {code} /* FIXME appli

[jira] [Commented] (YARN-5611) Provide an API to update lifetime of an application.

2016-10-19 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15589343#comment-15589343 ] Jian He commented on YARN-5611: --- As you noted, making it polling is introducing all these com

[jira] [Commented] (YARN-5690) Integrate native services modules into maven build

2016-10-18 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15586867#comment-15586867 ] Jian He commented on YARN-5690: --- Hi [~billie.rinaldi], I tested "yarn slider" command locally

[jira] [Commented] (YARN-5611) Provide an API to update lifetime of an application.

2016-10-18 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15586506#comment-15586506 ] Jian He commented on YARN-5611: --- Thanks Rohith, - AbstractLivenessMonitor#getMonitorStartTime

[jira] [Issue Comment Deleted] (YARN-5716) Add global scheduler interface definition and update CapacityScheduler to use it.

2016-10-18 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5716: -- Comment: was deleted (was: - Fix unnecessary format changes in CapacityScheduler - Below null check is unnecessa

[jira] [Commented] (YARN-5716) Add global scheduler interface definition and update CapacityScheduler to use it.

2016-10-18 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15586194#comment-15586194 ] Jian He commented on YARN-5716: --- - Fix unnecessary format changes in CapacityScheduler - Belo

[jira] [Commented] (YARN-5716) Add global scheduler interface definition and update CapacityScheduler to use it.

2016-10-18 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15586192#comment-15586192 ] Jian He commented on YARN-5716: --- - Fix unnecessary format changes in CapacityScheduler - Belo

[jira] [Commented] (YARN-5611) Provide an API to update lifetime of an application.

2016-10-16 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15580786#comment-15580786 ] Jian He commented on YARN-5611: --- - The first if condition already checks whether the app is e

[jira] [Commented] (YARN-5716) Add global scheduler interface definition and update CapacityScheduler to use it.

2016-10-14 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15576530#comment-15576530 ] Jian He commented on YARN-5716: --- - why volatile {code} private volatile BlockingQueue>

[jira] [Commented] (YARN-5735) Make the service REST API use the app timeout feature YARN-4205

2016-10-13 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574295#comment-15574295 ] Jian He commented on YARN-5735: --- updated the patch to fix the issue > Make the service REST

[jira] [Updated] (YARN-5735) Make the service REST API use the app timeout feature YARN-4205

2016-10-13 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5735: -- Attachment: YARN-5735-yarn-native-services.002.patch > Make the service REST API use the app timeout feature YARN

[jira] [Commented] (YARN-5735) Make the service REST API use the app timeout feature YARN-4205

2016-10-13 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573887#comment-15573887 ] Jian He commented on YARN-5735: --- oh, thanks for checking.. I thought it's milliseconds.. will

[jira] [Updated] (YARN-5735) Make the service REST API use the app timeout feature YARN-4205

2016-10-13 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5735: -- Attachment: YARN-5735-yarn-native-services.001.patch yep, updated, forgot about that. thank you. > Make the ser

[jira] [Updated] (YARN-5735) Make the service REST API use the app timeout feature YARN-4205

2016-10-13 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5735: -- Attachment: YARN-5735.1.patch > Make the service REST API use the app timeout feature YARN-4205 > ---

[jira] [Updated] (YARN-5735) Make the service REST API use the app timeout feature YARN-4205

2016-10-13 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5735: -- Summary: Make the service REST API use the app timeout feature YARN-4205 (was: Make the service REST API use the

[jira] [Created] (YARN-5735) Make the service REST API use the timeout feature YARN-4205

2016-10-13 Thread Jian He (JIRA)
Jian He created YARN-5735: - Summary: Make the service REST API use the timeout feature YARN-4205 Key: YARN-5735 URL: https://issues.apache.org/jira/browse/YARN-5735 Project: Hadoop YARN Issue Type:

[jira] [Commented] (YARN-5729) Bug fixes identified during testing

2016-10-13 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573032#comment-15573032 ] Jian He commented on YARN-5729: --- - why is this needed ? {code} this.launchTime = (Date) lau

[jira] [Commented] (YARN-4597) Add SCHEDULE to NM container lifecycle

2016-10-12 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569147#comment-15569147 ] Jian He commented on YARN-4597: --- bq. This will preserve the serial nature of operation (and

[jira] [Commented] (YARN-4597) Add SCHEDULE to NM container lifecycle

2016-10-11 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15567053#comment-15567053 ] Jian He commented on YARN-4597: --- Glanced through the patch, few comments: - Wondering why Kil

[jira] [Commented] (YARN-5690) Integrate native services modules into maven build

2016-10-10 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15564074#comment-15564074 ] Jian He commented on YARN-5690: --- - The yarn script file has a very long line that needs to be

[jira] [Commented] (YARN-5610) Initial code for native services REST API

2016-10-10 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15563033#comment-15563033 ] Jian He commented on YARN-5610: --- bq. SliderClient reads queue name from YARN_LABEL_EXPRESSION

[jira] [Commented] (YARN-5610) Initial code for native services REST API

2016-10-03 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15544389#comment-15544389 ] Jian He commented on YARN-5610: --- - The component name is used in the compNameArtifactIdMap bu

[jira] [Commented] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler

2016-10-03 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543868#comment-15543868 ] Jian He commented on YARN-3139: --- latest patch looks good > Improve locks in AbstractYarnSche

[jira] [Comment Edited] (YARN-5610) Initial code for native services REST API

2016-09-30 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535302#comment-15535302 ] Jian He edited comment on YARN-5610 at 9/30/16 7:30 AM: bq. No, I m

[jira] [Commented] (YARN-5610) Initial code for native services REST API

2016-09-30 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535302#comment-15535302 ] Jian He commented on YARN-5610: --- bq. No, I meant that our REST API yaml document specifies th

[jira] [Commented] (YARN-4205) Add a service for monitoring application life time out

2016-09-29 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535013#comment-15535013 ] Jian He commented on YARN-4205: --- [~cnauroth], [~gtCarrera9], thanks for helping on this ! my

[jira] [Commented] (YARN-5677) RM can be in active-active state for an extended period

2016-09-29 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15532872#comment-15532872 ] Jian He commented on YARN-5677: --- bq. Just tested the leader election (with the right property

[jira] [Comment Edited] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler

2016-09-28 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529409#comment-15529409 ] Jian He edited comment on YARN-3139 at 9/28/16 12:10 PM: - Patch loo

[jira] [Commented] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler

2016-09-28 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529409#comment-15529409 ] Jian He commented on YARN-3139: --- Patch looks good to me. bq. Instead of taking readLock, I

[jira] [Commented] (YARN-4205) Add a service for monitoring application life time out

2016-09-28 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529380#comment-15529380 ] Jian He commented on YARN-4205: --- latest patch looks good to me > Add a service for monitorin

[jira] [Commented] (YARN-5677) RM can be in active-active state for an extended period

2016-09-27 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528575#comment-15528575 ] Jian He commented on YARN-5677: --- [~templedf], this curator based leader election code is new

[jira] [Commented] (YARN-5610) Initial code for native services REST API

2016-09-27 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528516#comment-15528516 ] Jian He commented on YARN-5610: --- bq. This is a dirty way how lifetime is handled in Slider. I

[jira] [Commented] (YARN-4205) Add a service for monitoring application life time out

2016-09-27 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15525545#comment-15525545 ] Jian He commented on YARN-4205: --- Or, we could also change the ApplicationTimeouts class to a

[jira] [Commented] (YARN-4205) Add a service for monitoring application life time out

2016-09-27 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15525507#comment-15525507 ] Jian He commented on YARN-4205: --- [~gsaha], Current ApplicationTimeout has getLifetime which i

[jira] [Commented] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler

2016-09-27 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15525475#comment-15525475 ] Jian He commented on YARN-3139: --- - I remember previous intention was trying avoid lock. Inst

[jira] [Commented] (YARN-5610) Initial code for native services REST API

2016-09-26 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15525251#comment-15525251 ] Jian He commented on YARN-5610: --- bq. since the create action needs to know the path to upload

[jira] [Commented] (YARN-5677) RM can be in active-active state for an extended period

2016-09-26 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15525172#comment-15525172 ] Jian He commented on YARN-5677: --- bq. when the active RM loses contact with the ZK node Defini

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support re-initialization of Containers with new launchContext

2016-09-26 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15523568#comment-15523568 ] Jian He commented on YARN-5620: --- In that case, IIUC, if a LocalizedResource is referenced by

[jira] [Commented] (YARN-5620) Core changes in NodeManager to support re-initialization of Containers with new launchContext

2016-09-26 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15523515#comment-15523515 ] Jian He commented on YARN-5620: --- [~asuresh], I think we need one more jira to release the old

[jira] [Commented] (YARN-5621) Support LinuxContainerExecutor to create symlinks for continuously localized resources

2016-09-26 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15522339#comment-15522339 ] Jian He commented on YARN-5621: --- I still have hesitation on this approach to spawn an additio

[jira] [Commented] (YARN-5315) Standby RM keep sending start am container request to NM

2016-09-25 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15522182#comment-15522182 ] Jian He commented on YARN-5315: --- [~sandflee], I think your first patch is fine. In the 2nd p

[jira] [Commented] (YARN-5609) Expose upgrade and restart API in ContainerManagementProtocol

2016-09-25 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15520518#comment-15520518 ] Jian He commented on YARN-5609: --- yep, sounds good to me. > Expose upgrade and restart API i

<    6   7   8   9   10   11   12   13   14   15   >