[jira] [Updated] (YARN-7036) [ATSv2] : NPE for recovered finished applications in NM during upgrade

2017-08-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7036: - Issue Type: Sub-task (was: Bug) Parent: YARN-5355 > [ATSv2] : NPE for recovered finished

[jira] [Commented] (YARN-6409) RM does not blacklist node for AM launch failures

2017-05-02 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993457#comment-15993457 ] Haibo Chen commented on YARN-6409: -- Yeah, NMs can be unresponsive during the heartbeat window, from what

[jira] [Created] (YARN-6586) YARN to facilitate HTTPS in AM web server

2017-05-11 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-6586: Summary: YARN to facilitate HTTPS in AM web server Key: YARN-6586 URL: https://issues.apache.org/jira/browse/YARN-6586 Project: Hadoop YARN Issue Type: Improvement

[jira] [Commented] (YARN-5841) Report only local collectors on node upon resync with RM after RM fails over

2017-05-11 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006885#comment-16006885 ] Haibo Chen commented on YARN-5841: -- Closing this as not a problem. Please reopen if you disagree > Report

[jira] [Resolved] (YARN-5841) Report only local collectors on node upon resync with RM after RM fails over

2017-05-11 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen resolved YARN-5841. -- Resolution: Not A Problem > Report only local collectors on node upon resync with RM after RM fails over

[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps

2017-05-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005053#comment-16005053 ] Haibo Chen commented on YARN-6457: -- Thanks [~vrozov] for pointing out SSLFactory. I have noticed that

[jira] [Updated] (YARN-6409) RM does not blacklist node for AM launch failures

2017-05-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6409: - Attachment: YARN-6409.02.patch Updated the patch to only blacklist a node upon AM launch failure due to

[jira] [Commented] (YARN-6409) RM does not blacklist node for AM launch failures

2017-05-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005365#comment-16005365 ] Haibo Chen commented on YARN-6409: -- Going three levels down so that the failures indicated in the above

[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps

2017-05-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005377#comment-16005377 ] Haibo Chen commented on YARN-6457: -- Yes. only configuration properties specified in ssl-server.xml and

[jira] [Commented] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002918#comment-16002918 ] Haibo Chen commented on YARN-6566: -- Can you please elaborate on how the web UI layout is damaged (Maybe a

[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003003#comment-16003003 ] Haibo Chen commented on YARN-6457: -- Will do. Did not realize that 2.8.1 has been cut. Thanks for the

[jira] [Commented] (YARN-6566) add a property for a hadoop job to identified the full hive HQL script text in hadoop web view in multi-users environment

2017-05-08 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002006#comment-16002006 ] Haibo Chen commented on YARN-6566: -- Do you think that you can reuse application tags for this purpose? >

[jira] [Commented] (YARN-6559) Findbugs warning in YARN-5355 branch

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003410#comment-16003410 ] Haibo Chen commented on YARN-6559: -- Wait, this looks like YARN-6518? Should we cherry-pick that instead?

[jira] [Commented] (YARN-3872) TimelineReader Web UI Implementation

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003512#comment-16003512 ] Haibo Chen commented on YARN-3872: -- [~varun_impala_149e] Is the reader UI based on YARN web ui 2.0, or

[jira] [Commented] (YARN-6561) Update exception information during timeline collector aux service initialization

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003418#comment-16003418 ] Haibo Chen commented on YARN-6561: -- +1 on the patch. Will wait until the end of today to commit > Update

[jira] [Commented] (YARN-3545) Investigate the concurrency issue with the map of timeline collector

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003479#comment-16003479 ] Haibo Chen commented on YARN-3545: -- Looking at this as it seems related to YARN-6563. [~vrushalic], still

[jira] [Commented] (YARN-6559) Findbugs warning in YARN-5355 branch

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003402#comment-16003402 ] Haibo Chen commented on YARN-6559: -- Thanks for pointing that out. +1 > Findbugs warning in YARN-5355

[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1 in REST output

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003537#comment-16003537 ] Haibo Chen commented on YARN-5094: -- [~gtCarrera9] Do you mind if I take over? I am looking at YARN-5169,

[jira] [Commented] (YARN-6435) [ATSv2] Can't retrieve more than 1000 versions of metrics in time series

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003634#comment-16003634 ] Haibo Chen commented on YARN-6435: -- Thanks for the patch, [~rohithsharma]. The max version is still being

[jira] [Commented] (YARN-3872) TimelineReader Web UI Implementation

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003958#comment-16003958 ] Haibo Chen commented on YARN-3872: -- Thanks [~rohithsharma] for the pointer! > TimelineReader Web UI

[jira] [Commented] (YARN-6435) [ATSv2] Can't retrieve more than 1000 versions of metrics in time series

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003963#comment-16003963 ] Haibo Chen commented on YARN-6435: -- LGTM +1. Will commit it shortly > [ATSv2] Can't retrieve more than

[jira] [Commented] (YARN-3545) Investigate the concurrency issue with the map of timeline collector

2017-05-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005125#comment-16005125 ] Haibo Chen commented on YARN-3545: -- Thanks, Vrushali. Given the access pattern on TimelineCollectorManager

[jira] [Commented] (YARN-1593) support out-of-proc AuxiliaryServices

2017-05-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010805#comment-16010805 ] Haibo Chen commented on YARN-1593: -- It seems system services are now addressed by YARN-6601 separately. Do

[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.

2017-05-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010938#comment-16010938 ] Haibo Chen commented on YARN-6323: -- Thanks for the patch [~vrushalic]! I wonder if we should, to include

[jira] [Commented] (YARN-6601) Allow service to be started as System Services during serviceapi start up

2017-05-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010783#comment-16010783 ] Haibo Chen commented on YARN-6601: -- Thanks. +1 on addressing system container and system services

[jira] [Commented] (YARN-5705) [YARN-3368] Add support for Timeline V2 to new web UI

2017-05-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16015038#comment-16015038 ] Haibo Chen commented on YARN-5705: -- [~akhilpb] The latest patches no longer applies. Any plan to get this

[jira] [Updated] (YARN-6409) RM does not blacklist node for AM launch failures

2017-05-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6409: - Attachment: YARN-6409.03.patch > RM does not blacklist node for AM launch failures >

[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.

2017-05-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011262#comment-16011262 ] Haibo Chen commented on YARN-6323: -- Thanks for pointing YARN-6555! I forgot that. A few comments: bq.

[jira] [Commented] (YARN-6409) RM does not blacklist node for AM launch failures

2017-05-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011322#comment-16011322 ] Haibo Chen commented on YARN-6409: -- Thanks [~rkanter] for your review! I am not familiar with the RPC

[jira] [Created] (YARN-6604) Allow metric TTL for Application table to be specified through cmd

2017-05-15 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-6604: Summary: Allow metric TTL for Application table to be specified through cmd Key: YARN-6604 URL: https://issues.apache.org/jira/browse/YARN-6604 Project: Hadoop YARN

[jira] [Updated] (YARN-6604) Allow metric TTL for Application table to be specified through cmd

2017-05-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6604: - Issue Type: Sub-task (was: Improvement) Parent: YARN-5355 > Allow metric TTL for Application

[jira] [Updated] (YARN-6316) Provide help information and documentation for TimelineSchemaCreator

2017-05-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6316: - Attachment: YARN-6316.00.patch Thanks for the review, [~vrushalic]. I updated the patch accordingly. Also

[jira] [Commented] (YARN-6601) Allow service to be started as System Services during serviceapi start up

2017-05-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010695#comment-16010695 ] Haibo Chen commented on YARN-6601: -- [~rohithsharma], the doc seems to cover part of YARN-1593. Is the

[jira] [Commented] (YARN-6564) RM service is shutting down when ATS v2 is enabled

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004037#comment-16004037 ] Haibo Chen commented on YARN-6564: -- [~rohithsharma] Now that HADOOP-14386 has been resolved, what is the

[jira] [Commented] (YARN-6435) [ATSv2] Can't retrieve more than 1000 versions of metrics in time series

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004061#comment-16004061 ] Haibo Chen commented on YARN-6435: -- Will do > [ATSv2] Can't retrieve more than 1000 versions of metrics

[jira] [Commented] (YARN-6518) Fix warnings from Spotbugs in hadoop-yarn-server-timelineservice

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004075#comment-16004075 ] Haibo Chen commented on YARN-6518: -- Thanks [~Naganarasimha] ! > Fix warnings from Spotbugs in

[jira] [Assigned] (YARN-5841) Report only local collectors on node upon resync with RM after RM fails over

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen reassigned YARN-5841: Assignee: Haibo Chen > Report only local collectors on node upon resync with RM after RM fails over

[jira] [Updated] (YARN-6561) Update exception message during timeline collector aux service initialization

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6561: - Summary: Update exception message during timeline collector aux service initialization (was: Update

[jira] [Updated] (YARN-6435) [ATSv2] Can't retrieve more than 1000 versions of metrics in time series

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6435: - Fix Version/s: YARN-5355-branch-2 > [ATSv2] Can't retrieve more than 1000 versions of metrics in time

[jira] [Commented] (YARN-6559) Findbugs warning in YARN-5355 branch

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004077#comment-16004077 ] Haibo Chen commented on YARN-6559: -- YARN-6518 now has been cherry-picked into YARN-5355 as well. Will

[jira] [Commented] (YARN-6140) start time key in NM leveldb store should be removed when container is removed

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004087#comment-16004087 ] Haibo Chen commented on YARN-6140: -- [~ajithshetty], any update on this? > start time key in NM leveldb

[jira] [Commented] (YARN-6518) Fix warnings from Spotbugs in hadoop-yarn-server-timelineservice

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003822#comment-16003822 ] Haibo Chen commented on YARN-6518: -- [~Naganarasimha] Can you please cherry-pick this into YARN-5355 branch

[jira] [Commented] (YARN-5841) Report only local collectors on node upon resync with RM after RM fails over

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003745#comment-16003745 ] Haibo Chen commented on YARN-5841: -- NMs maintains a set of registered collectors that they send to RM

[jira] [Commented] (YARN-6559) Findbugs warning in YARN-5355 branch

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003823#comment-16003823 ] Haibo Chen commented on YARN-6559: -- I don't see YARN-6518 in YARN-5355 branch either. I have asked Naga in

[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002942#comment-16002942 ] Haibo Chen commented on YARN-6457: -- [~sanjaypujare] I have backported this to 2.7.4+ and 2.8.1+ as well >

[jira] [Updated] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps

2017-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6457: - Fix Version/s: 2.8.1 2.7.4 > Allow custom SSL configuration to be supplied in WebApps >

[jira] [Commented] (YARN-6316) Provide help information and documentation for TimelineSchemaCreator

2017-05-12 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16008394#comment-16008394 ] Haibo Chen commented on YARN-6316: -- [~vrushalic], [~rohithsharma], [~varun_saxena], can you kindly review?

[jira] [Commented] (YARN-6580) Incorrect logger for FairSharePolicy

2017-05-12 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16008636#comment-16008636 ] Haibo Chen commented on YARN-6580: -- +1. Checking this in. > Incorrect logger for FairSharePolicy >

[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps

2017-05-10 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005297#comment-16005297 ] Haibo Chen commented on YARN-6457: -- I think yes. Prior to this patch, all SSL configuration parameters

[jira] [Created] (YARN-6706) Refactor ContainerScheduler to make oversubscription change easier

2017-06-09 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-6706: Summary: Refactor ContainerScheduler to make oversubscription change easier Key: YARN-6706 URL: https://issues.apache.org/jira/browse/YARN-6706 Project: Hadoop YARN

[jira] [Updated] (YARN-6706) Refactor ContainerScheduler to make oversubscription change easier

2017-06-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6706: - Issue Type: Sub-task (was: Task) Parent: YARN-1011 > Refactor ContainerScheduler to make

[jira] [Updated] (YARN-6706) Refactor ContainerScheduler to make oversubscription change easier

2017-06-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6706: - Attachment: YARN-6706-YARN-1011.00.patch > Refactor ContainerScheduler to make oversubscription change

[jira] [Updated] (YARN-6670) Add separate NM overallocation thresholds for cpu and memory

2017-06-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6670: - Attachment: YARN-6670-YARN-1011.03.patch > Add separate NM overallocation thresholds for cpu and memory >

[jira] [Commented] (YARN-6706) Refactor ContainerScheduler to make oversubscription change easier

2017-06-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045233#comment-16045233 ] Haibo Chen commented on YARN-6706: -- [~arun.sur...@gmail.com] [~kkaranasos] please kindly review >

[jira] [Commented] (YARN-6706) Refactor ContainerScheduler to make oversubscription change easier

2017-06-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045229#comment-16045229 ] Haibo Chen commented on YARN-6706: -- This patch does the following: 1) Changes the

[jira] [Commented] (YARN-6670) Add separate NM overallocation thresholds for cpu and memory

2017-06-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045024#comment-16045024 ] Haibo Chen commented on YARN-6670: -- Thanks [~kasha] for your review! I have updated the patch to address

[jira] [Created] (YARN-6705) Add separate NM preemption thresholds for cpu and memory

2017-06-09 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-6705: Summary: Add separate NM preemption thresholds for cpu and memory Key: YARN-6705 URL: https://issues.apache.org/jira/browse/YARN-6705 Project: Hadoop YARN Issue

[jira] [Comment Edited] (YARN-6670) Add separate NM overallocation thresholds for cpu and memory

2017-06-12 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16046882#comment-16046882 ] Haibo Chen edited comment on YARN-6670 at 6/12/17 6:12 PM: --- IIUC, this

[jira] [Commented] (YARN-6670) Add separate NM overallocation thresholds for cpu and memory

2017-06-12 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16046882#comment-16046882 ] Haibo Chen commented on YARN-6670: -- IIUC, this effectively turns the overallocation threshold from being

[jira] [Commented] (YARN-6047) Documentation updates for TimelineService v2

2017-06-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16056261#comment-16056261 ] Haibo Chen commented on YARN-6047: -- I think we should explicitly call out the data loss issue when HBase

[jira] [Commented] (YARN-6728) Job will run slow when the performance of defaultFs degrades and the log-aggregation is enable.

2017-06-23 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061162#comment-16061162 ] Haibo Chen commented on YARN-6728: -- Seems a duplicate of YARN-6396 > Job will run slow when the

[jira] [Commented] (YARN-6733) Add table for storing sub-application entities

2017-06-23 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061175#comment-16061175 ] Haibo Chen commented on YARN-6733: -- I have a few questions on the Tez use case. 1) Is the sub_application

[jira] [Updated] (YARN-1011) [Umbrella] Schedule containers based on utilization of currently allocated containers

2017-06-23 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-1011: - Attachment: yarn-1011-design-v3.pdf Updated design doc based on recent discussion with [~kkaranasos] and

[jira] [Updated] (YARN-6739) Crash NM at start time if oversubscription is on but LinuxContainerExcutor or cgroup is off

2017-06-23 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6739: - Issue Type: Sub-task (was: Bug) Parent: YARN-1011 > Crash NM at start time if oversubscription is

[jira] [Created] (YARN-6739) Crash NM at start time if oversubscription is on but LinuxContainerExcutor or cgroup is off

2017-06-23 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-6739: Summary: Crash NM at start time if oversubscription is on but LinuxContainerExcutor or cgroup is off Key: YARN-6739 URL: https://issues.apache.org/jira/browse/YARN-6739

[jira] [Resolved] (YARN-6671) Add container type awareness in ResourceHandlers.

2017-06-23 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen resolved YARN-6671. -- Resolution: Not A Problem > Add container type awareness in ResourceHandlers. >

[jira] [Commented] (YARN-6671) Add container type awareness in ResourceHandlers.

2017-06-23 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061671#comment-16061671 ] Haibo Chen commented on YARN-6671: -- Container object is passed in ResourceHandlers, and container type can

[jira] [Commented] (YARN-6706) Refactor ContainerScheduler to make oversubscription change easier

2017-06-19 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16054479#comment-16054479 ] Haibo Chen commented on YARN-6706: -- [~leftnoteasy] I have shared the latest design doc with you. >

[jira] [Updated] (YARN-6706) Refactor ContainerScheduler to make oversubscription change easier

2017-06-19 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6706: - Attachment: YARN-6706-YARN-1011.01.patch > Refactor ContainerScheduler to make oversubscription change

[jira] [Commented] (YARN-6706) Refactor ContainerScheduler to make oversubscription change easier

2017-06-19 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16054388#comment-16054388 ] Haibo Chen commented on YARN-6706: -- Patch updated to address checkstyle and unit test issues. The findbugs

[jira] [Commented] (YARN-6670) Add separate NM overallocation thresholds for cpu and memory

2017-06-19 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16054513#comment-16054513 ] Haibo Chen commented on YARN-6670: -- Yep, filed and linked YARN-6723 for further discussion of

[jira] [Created] (YARN-6723) NM overallocation based on over-time rather than snapshot utilization

2017-06-19 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-6723: Summary: NM overallocation based on over-time rather than snapshot utilization Key: YARN-6723 URL: https://issues.apache.org/jira/browse/YARN-6723 Project: Hadoop YARN

[jira] [Updated] (YARN-6723) NM overallocation based on over-time rather than snapshot utilization

2017-06-19 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6723: - Issue Type: Sub-task (was: Bug) Parent: YARN-1011 > NM overallocation based on over-time rather

[jira] [Updated] (YARN-6723) NM overallocation based on over-time rather than snapshot utilization

2017-06-19 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6723: - Description: To continue discussion on Miklos's idea in YARN-6670 of "Usually the CPU usage fluctuates

[jira] [Commented] (YARN-6628) Unexpected jackson-core-2.2.3 dependency introduced

2017-05-19 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16018206#comment-16018206 ] Haibo Chen commented on YARN-6628: -- In branch-2.8, find hadoop-dist -name "jackson-core*jar" returns

[jira] [Commented] (YARN-5705) [YARN-3368] Add support for Timeline V2 to new web UI

2017-05-19 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16018334#comment-16018334 ] Haibo Chen commented on YARN-5705: -- +1 as well on the committing the patch. > [YARN-3368] Add support

[jira] [Commented] (YARN-6555) Enable flow context read (& corresponding write) for recovering application with NM restart

2017-05-23 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021658#comment-16021658 ] Haibo Chen commented on YARN-6555: -- Thanks [~rohithsharma] for the patch! A few comments. 1) Do you think

[jira] [Comment Edited] (YARN-6555) Enable flow context read (& corresponding write) for recovering application with NM restart

2017-05-23 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021658#comment-16021658 ] Haibo Chen edited comment on YARN-6555 at 5/23/17 6:51 PM: --- Thanks

[jira] [Comment Edited] (YARN-6555) Enable flow context read (& corresponding write) for recovering application with NM restart

2017-05-23 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021658#comment-16021658 ] Haibo Chen edited comment on YARN-6555 at 5/23/17 6:51 PM: --- Thanks

[jira] [Created] (YARN-6659) New web UI query history applications from ATSv2

2017-05-26 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-6659: Summary: New web UI query history applications from ATSv2 Key: YARN-6659 URL: https://issues.apache.org/jira/browse/YARN-6659 Project: Hadoop YARN Issue Type: Wish

[jira] [Created] (YARN-6660) Display containers of applications returned from ATSv2

2017-05-26 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-6660: Summary: Display containers of applications returned from ATSv2 Key: YARN-6660 URL: https://issues.apache.org/jira/browse/YARN-6660 Project: Hadoop YARN Issue Type:

[jira] [Updated] (YARN-6660) Display containers of applications returned from ATSv2

2017-05-26 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6660: - Issue Type: Sub-task (was: Wish) Parent: YARN-3368 > Display containers of applications returned

[jira] [Updated] (YARN-6658) Remove columnFor() methods of Columns in HBaseTimeline backend

2017-05-26 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6658: - Attachment: YARN-6658.00.patch > Remove columnFor() methods of Columns in HBaseTimeline backend >

[jira] [Updated] (YARN-6659) New web UI query history applications from ATSv2

2017-05-26 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6659: - Issue Type: Sub-task (was: Wish) Parent: YARN-3368 > New web UI query history applications from

[jira] [Created] (YARN-6658) Remove columnFor() methods of Columns in HBaseTimeline backend

2017-05-26 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-6658: Summary: Remove columnFor() methods of Columns in HBaseTimeline backend Key: YARN-6658 URL: https://issues.apache.org/jira/browse/YARN-6658 Project: Hadoop YARN

[jira] [Updated] (YARN-6658) Remove columnFor() methods of Columns in HBaseTimeline backend

2017-05-26 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6658: - Issue Type: Sub-task (was: Improvement) Parent: YARN-5355 > Remove columnFor() methods of Columns

[jira] [Resolved] (YARN-6660) Display containers of applications returned from ATSv2

2017-05-26 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen resolved YARN-6660. -- Resolution: Duplicate > Display containers of applications returned from ATSv2 >

[jira] [Updated] (YARN-6657) The column family indicated in AppToFlowTable class javadoc is wrong

2017-05-26 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6657: - Issue Type: Sub-task (was: Bug) Parent: YARN-5355 > The column family indicated in AppToFlowTable

[jira] [Created] (YARN-6656) The column family indicated in AppToFlowTable class javadoc is wrong

2017-05-26 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-6656: Summary: The column family indicated in AppToFlowTable class javadoc is wrong Key: YARN-6656 URL: https://issues.apache.org/jira/browse/YARN-6656 Project: Hadoop YARN

[jira] [Created] (YARN-6657) The column family indicated in AppToFlowTable class javadoc is wrong

2017-05-26 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-6657: Summary: The column family indicated in AppToFlowTable class javadoc is wrong Key: YARN-6657 URL: https://issues.apache.org/jira/browse/YARN-6657 Project: Hadoop YARN

[jira] [Commented] (YARN-6660) Display containers of applications returned from ATSv2

2017-05-26 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16026966#comment-16026966 ] Haibo Chen commented on YARN-6660: -- Oops, this seems a duplicate of YARN-5512. Closing this. > Display

[jira] [Updated] (YARN-6657) The column family indicated in AppToFlowTable class javadoc is wrong

2017-05-26 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6657: - Attachment: YARN-6658.00.patch > The column family indicated in AppToFlowTable class javadoc is wrong >

[jira] [Created] (YARN-6651) Flow Activity page should specify 'metricstoretrieve' in its query to ATSv2 to get back CPU and memory

2017-05-25 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-6651: Summary: Flow Activity page should specify 'metricstoretrieve' in its query to ATSv2 to get back CPU and memory Key: YARN-6651 URL: https://issues.apache.org/jira/browse/YARN-6651

[jira] [Updated] (YARN-6651) Flow Activity should specify 'metricstoretrieve' in its query to ATSv2 to retrieve CPU and memory

2017-05-25 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6651: - Summary: Flow Activity should specify 'metricstoretrieve' in its query to ATSv2 to retrieve CPU and memory

[jira] [Updated] (YARN-6651) Flow Activity should specify 'metricstoretrieve' in its query to ATSv2 to retrieve CPU and memory

2017-05-25 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6651: - Issue Type: Task (was: Bug) > Flow Activity should specify 'metricstoretrieve' in its query to ATSv2 to

[jira] [Updated] (YARN-6651) Flow Activity should specify 'metricstoretrieve' in its query to ATSv2 to retrieve CPU and memory

2017-05-25 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6651: - Description: When you click on Flow Acitivity => \{a flow\} => flow runs, the web server sends a REST

[jira] [Updated] (YARN-6651) Flow Activity should specify 'metricstoretrieve' in its query to ATSv2 to retrieve CPU and memory

2017-05-25 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6651: - Issue Type: Sub-task (was: Task) Parent: YARN-3368 > Flow Activity should specify

[jira] [Updated] (YARN-6653) Retrieve CPU and MEMORY metrics for applications in a flow run

2017-05-25 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6653: - Issue Type: Sub-task (was: Bug) Parent: YARN-3368 > Retrieve CPU and MEMORY metrics for

[jira] [Created] (YARN-6653) Retrieve CPU and MEMORY metrics for applications in a flow run

2017-05-25 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-6653: Summary: Retrieve CPU and MEMORY metrics for applications in a flow run Key: YARN-6653 URL: https://issues.apache.org/jira/browse/YARN-6653 Project: Hadoop YARN

[jira] [Updated] (YARN-6652) Merge flow info and flow runs

2017-05-25 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6652: - Issue Type: Sub-task (was: Improvement) Parent: YARN-3368 > Merge flow info and flow runs >

<    2   3   4   5   6   7   8   9   10   11   >