[jira] [Updated] (YARN-8468) Limit container sizes per queue in FairScheduler

[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8468: - Labels: (was: patch) > Limit container sizes per queue in FairScheduler >

[jira] [Commented] (YARN-8605) TestDominantResourceFairnessPolicy.testModWhileSorting: flaky

[ https://issues.apache.org/jira/browse/YARN-8605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564146#comment-16564146 ] Haibo Chen commented on YARN-8605: -- +1 on the patch. Checking in shortly. >

[jira] [Updated] (YARN-8605) TestDominantResourceFairnessPolicy.testModWhileSorting is flaky

[ https://issues.apache.org/jira/browse/YARN-8605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8605: - Summary: TestDominantResourceFairnessPolicy.testModWhileSorting is flaky (was:

[jira] [Commented] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart

[ https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564137#comment-16564137 ] Haibo Chen commented on YARN-6966: -- I created HADOOP-15644 to fix the branch-2 docker build issue. >

[jira] [Commented] (YARN-7948) Enable refreshing maximum allocation for multiple resource types

[ https://issues.apache.org/jira/browse/YARN-7948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564049#comment-16564049 ] Haibo Chen commented on YARN-7948: -- Thanks [~snemeth] for updating the patch. I have a few follow up

[jira] [Commented] (YARN-8590) Fair scheduler promotion does not update container execution type and token

[ https://issues.apache.org/jira/browse/YARN-8590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562597#comment-16562597 ] Haibo Chen commented on YARN-8590: -- The findbugs and unit test failure are unrelated. WIll update the

[jira] [Commented] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart

[ https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562091#comment-16562091 ] Haibo Chen commented on YARN-6966: -- Reattached the branch-2 patch to trigger a Jenkins job. >

[jira] [Updated] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart

[ https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6966: - Attachment: YARN-6966-branch-2.002.patch > NodeManager metrics may return wrong negative values when NM

[jira] [Updated] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart

[ https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6966: - Fix Version/s: 3.0.4 > NodeManager metrics may return wrong negative values when NM restart >

[jira] [Commented] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart

[ https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562088#comment-16562088 ] Haibo Chen commented on YARN-6966: -- +1 on the branch-3.0 patch. Checking it in shortly > NodeManager

[jira] [Commented] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart

[ https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560503#comment-16560503 ] Haibo Chen commented on YARN-6966: -- The branch-2 patch does not compile for me either > NodeManager

[jira] [Commented] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart

[ https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560486#comment-16560486 ] Haibo Chen commented on YARN-6966: -- [~snemeth] We should not be doing patch for branch-3.0.0, as it is

[jira] [Commented] (YARN-7948) Enable refreshing maximum allocation for multiple resource types

[ https://issues.apache.org/jira/browse/YARN-7948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560478#comment-16560478 ] Haibo Chen commented on YARN-7948: -- Memory and CPU are always default resources, so I think it's

[jira] [Updated] (YARN-8590) Fair scheduler promotion does not update container execution type and token

[ https://issues.apache.org/jira/browse/YARN-8590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8590: - Attachment: YARN-8590-YARN-1011.01.patch > Fair scheduler promotion does not update container execution

[jira] [Updated] (YARN-8590) Fair scheduler promotion does not update container execution type and token

[ https://issues.apache.org/jira/browse/YARN-8590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8590: - Attachment: (was: YARN-8590-YARN-1011.01.patch) > Fair scheduler promotion does not update container

[jira] [Commented] (YARN-8590) Fair scheduler promotion does not update container execution type and token

[ https://issues.apache.org/jira/browse/YARN-8590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560030#comment-16560030 ] Haibo Chen commented on YARN-8590: -- Updated the patch to increment container token identifier version so

[jira] [Updated] (YARN-8590) Fair scheduler promotion does not update container execution type and token

[ https://issues.apache.org/jira/browse/YARN-8590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8590: - Attachment: YARN-8590-YARN-1011.01.patch > Fair scheduler promotion does not update container execution

[jira] [Updated] (YARN-8590) Fair scheduler promotion does not update container execution type and token

[ https://issues.apache.org/jira/browse/YARN-8590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8590: - Attachment: YARN-8590-YARN-1011.00.patch > Fair scheduler promotion does not update container execution

[jira] [Created] (YARN-8590) Fair scheduler promotion does not update container execution type and token

Haibo Chen created YARN-8590: Summary: Fair scheduler promotion does not update container execution type and token Key: YARN-8590 URL: https://issues.apache.org/jira/browse/YARN-8590 Project: Hadoop YARN

[jira] [Commented] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart

[ https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16556155#comment-16556155 ] Haibo Chen commented on YARN-6966: -- Yes, please add a patch for branch-3.0. > NodeManager metrics may

[jira] [Commented] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart

[ https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554735#comment-16554735 ] Haibo Chen commented on YARN-6966: -- I have backported it to branch-3.1, the cherry-pick into branch-3.0

[jira] [Updated] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart

[ https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6966: - Fix Version/s: 3.1.2 > NodeManager metrics may return wrong negative values when NM restart >

[jira] [Commented] (YARN-7133) Clean up lock-try order in fair scheduler

[ https://issues.apache.org/jira/browse/YARN-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554719#comment-16554719 ] Haibo Chen commented on YARN-7133: -- Makes sense. +1. Checking this in shortly. > Clean up lock-try order

[jira] [Commented] (YARN-7133) Clean up lock-try order in fair scheduler

[ https://issues.apache.org/jira/browse/YARN-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553197#comment-16553197 ] Haibo Chen commented on YARN-7133: -- [~snemeth] I am not quite familiar with the reasons why putting

[jira] [Commented] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart

[ https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553183#comment-16553183 ] Haibo Chen commented on YARN-6966: -- TestContainerSchedulerRecovery.createRecoveredContainerState() is

[jira] [Commented] (YARN-8498) Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04

[ https://issues.apache.org/jira/browse/YARN-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551471#comment-16551471 ] Haibo Chen commented on YARN-8498: -- I have finally been able to setup a Ubuntu 18.04 machine to reproduce

[jira] [Commented] (YARN-6964) Fair scheduler misuses Resources operations

[ https://issues.apache.org/jira/browse/YARN-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551046#comment-16551046 ] Haibo Chen commented on YARN-6964: -- +1. Will check it in shortly. > Fair scheduler misuses Resources

[jira] [Commented] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart

[ https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551028#comment-16551028 ] Haibo Chen commented on YARN-6966: -- +1 from me pending some other checkstyle issues  I came across 1)

[jira] [Commented] (YARN-6964) Fair scheduler misuses Resources operations

[ https://issues.apache.org/jira/browse/YARN-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550082#comment-16550082 ] Haibo Chen commented on YARN-6964: -- [~snemeth] the compilation failed. Can you please update the patch?

[jira] [Commented] (YARN-8270) Adding JMX Metrics for Timeline Collector and Reader

[ https://issues.apache.org/jira/browse/YARN-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550063#comment-16550063 ] Haibo Chen commented on YARN-8270: -- [~vrushalic] [~Sushil-K-S] My apologies for my late response. Please

[jira] [Commented] (YARN-8468) Limit container sizes per queue in FairScheduler

[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550031#comment-16550031 ] Haibo Chen commented on YARN-8468: -- [~bsteinbach] Can you please address [~wilfreds]'s comments? > Limit

[jira] [Commented] (YARN-7300) DiskValidator is not used in LocalDirAllocator

[ https://issues.apache.org/jira/browse/YARN-7300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550012#comment-16550012 ] Haibo Chen commented on YARN-7300: -- The unit test is unrelated. Checking this in shortly. >

[jira] [Commented] (YARN-6964) Fair scheduler misuses Resources operations

[ https://issues.apache.org/jira/browse/YARN-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550007#comment-16550007 ] Haibo Chen commented on YARN-6964: -- The 12th patch LGTM. I have triggered the jenkins job to see if there

[jira] [Commented] (YARN-6995) Improve use of ResourceNotFoundException in resource types code

[ https://issues.apache.org/jira/browse/YARN-6995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549957#comment-16549957 ] Haibo Chen commented on YARN-6995: -- +1 on the latest patch. Will check it in shortly. > Improve use of

[jira] [Commented] (YARN-8436) FSParentQueue: Comparison method violates its general contract

[ https://issues.apache.org/jira/browse/YARN-8436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549803#comment-16549803 ] Haibo Chen commented on YARN-8436: -- I see. +1 on the latest patch. Will check it into trunk shortly. >

[jira] [Commented] (YARN-7300) DiskValidator is not used in LocalDirAllocator

[ https://issues.apache.org/jira/browse/YARN-7300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547101#comment-16547101 ] Haibo Chen commented on YARN-7300: -- +1 on the 02 patch pending Jenkins. > DiskValidator is not used in

[jira] [Commented] (YARN-8436) FSParentQueue: Comparison method violates its general contract

[ https://issues.apache.org/jira/browse/YARN-8436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547096#comment-16547096 ] Haibo Chen commented on YARN-8436: -- Thanks [~wilfreds] for the patch! I have some minor comments 1)

[jira] [Commented] (YARN-8498) Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04

[ https://issues.apache.org/jira/browse/YARN-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546889#comment-16546889 ] Haibo Chen commented on YARN-8498: -- My apologies for the late response. I'll take a look at this later

[jira] [Commented] (YARN-7300) DiskValidator is not used in LocalDirAllocator

[ https://issues.apache.org/jira/browse/YARN-7300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545587#comment-16545587 ] Haibo Chen commented on YARN-7300: -- Thanks for the patch [~snemeth]! I have a few minor comments 1) In

[jira] [Commented] (YARN-6995) Improve use of ResourceNotFoundException in resource types code

[ https://issues.apache.org/jira/browse/YARN-6995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545553#comment-16545553 ] Haibo Chen commented on YARN-6995: -- Thanks [~snemeth] for picking this up! It looks like the 07 patch

[jira] [Commented] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart

[ https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534066#comment-16534066 ] Haibo Chen commented on YARN-6966: -- Thanks [~fly_in_gis] for the original report and patch, and

[jira] [Commented] (YARN-7556) Fair scheduler configuration should allow resource types in the minResources and maxResources properties

[ https://issues.apache.org/jira/browse/YARN-7556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16533949#comment-16533949 ] Haibo Chen commented on YARN-7556: -- +1 on the latest patch. Will commit it to trunk shortly. > Fair

[jira] [Commented] (YARN-6672) Add NM preemption of opportunistic containers when utilization goes high

[ https://issues.apache.org/jira/browse/YARN-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16530517#comment-16530517 ] Haibo Chen commented on YARN-6672: -- Thanks [~elgoiri] for the extensive reviews, and [~szegedim] for

[jira] [Commented] (YARN-6672) Add NM preemption of opportunistic containers when utilization goes high

[ https://issues.apache.org/jira/browse/YARN-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528269#comment-16528269 ] Haibo Chen commented on YARN-6672: -- Thanks [~elgoiri] for the review! {quote} * In

[jira] [Updated] (YARN-6672) Add NM preemption of opportunistic containers when utilization goes high

[ https://issues.apache.org/jira/browse/YARN-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6672: - Attachment: YARN-6672-YARN-1011.06.patch > Add NM preemption of opportunistic containers when utilization

[jira] [Commented] (YARN-7334) Add documentation of oversubscription

[ https://issues.apache.org/jira/browse/YARN-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528267#comment-16528267 ] Haibo Chen commented on YARN-7334: -- Sure, [~elgoiri] I will get on this once we are done with more

[jira] [Updated] (YARN-6672) Add NM preemption of opportunistic containers when utilization goes high

[ https://issues.apache.org/jira/browse/YARN-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6672: - Attachment: YARN-6672-YARN-1011.05.patch > Add NM preemption of opportunistic containers when utilization

[jira] [Commented] (YARN-6672) Add NM preemption of opportunistic containers when utilization goes high

[ https://issues.apache.org/jira/browse/YARN-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527882#comment-16527882 ] Haibo Chen commented on YARN-6672: -- Thanks [~elgoiri] for the follow-up review!  {quote} * Not sure

[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate

[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526926#comment-16526926 ] Haibo Chen commented on YARN-1013: -- {quote} where is the enforcement flag? {quote} It is per

[jira] [Assigned] (YARN-8470) Fair scheduler exception with SLS

[ https://issues.apache.org/jira/browse/YARN-8470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen reassigned YARN-8470: Assignee: Haibo Chen > Fair scheduler exception with SLS > - > >

[jira] [Updated] (YARN-6672) Add NM preemption of opportunistic containers when utilization goes high

[ https://issues.apache.org/jira/browse/YARN-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6672: - Attachment: YARN-6672-YARN-1011.04.patch > Add NM preemption of opportunistic containers when utilization

[jira] [Commented] (YARN-6672) Add NM preemption of opportunistic containers when utilization goes high

[ https://issues.apache.org/jira/browse/YARN-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524371#comment-16524371 ] Haibo Chen commented on YARN-6672: -- Thanks for the review, [~elgoiri]! I updated the patch to address

[jira] [Updated] (YARN-6672) Add NM preemption of opportunistic containers when utilization goes high

[ https://issues.apache.org/jira/browse/YARN-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6672: - Attachment: YARN-6672-YARN-1011.03.patch > Add NM preemption of opportunistic containers when utilization

[jira] [Commented] (YARN-8461) Support strict memory control on individual container with elastic control memory mechanism

[ https://issues.apache.org/jira/browse/YARN-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524273#comment-16524273 ] Haibo Chen commented on YARN-8461: -- Good catch. I updated the patch. > Support strict memory control on

[jira] [Updated] (YARN-8461) Support strict memory control on individual container with elastic control memory mechanism

[ https://issues.apache.org/jira/browse/YARN-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8461: - Attachment: YARN-8461.03.patch > Support strict memory control on individual container with elastic

[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate

[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524129#comment-16524129 ] Haibo Chen commented on YARN-1013: -- To add a bit of context to YARN-1015 (or YARN-1011 as a whole), we

[jira] [Commented] (YARN-8461) Support strict memory control on individual container with elastic control memory mechanism

[ https://issues.apache.org/jira/browse/YARN-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524106#comment-16524106 ] Haibo Chen commented on YARN-8461: -- I updated the patch again to address the unit test failure. >

[jira] [Updated] (YARN-8461) Support strict memory control on individual container with elastic control memory mechanism

[ https://issues.apache.org/jira/browse/YARN-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8461: - Attachment: YARN-8461.02.patch > Support strict memory control on individual container with elastic

[jira] [Updated] (YARN-8461) Support strict memory control on individual container with elastic control memory mechanism

[ https://issues.apache.org/jira/browse/YARN-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8461: - Attachment: YARN-8461.01.patch > Support strict memory control on individual container with elastic

[jira] [Commented] (YARN-8461) Support strict memory control on individual container with elastic control memory mechanism

[ https://issues.apache.org/jira/browse/YARN-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523992#comment-16523992 ] Haibo Chen commented on YARN-8461: -- Thanks for the review, [~szegedim]! I have updated the patch to

[jira] [Updated] (YARN-8462) Resource Manager shutdown with FATAL Exception

[ https://issues.apache.org/jira/browse/YARN-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8462: - Component/s: capacity scheduler > Resource Manager shutdown with FATAL Exception >

[jira] [Updated] (YARN-8461) Support strict memory control on individual container with elastic control memory mechanism

[ https://issues.apache.org/jira/browse/YARN-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8461: - Attachment: YARN-8461.00.patch > Support strict memory control on individual container with elastic

[jira] [Created] (YARN-8461) Support strict memory control on individual container with elastic control memory mechanism

Haibo Chen created YARN-8461: Summary: Support strict memory control on individual container with elastic control memory mechanism Key: YARN-8461 URL: https://issues.apache.org/jira/browse/YARN-8461

[jira] [Updated] (YARN-6672) Add NM preemption of opportunistic containers when utilization goes high

[ https://issues.apache.org/jira/browse/YARN-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6672: - Attachment: YARN-6672-YARN-1011.02.patch > Add NM preemption of opportunistic containers when utilization

[jira] [Updated] (YARN-6672) Add NM preemption of opportunistic containers when utilization goes high

[ https://issues.apache.org/jira/browse/YARN-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6672: - Attachment: YARN-6672-YARN-1011.01.patch > Add NM preemption of opportunistic containers when utilization

[jira] [Commented] (YARN-6672) Add NM preemption of opportunistic containers when utilization goes high

[ https://issues.apache.org/jira/browse/YARN-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520853#comment-16520853 ] Haibo Chen commented on YARN-6672: -- Attached a patch to do the NM periodical preemption. In the meantime,

[jira] [Updated] (YARN-6672) Add NM preemption of opportunistic containers when utilization goes high

[ https://issues.apache.org/jira/browse/YARN-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6672: - Attachment: YARN-6672-YARN-1011.00.patch > Add NM preemption of opportunistic containers when utilization

[jira] [Commented] (YARN-8427) Don't start opportunistic containers at container scheduler/finish event with over-allocation

[ https://issues.apache.org/jira/browse/YARN-8427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520817#comment-16520817 ] Haibo Chen commented on YARN-8427: -- Thanks [~szegedim] for your review! > Don't start opportunistic

[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate

[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519860#comment-16519860 ] Haibo Chen commented on YARN-1013: -- [~cheersyang] FYI, there is YARN-6794 that does container promotion

[jira] [Updated] (YARN-8427) Don't start opportunistic containers at container scheduler/finish event with over-allocation

[ https://issues.apache.org/jira/browse/YARN-8427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8427: - Attachment: YARN-8427-YARN-1011.01.patch > Don't start opportunistic containers at container

[jira] [Updated] (YARN-8437) Build oom-listener fails on older versions

[ https://issues.apache.org/jira/browse/YARN-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8437: - Summary: Build oom-listener fails on older versions (was: Build oom-listener on older versions) > Build

[jira] [Commented] (YARN-8437) Build oom-listener on older versions

[ https://issues.apache.org/jira/browse/YARN-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16518386#comment-16518386 ] Haibo Chen commented on YARN-8437: -- +1 on the patch. Will commit it shortly. > Build oom-listener on

[jira] [Updated] (YARN-8427) Don't start opportunistic containers at container scheduler/finish event with over-allocation

[ https://issues.apache.org/jira/browse/YARN-8427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8427: - Attachment: YARN-8427-YARN-1011.00.patch > Don't start opportunistic containers at container

[jira] [Comment Edited] (YARN-6675) Add NM support to launch opportunistic containers based on overallocation

[ https://issues.apache.org/jira/browse/YARN-6675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16518346#comment-16518346 ] Haibo Chen edited comment on YARN-6675 at 6/20/18 4:56 PM: --- As discussed in

[jira] [Comment Edited] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16518344#comment-16518344 ] Haibo Chen edited comment on YARN-8250 at 6/20/18 4:56 PM: --- Closing this in

[jira] [Commented] (YARN-6675) Add NM support to launch opportunistic containers based on overallocation

[ https://issues.apache.org/jira/browse/YARN-6675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16518346#comment-16518346 ] Haibo Chen commented on YARN-6675: -- As discussed in YARN-8250, we'll continue with YARN-6675 + YARN-8247.

[jira] [Commented] (YARN-6794) Fair Scheduler to explicitly promote OPPORTUNISITIC containers locally at the node where they're running

[ https://issues.apache.org/jira/browse/YARN-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16518293#comment-16518293 ] Haibo Chen commented on YARN-6794: -- Indeed. I have updated the patch to address that and the checkstyle

[jira] [Updated] (YARN-6794) Fair Scheduler to explicitly promote OPPORTUNISITIC containers locally at the node where they're running

[ https://issues.apache.org/jira/browse/YARN-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6794: - Attachment: YARN-6794-YARN-1011.02.patch > Fair Scheduler to explicitly promote OPPORTUNISITIC containers

[jira] [Updated] (YARN-6794) Fair Scheduler to explicitly promote OPPORTUNISITIC containers locally at the node where they're running

[ https://issues.apache.org/jira/browse/YARN-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6794: - Attachment: YARN-6794-YARN-1011.01.patch > Fair Scheduler to explicitly promote OPPORTUNISITIC containers

[jira] [Commented] (YARN-6794) Fair Scheduler to explicitly promote OPPORTUNISITIC containers locally at the node where they're running

[ https://issues.apache.org/jira/browse/YARN-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517513#comment-16517513 ] Haibo Chen commented on YARN-6794: -- Thanks [~szegedim] for the review!. {quote}I think this should refer

[jira] [Commented] (YARN-6586) YARN to facilitate HTTPS in AM web server

[ https://issues.apache.org/jira/browse/YARN-6586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16513038#comment-16513038 ] Haibo Chen commented on YARN-6586: -- [~rkanter]  I have one question about setting the lifetime of the

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16512924#comment-16512924 ] Haibo Chen commented on YARN-8250: -- Thanks [~kkaranasos] for your comments and suggestions! {quote}I

[jira] [Created] (YARN-8427) Don't start opportunistic containers at container scheduler/finish event with over-allocation

Haibo Chen created YARN-8427: Summary: Don't start opportunistic containers at container scheduler/finish event with over-allocation Key: YARN-8427 URL: https://issues.apache.org/jira/browse/YARN-8427

[jira] [Updated] (YARN-8427) Don't start opportunistic containers at container scheduler/finish event with over-allocation

[ https://issues.apache.org/jira/browse/YARN-8427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8427: - Issue Type: Sub-task (was: Improvement) Parent: YARN-1011 > Don't start opportunistic containers

[jira] [Commented] (YARN-8270) Adding JMX Metrics for Timeline Collector and Reader

[ https://issues.apache.org/jira/browse/YARN-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511938#comment-16511938 ] Haibo Chen commented on YARN-8270: -- Thanks [~Sushil-K-S] for updating the patch! The unit test failures

[jira] [Assigned] (YARN-6409) RM does not blacklist node for AM launch failures

[ https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen reassigned YARN-6409: Assignee: Kanwaljeet Sachdev (was: Haibo Chen) > RM does not blacklist node for AM launch

[jira] [Resolved] (YARN-6800) Add opportunity to start containers while periodically checking for preemption

[ https://issues.apache.org/jira/browse/YARN-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen resolved YARN-6800. -- Resolution: Implemented Fix Version/s: YARN-1011 Resolving this as it is already done as part of

[jira] [Commented] (YARN-8325) Miscellaneous QueueManager code clean up

[ https://issues.apache.org/jira/browse/YARN-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509893#comment-16509893 ] Haibo Chen commented on YARN-8325: -- +1. Checking in shortly. > Miscellaneous QueueManager code clean up

[jira] [Commented] (YARN-6931) Make the aggregation interval in AppLevelTimelineCollector configurable

[ https://issues.apache.org/jira/browse/YARN-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509884#comment-16509884 ] Haibo Chen commented on YARN-6931: -- +1. Checking in shortly. > Make the aggregation interval in

[jira] [Commented] (YARN-8325) Miscellaneous QueueManager code clean up

[ https://issues.apache.org/jira/browse/YARN-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508485#comment-16508485 ] Haibo Chen commented on YARN-8325: -- [~snemeth] the patch no longer applies. Can you update it? >

[jira] [Commented] (YARN-8321) AllocationFileLoaderService. getAllocationFile() should be declared as VisibleForTest

[ https://issues.apache.org/jira/browse/YARN-8321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508479#comment-16508479 ] Haibo Chen commented on YARN-8321: -- +1. Checking it in shortly. > AllocationFileLoaderService.

[jira] [Commented] (YARN-8322) Change log level when there is an IOException when the allocation file is loaded

[ https://issues.apache.org/jira/browse/YARN-8322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508475#comment-16508475 ] Haibo Chen commented on YARN-8322: -- +1 > Change log level when there is an IOException when the

[jira] [Commented] (YARN-8323) FairScheduler.allocConf should be declared as volatile

[ https://issues.apache.org/jira/browse/YARN-8323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508458#comment-16508458 ] Haibo Chen commented on YARN-8323: -- +1. Checking in shortly. > FairScheduler.allocConf should be

[jira] [Commented] (YARN-6931) Make the aggregation interval in AppLevelTimelineCollector configurable

[ https://issues.apache.org/jira/browse/YARN-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508447#comment-16508447 ] Haibo Chen commented on YARN-6931: -- Thanks [~abmodi] for the patch. I have one minor comment.  We can

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508385#comment-16508385 ] Haibo Chen commented on YARN-8250: -- In the meantime, I'll try to see how well YARN-6675 works with some

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508382#comment-16508382 ] Haibo Chen commented on YARN-8250: -- My apologies, Arun. I did not mean to indicate in any way you are

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506296#comment-16506296 ] Haibo Chen commented on YARN-8250: -- Ping [~asuresh].  > Create another implementation of

[jira] [Commented] (YARN-6677) Preempt opportunistic containers when root container cgroup goes over memory limit

[ https://issues.apache.org/jira/browse/YARN-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506151#comment-16506151 ] Haibo Chen commented on YARN-6677: -- Thanks for the review, [~szegedim]! > Preempt opportunistic

[jira] [Updated] (YARN-6794) Fair Scheduler to explicitly promote OPPORTUNISITIC containers locally at the node where they're running

[ https://issues.apache.org/jira/browse/YARN-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6794: - Attachment: YARN-6794-YARN-1011.00.patch > Fair Scheduler to explicitly promote OPPORTUNISITIC containers

[jira] [Commented] (YARN-6677) Preempt opportunistic containers when root container cgroup goes over memory limit

[ https://issues.apache.org/jira/browse/YARN-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504288#comment-16504288 ] Haibo Chen commented on YARN-6677: -- Updated the patch again to address the new findbugs issue > Preempt

<    1   2   3   4   5   6   7   8   9   10   >