[jira] [Updated] (YARN-1015) FS should watch resource utilization of containers and allocate speculative containers if appropriate

2017-10-23 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-1015: - Attachment: YARN-1015-YARN-1011.prelim.patch Upload a preliminary patch on top of the latest (v8)

[jira] [Commented] (YARN-7460) Exclude findbugs warnings on SchedulerNode.numGuaranteedContainers and numOpportunisticContainers

2017-11-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253856#comment-16253856 ] Haibo Chen commented on YARN-7460: -- Also, the volatile variables are there long before YARN-4511, so I

[jira] [Commented] (YARN-7390) All reservation related test cases failed when TestYarnClient runs against Fair Scheduler.

2017-11-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254205#comment-16254205 ] Haibo Chen commented on YARN-7390: -- +1. Will commit it later today if there is no objection > All

[jira] [Updated] (YARN-7491) Make sure AM is not scheduled on an opportunistic container

2017-11-14 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7491: - Issue Type: Sub-task (was: Bug) Parent: YARN-1011 > Make sure AM is not scheduled on an

[jira] [Commented] (YARN-1015) FS should watch node resource utilization and allocate opportunistic containers if appropriate

2017-11-14 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252212#comment-16252212 ] Haibo Chen commented on YARN-1015: -- [~asuresh] Have you got a change to look at the latest patch yet? >

[jira] [Commented] (YARN-7491) Make sure AM is not scheduled on an opportunistic container

2017-11-27 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268065#comment-16268065 ] Haibo Chen commented on YARN-7491: -- That's because of this {code} @@ -491,7 +491,7 @@ private RMAppImpl

[jira] [Commented] (YARN-7491) Make sure AM is not scheduled on an opportunistic container

2017-11-28 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269330#comment-16269330 ] Haibo Chen commented on YARN-7491: -- Thanks [~miklos.szeg...@cloudera.com] for the comments. I have updated

[jira] [Updated] (YARN-7491) Make sure AM is not scheduled on an opportunistic container

2017-11-28 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7491: - Attachment: YARN-7491-YARN-1011.01.patch > Make sure AM is not scheduled on an opportunistic container >

[jira] [Commented] (YARN-7491) Make sure AM is not scheduled on an opportunistic container

2017-11-28 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269506#comment-16269506 ] Haibo Chen commented on YARN-7491: -- Thanks [~miklos.szeg...@cloudera.com] for the reviews! > Make sure AM

[jira] [Commented] (YARN-7213) [Umbrella] Test and validate HBase-2.0.x with Atsv2

2017-11-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16263838#comment-16263838 ] Haibo Chen commented on YARN-7213: -- Sorry for the delay, [~openinx]. The issue turns out to be a misuse of

[jira] [Updated] (YARN-7213) [Umbrella] Test and validate HBase-2.0.x with Atsv2

2017-11-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7213: - Attachment: YARN-7213.prelim.patch > [Umbrella] Test and validate HBase-2.0.x with Atsv2 >

[jira] [Commented] (YARN-7213) [Umbrella] Test and validate HBase-2.0.x with Atsv2

2017-11-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16263850#comment-16263850 ] Haibo Chen commented on YARN-7213: -- The patch includes changes to the final HBase FilterList that is built

[jira] [Created] (YARN-7581) ATSv2 does not construct HBase filters correctly in HBase 2.0

2017-11-29 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-7581: Summary: ATSv2 does not construct HBase filters correctly in HBase 2.0 Key: YARN-7581 URL: https://issues.apache.org/jira/browse/YARN-7581 Project: Hadoop YARN

[jira] [Updated] (YARN-7581) ATSv2 does not construct HBase filters correctly in HBase 2.0

2017-11-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7581: - Issue Type: Sub-task (was: Bug) Parent: YARN-7213 > ATSv2 does not construct HBase filters

[jira] [Updated] (YARN-7581) ATSv2 does not construct HBase filters correctly in HBase 2.0

2017-11-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7581: - Description: TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and

[jira] [Commented] (YARN-7581) ATSv2 does not construct HBase filters correctly in HBase 2.0

2017-11-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271360#comment-16271360 ] Haibo Chen commented on YARN-7581: -- We have the relevant data (entity 5 that is supposed to be returned in

[jira] [Updated] (YARN-7581) ATSv2 does not construct HBase filters correctly in HBase 2.0

2017-11-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7581: - Attachment: YARN-7581.prelim.patch > ATSv2 does not construct HBase filters correctly in HBase 2.0 >

[jira] [Updated] (YARN-7346) Fix compilation errors against hbase2 alpha release

2017-11-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7346: - Attachment: YARN-7346.prelim1.patch > Fix compilation errors against hbase2 alpha release >

[jira] [Commented] (YARN-7346) Fix compilation errors against hbase2 alpha release

2017-11-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271852#comment-16271852 ] Haibo Chen commented on YARN-7346: -- update the patch to address issues identified in the jenkins reports

[jira] [Commented] (YARN-7491) Make sure AM is not scheduled on an opportunistic container

2017-11-27 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267819#comment-16267819 ] Haibo Chen commented on YARN-7491: -- Thanks for the review [~miklos.szeg...@cloudera.com]! I think it is

[jira] [Commented] (YARN-6921) Allow resource request to opt out of oversubscription in Fair Scheduler

2017-11-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16263041#comment-16263041 ] Haibo Chen commented on YARN-6921: -- Thanks for the review, [~miklos.szeg...@cloudera.com]! > Allow

[jira] [Updated] (YARN-7491) Make sure AM is not scheduled on an opportunistic container

2017-11-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7491: - Attachment: YARN-7491-YARN-1011.00.patch > Make sure AM is not scheduled on an opportunistic container >

[jira] [Updated] (YARN-7337) Expose per-node over-allocation info in Node Report

2017-11-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7337: - Fix Version/s: YARN-1011 > Expose per-node over-allocation info in Node Report >

[jira] [Updated] (YARN-6921) Allow resource request to opt out of oversubscription in Fair Scheduler

2017-11-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6921: - Fix Version/s: YARN-1011 > Allow resource request to opt out of oversubscription in Fair Scheduler >

[jira] [Updated] (YARN-6750) Add a configuration to cap how much a NM can be overallocated

2017-11-22 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6750: - Fix Version/s: YARN-1011 > Add a configuration to cap how much a NM can be overallocated >

[jira] [Updated] (YARN-7346) Fix compilation errors against hbase2 alpha release

2017-11-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7346: - Attachment: YARN-7346.prelim2.patch > Fix compilation errors against hbase2 alpha release >

[jira] [Commented] (YARN-7581) ATSv2 does not construct HBase filters correctly in HBase 2.0

2017-11-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271415#comment-16271415 ] Haibo Chen commented on YARN-7581: -- The final HBase filter is constructed in entity readers as A

[jira] [Commented] (YARN-7213) [Umbrella] Test and validate HBase-2.0.x with Atsv2

2017-11-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271425#comment-16271425 ] Haibo Chen commented on YARN-7213: -- [~rohithsharma] Thanks for your comments! I have submitted a patch

[jira] [Comment Edited] (YARN-7346) Fix compilation errors against hbase2 alpha release

2017-11-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271276#comment-16271276 ] Haibo Chen edited comment on YARN-7346 at 11/29/17 6:14 PM: Attached a patch

[jira] [Updated] (YARN-7346) Fix compilation errors against hbase2 alpha release

2017-11-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7346: - Attachment: YARN-7581.prelim.patch Attached a patch that upgrades HBase from 1.2.6 to 2.0-alpha4 now that

[jira] [Commented] (YARN-7581) ATSv2 does not construct HBase filters correctly in HBase 2.0

2017-11-29 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271320#comment-16271320 ] Haibo Chen commented on YARN-7581: -- Based on the discussion I had with our internal HBase folks, the issue

[jira] [Updated] (YARN-7602) NM should reference the singleton JvmMetrics instance

2017-12-03 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7602: - Attachment: YARN-7602.00.patch > NM should reference the singleton JvmMetrics instance >

[jira] [Commented] (YARN-7346) Fix compilation errors against hbase2 alpha release

2017-11-30 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273982#comment-16273982 ] Haibo Chen commented on YARN-7346: -- Looks like it is related to HBase netty shading. See HBASE-18271 and

[jira] [Updated] (YARN-7346) Fix compilation errors against hbase2 alpha release

2017-11-30 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7346: - Attachment: YARN-7346.00.patch > Fix compilation errors against hbase2 alpha release >

[jira] [Comment Edited] (YARN-7346) Fix compilation errors against hbase2 alpha release

2017-11-30 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273982#comment-16273982 ] Haibo Chen edited comment on YARN-7346 at 12/1/17 6:05 AM: --- Looks like it is

[jira] [Commented] (YARN-7213) [Umbrella] Test and validate HBase-2.0.x with Atsv2

2017-11-16 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255427#comment-16255427 ] Haibo Chen commented on YARN-7213: -- [~rohithsharma] One of the test failures, testGetEntitiesConfigFilter

[jira] [Commented] (YARN-7213) [Umbrella] Test and validate HBase-2.0.x with Atsv2

2017-11-16 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255448#comment-16255448 ] Haibo Chen commented on YARN-7213: -- [~openinx] The problem seems to be around

[jira] [Commented] (YARN-1015) FS should watch node resource utilization and allocate opportunistic containers if appropriate

2017-11-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257112#comment-16257112 ] Haibo Chen commented on YARN-1015: -- Thanks [~asuresh] and [~miklos.szeg...@cloudera.com] for the reviews!

[jira] [Updated] (YARN-6921) Allow resource request to opt out of oversubscription in Fair Scheduler

2017-11-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6921: - Attachment: YARN-6921-YARN-1011.00.patch > Allow resource request to opt out of oversubscription in Fair

[jira] [Updated] (YARN-6750) Add a configuration to cap how much a NM can be overallocated

2017-11-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6750: - Attachment: YARN-6750-YARN-1011.00.patch > Add a configuration to cap how much a NM can be overallocated >

[jira] [Commented] (YARN-7337) Expose per-node over-allocation info in Node Report

2017-11-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257532#comment-16257532 ] Haibo Chen commented on YARN-7337: -- The license warning and findbug issue are unrelated, so are the

[jira] [Updated] (YARN-7337) Expose per-node over-allocation info in Node Report

2017-11-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7337: - Attachment: YARN-7337-YARN-1011.04.patch > Expose per-node over-allocation info in Node Report >

[jira] [Created] (YARN-7491) Make sure AM is not scheduled on an opportunistic container

2017-11-13 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-7491: Summary: Make sure AM is not scheduled on an opportunistic container Key: YARN-7491 URL: https://issues.apache.org/jira/browse/YARN-7491 Project: Hadoop YARN Issue

[jira] [Updated] (YARN-7531) ResourceRequest.equal does not check ExecutionTypeRequest.enforceExecutionType()

2017-11-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7531: - Attachment: YARN-7531.01.patch Thanks [~yufeigu] for the review. I added a unit test that covers the bug.

[jira] [Updated] (YARN-6750) Add a configuration to cap how much a NM can be overallocated

2017-11-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6750: - Attachment: YARN-6750-YARN-1011.01.patch Thanks [~miklos.szeg...@cloudera.com] for the review. I address

[jira] [Commented] (YARN-6750) Add a configuration to cap how much a NM can be overallocated

2017-11-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260130#comment-16260130 ] Haibo Chen commented on YARN-6750: -- The findbug warning is unrelated. Will address the checkstyle issue in

[jira] [Updated] (YARN-6750) Add a configuration to cap how much a NM can be overallocated

2017-11-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6750: - Attachment: YARN-6750-YARN-1011.02.patch > Add a configuration to cap how much a NM can be overallocated >

[jira] [Commented] (YARN-6921) Allow resource request to opt out of oversubscription in Fair Scheduler

2017-11-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260108#comment-16260108 ] Haibo Chen commented on YARN-6921: -- YARN-5652 tracks

[jira] [Updated] (YARN-6921) Allow resource request to opt out of oversubscription in Fair Scheduler

2017-11-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6921: - Attachment: YARN-6921-YARN-1011.01.patch > Allow resource request to opt out of oversubscription in Fair

[jira] [Commented] (YARN-7531) ResourceRequest.equal does not check ExecutionTypeRequest.enforceExecutionType()

2017-11-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260120#comment-16260120 ] Haibo Chen commented on YARN-7531: -- The findbug warning is unrelated. > ResourceRequest.equal does not

[jira] [Commented] (YARN-6921) Allow resource request to opt out of oversubscription in Fair Scheduler

2017-11-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261046#comment-16261046 ] Haibo Chen commented on YARN-6921: -- TestIncreaseAllocationExpirer

[jira] [Comment Edited] (YARN-7337) Expose per-node over-allocation info in Node Report

2017-11-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261072#comment-16261072 ] Haibo Chen edited comment on YARN-7337 at 11/21/17 5:18 PM: The findbug issue

[jira] [Commented] (YARN-7548) TestCapacityOverTimePolicy.testAllocation is flaky

2017-11-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261090#comment-16261090 ] Haibo Chen commented on YARN-7548: -- CC [~subru] > TestCapacityOverTimePolicy.testAllocation is flaky >

[jira] [Updated] (YARN-7337) Expose per-node over-allocation info in Node Report

2017-11-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7337: - Attachment: YARN-7337-YARN-1011.05.patch I have updated the patch to address your comments

[jira] [Commented] (YARN-7337) Expose per-node over-allocation info in Node Report

2017-11-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261072#comment-16261072 ] Haibo Chen commented on YARN-7337: -- The findbug issue is unrelated. There were no unit test failure in the

[jira] [Created] (YARN-7548) TestCapacityOverTimePolicy.testAllocation is flaky

2017-11-21 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-7548: Summary: TestCapacityOverTimePolicy.testAllocation is flaky Key: YARN-7548 URL: https://issues.apache.org/jira/browse/YARN-7548 Project: Hadoop YARN Issue Type: Bug

[jira] [Commented] (YARN-6750) Add a configuration to cap how much a NM can be overallocated

2017-11-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261048#comment-16261048 ] Haibo Chen commented on YARN-6750: -- The findbug issue is unrelated and the checkstyle warning is bogus. >

[jira] [Commented] (YARN-7390) All reservation related test cases failed when TestYarnClient runs against Fair Scheduler.

2017-11-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261291#comment-16261291 ] Haibo Chen commented on YARN-7390: -- Reopened to trigger the branch-2 jenkins job. > All reservation

[jira] [Reopened] (YARN-7390) All reservation related test cases failed when TestYarnClient runs against Fair Scheduler.

2017-11-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen reopened YARN-7390: -- > All reservation related test cases failed when TestYarnClient runs against > Fair Scheduler. >

[jira] [Commented] (YARN-7390) All reservation related test cases failed when TestYarnClient runs against Fair Scheduler.

2017-11-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261312#comment-16261312 ] Haibo Chen commented on YARN-7390: -- +1 on the branch-2 patch pending Jenkins. > All reservation related

[jira] [Commented] (YARN-7390) All reservation related test cases failed when TestYarnClient runs against Fair Scheduler.

2017-11-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261504#comment-16261504 ] Haibo Chen commented on YARN-7390: -- The unit test issue is bogus, no actual unit test failures caused by

[jira] [Commented] (YARN-7337) Expose per-node over-allocation info in Node Report

2017-11-21 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261668#comment-16261668 ] Haibo Chen commented on YARN-7337: -- NO unit test failures even though there is a -1 from unit test task.

[jira] [Created] (YARN-7514) TestAggregatedLogDeletionService.testRefreshLogRetentionSettings is flaky

2017-11-16 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-7514: Summary: TestAggregatedLogDeletionService.testRefreshLogRetentionSettings is flaky Key: YARN-7514 URL: https://issues.apache.org/jira/browse/YARN-7514 Project: Hadoop YARN

[jira] [Updated] (YARN-7514) TestAggregatedLogDeletionService.testRefreshLogRetentionSettings is flaky

2017-11-16 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7514: - Description: TestAggregatedLogDeletionService.testRefreshLogRetentionSettings fails occasionally with

[jira] [Commented] (YARN-7213) [Umbrella] Test and validate HBase-2.0.x with Atsv2

2017-11-16 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256141#comment-16256141 ] Haibo Chen commented on YARN-7213: -- [~te...@apache.org] Please hold your further investigation. Our

[jira] [Comment Edited] (YARN-7213) [Umbrella] Test and validate HBase-2.0.x with Atsv2

2017-11-16 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256466#comment-16256466 ] Haibo Chen edited comment on YARN-7213 at 11/17/17 4:42 AM: [~openinx],

[jira] [Commented] (YARN-7213) [Umbrella] Test and validate HBase-2.0.x with Atsv2

2017-11-16 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256466#comment-16256466 ] Haibo Chen commented on YARN-7213: -- [~openinx], [~te...@apache.org] Here is a test that you can add to

[jira] [Comment Edited] (YARN-7213) [Umbrella] Test and validate HBase-2.0.x with Atsv2

2017-11-16 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256466#comment-16256466 ] Haibo Chen edited comment on YARN-7213 at 11/17/17 4:44 AM: [~openinx],

[jira] [Commented] (YARN-7213) [Umbrella] Test and validate HBase-2.0.x with Atsv2

2017-11-16 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256470#comment-16256470 ] Haibo Chen commented on YARN-7213: -- cc [~varun_saxena] and [~vrushalic] on this issue. > [Umbrella] Test

[jira] [Updated] (YARN-7213) [Umbrella] Test and validate HBase-2.0.x with Atsv2

2017-11-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7213: - Attachment: YARN-7213.wip.patch Upload a WIP patch that simply upgrade HBase to the latest branch-2 HEAD.

[jira] [Comment Edited] (YARN-7213) [Umbrella] Test and validate HBase-2.0.x with Atsv2

2017-11-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254635#comment-16254635 ] Haibo Chen edited comment on YARN-7213 at 11/16/17 2:01 AM: Upload a WIP patch

[jira] [Updated] (YARN-7531) ResourceRequest.equal does not check ExecutionTypeRequest.enforceExecutionType()

2017-11-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7531: - Priority: Major (was: Critical) > ResourceRequest.equal does not check >

[jira] [Created] (YARN-7531) ResourceRequest.equal does not check ExecutionTypeRequest.enforceExecutionType()

2017-11-17 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-7531: Summary: ResourceRequest.equal does not check ExecutionTypeRequest.enforceExecutionType() Key: YARN-7531 URL: https://issues.apache.org/jira/browse/YARN-7531 Project: Hadoop

[jira] [Updated] (YARN-7531) ResourceRequest.equal does not check ExecutionTypeRequest.enforceExecutionType()

2017-11-17 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7531: - Attachment: YARN-7531.prelim.patch Upload a preliminary patch to check what tests fails because of this

[jira] [Commented] (YARN-4511) Common scheduler changes supporting scheduler-specific implementations

2017-11-01 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234822#comment-16234822 ] Haibo Chen commented on YARN-4511: -- [~asuresh] [~leftnoteasy] Do you have more comments on the latest

[jira] [Commented] (YARN-7346) Fix compilation errors against hbase2 alpha release

2017-11-03 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16238152#comment-16238152 ] Haibo Chen commented on YARN-7346: -- Sorry for the delay folks. I'll resume working on this next week. >

[jira] [Commented] (YARN-7360) TestRM.testNMTokenSentForNormalContainer() should be scheduler agnostic

2017-11-06 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16241196#comment-16241196 ] Haibo Chen commented on YARN-7360: -- Thanks [~rkanter] for the review. Checking this in shortly. >

[jira] [Commented] (YARN-4511) Common scheduler changes supporting scheduler-specific implementations

2017-11-02 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16236032#comment-16236032 ] Haibo Chen commented on YARN-4511: -- Checking in this shortly. Thanks [~asuresh]

[jira] [Updated] (YARN-4511) Common scheduler changes to support scheduler-specific oversubscription implementations

2017-11-02 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-4511: - Summary: Common scheduler changes to support scheduler-specific oversubscription implementations (was:

[jira] [Updated] (YARN-1015) FS should watch node resource utilization and allocate opportunistic containers if appropriate

2017-11-02 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-1015: - Attachment: YARN-1015-YARN-1011.00.patch > FS should watch node resource utilization and allocate

[jira] [Updated] (YARN-7337) Expose per-node over-allocation info in Node Report

2017-11-02 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7337: - Attachment: YARN-7337-YARN-1011.00.patch > Expose per-node over-allocation info in Node Report >

[jira] [Updated] (YARN-7360) TestRM.testNMTokenSentForNormalContainer() should be scheduler agnostic

2017-11-02 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7360: - Attachment: YARN-7360.01.patch Upload the patch again to trigger Jenkins given I could not manage to

[jira] [Updated] (YARN-1015) FS should watch node resource utilization and allocate opportunistic containers if appropriate

2017-11-02 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-1015: - Attachment: YARN-1015-YARN-1011.01.patch Upload a new patch to address the unit test failures and most of

[jira] [Updated] (YARN-7456) TestAMRMClient.testAMRMClientWithContainerResourceChange[0] failed

2017-11-07 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7456: - Attachment: std.out Attached a log of the unit test failure. >

[jira] [Commented] (YARN-1015) FS should watch node resource utilization and allocate opportunistic containers if appropriate

2017-11-07 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16243117#comment-16243117 ] Haibo Chen commented on YARN-1015: -- bq. Could the allowed resource for overallocation change later while

[jira] [Commented] (YARN-7346) Fix compilation errors against hbase2 alpha release

2017-11-07 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16243152#comment-16243152 ] Haibo Chen commented on YARN-7346: -- [~tedyu] How does the conflict arise? In branch-2, the hbase jars are

[jira] [Updated] (YARN-7460) Exclude findbugs warnings on SchedulerNode.numGuaranteedContainers and numOpportunisticContainers

2017-11-07 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7460: - Issue Type: Sub-task (was: Bug) Parent: YARN-1011 > Exclude findbugs warnings on

[jira] [Created] (YARN-7460) Exclude findbugs warnings on SchedulerNode.numGuaranteedContainers and numOpportunisticContainers

2017-11-07 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-7460: Summary: Exclude findbugs warnings on SchedulerNode.numGuaranteedContainers and numOpportunisticContainers Key: YARN-7460 URL: https://issues.apache.org/jira/browse/YARN-7460

[jira] [Updated] (YARN-7460) Exclude findbugs warnings on SchedulerNode.numGuaranteedContainers and numOpportunisticContainers

2017-11-07 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7460: - Affects Version/s: YARN-1011 Target Version/s: YARN-1011 > Exclude findbugs warnings on

[jira] [Updated] (YARN-7460) Exclude findbugs warnings on SchedulerNode.numGuaranteedContainers and numOpportunisticContainers

2017-11-07 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7460: - Attachment: YARN-7460-YARN-1011.00.patch > Exclude findbugs warnings on

[jira] [Updated] (YARN-1015) FS should watch node resource utilization and allocate opportunistic containers if appropriate

2017-11-07 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-1015: - Attachment: YARN-1015-YARN-1011.02.patch Upload a patch to address comments on the promotion TODO, space

[jira] [Created] (YARN-7456) TestAMRMClient.testAMRMClientWithContainerResourceChange[0] failed

2017-11-07 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-7456: Summary: TestAMRMClient.testAMRMClientWithContainerResourceChange[0] failed Key: YARN-7456 URL: https://issues.apache.org/jira/browse/YARN-7456 Project: Hadoop YARN

[jira] [Comment Edited] (YARN-1015) FS should watch node resource utilization and allocate opportunistic containers if appropriate

2017-11-07 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16243117#comment-16243117 ] Haibo Chen edited comment on YARN-1015 at 11/8/17 12:05 AM: bq. Could the

[jira] [Commented] (YARN-7388) TestAMRestart should be scheduler agnostic

2017-11-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246296#comment-16246296 ] Haibo Chen commented on YARN-7388: -- [~subru] Let me know if it is still ok to cherry-pick this into 2.9.0.

[jira] [Commented] (YARN-7388) TestAMRestart should be scheduler agnostic

2017-11-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246247#comment-16246247 ] Haibo Chen commented on YARN-7388: -- Thanks [~rkanter] for the review. Will commit it shortly >

[jira] [Commented] (YARN-7388) TestAMRestart should be scheduler agnostic

2017-11-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246563#comment-16246563 ] Haibo Chen commented on YARN-7388: -- Sounds good. Thanks for the update! > TestAMRestart should be

[jira] [Commented] (YARN-7460) Exclude findbugs warnings on SchedulerNode.numGuaranteedContainers and numOpportunisticContainers

2017-11-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246574#comment-16246574 ] Haibo Chen commented on YARN-7460: -- This is so that read of the container numbers does not need to grab a

[jira] [Commented] (YARN-7337) Expose per-node over-allocation info in Node Report

2017-11-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246571#comment-16246571 ] Haibo Chen commented on YARN-7337: -- Thanks for the review, [~miklos.szeg...@cloudera.com]. bq. naming a

[jira] [Commented] (YARN-7337) Expose per-node over-allocation info in Node Report

2017-11-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246789#comment-16246789 ] Haibo Chen commented on YARN-7337: -- That sounds good to me. Will address it in the next patch. > Expose

[jira] [Updated] (YARN-7337) Expose per-node over-allocation info in Node Report

2017-11-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7337: - Attachment: YARN-7337-YARN-1011.02.patch > Expose per-node over-allocation info in Node Report >

<    5   6   7   8   9   10   11   12   13   14   >