[jira] [Commented] (YARN-6024) Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max

2016-12-27 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782342#comment-15782342 ] Hudson commented on YARN-6024: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #1104

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2016-12-27 Thread Naganarasimha G R (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782324#comment-15782324 ] Naganarasimha G R commented on YARN-6029: - Thanks [~wangda] & [~Tao Yang] bq. I t

[jira] [Commented] (YARN-6001) Improve moveApplicationQueues command line

2016-12-27 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782263#comment-15782263 ] Hudson commented on YARN-6001: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #1104

[jira] [Updated] (YARN-6024) Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max

2016-12-27 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-6024: -- Summary: Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved r

[jira] [Commented] (YARN-5969) FairShareComparator: Cache value of getResourceUsage for better performance

2016-12-27 Thread zhangshilong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782248#comment-15782248 ] zhangshilong commented on YARN-5969: Thanks [~yufeigu] for advice and review and [~kash

[jira] [Commented] (YARN-6001) Improve moveApplicationQueues command line

2016-12-27 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782244#comment-15782244 ] Sunil G commented on YARN-6001: --- Thanks [~rohithsharma] for review and commit and thanks [~yu

[jira] [Commented] (YARN-5906) Update AppSchedulingInfo to use SchedulingPlacementSet

2016-12-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782236#comment-15782236 ] Hadoop QA commented on YARN-5906: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (YARN-6001) Improve moveApplicationQueues command line

2016-12-27 Thread Rohith Sharma K S (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782234#comment-15782234 ] Rohith Sharma K S commented on YARN-6001: - committed to trunk/branch-2.. thanks Sun

[jira] [Commented] (YARN-6024) Capacity Scheduler continuous reservation looking doesn't work when queue's used+reserved = max

2016-12-27 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782202#comment-15782202 ] Sunil G commented on YARN-6024: --- +1 Committing. > Capacity Scheduler continuous reservation

[jira] [Commented] (YARN-6031) Application recovery failed after disabling node label

2016-12-27 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782197#comment-15782197 ] Sunil G commented on YARN-6031: --- Thanks [~Ying Zhang] for raising this issue. With the help

[jira] [Commented] (YARN-6024) Capacity Scheduler continuous reservation looking doesn't work when queue's used+reserved = max

2016-12-27 Thread Ying Zhang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782185#comment-15782185 ] Ying Zhang commented on YARN-6024: -- Thanks for the details:-) > Capacity Scheduler contin

[jira] [Updated] (YARN-6031) Application recovery failed after disabling node label

2016-12-27 Thread Ying Zhang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ying Zhang updated YARN-6031: - Description: Here is the repro steps: Enable node label, restart RM, configure CS properly, and run some jo

[jira] [Updated] (YARN-6032) SharedCacheManager cleaner task should rm InMemorySCMStore some cachedResources which does not exists in hdfs fs

2016-12-27 Thread Zhaofei Meng (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhaofei Meng updated YARN-6032: --- Summary: SharedCacheManager cleaner task should rm InMemorySCMStore some cachedResources which does no

[jira] [Commented] (YARN-6032) scm cleaner task should rm InMemorySCMStore some cachedResources which does not exists in hdfs fs

2016-12-27 Thread Zhaofei Meng (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782180#comment-15782180 ] Zhaofei Meng commented on YARN-6032: We should modify use interface in ClientProtocolSe

[jira] [Commented] (YARN-1492) truly shared cache for jars (jobjar/libjar)

2016-12-27 Thread Zhaofei Meng (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782173#comment-15782173 ] Zhaofei Meng commented on YARN-1492: Another problem YARN-6032 > truly shared cache fo

[jira] [Created] (YARN-6032) scm cleaner task should rm InMemorySCMStore some cachedResources which does not exists in hdfs fs

2016-12-27 Thread Zhaofei Meng (JIRA)
Zhaofei Meng created YARN-6032: -- Summary: scm cleaner task should rm InMemorySCMStore some cachedResources which does not exists in hdfs fs Key: YARN-6032 URL: https://issues.apache.org/jira/browse/YARN-6032

[jira] [Updated] (YARN-6031) Application recovery failed after disabling node label

2016-12-27 Thread Ying Zhang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ying Zhang updated YARN-6031: - Description: Here is the repro steps: Enable node label, restart RM, configure CS properly, and run some jo

[jira] [Comment Edited] (YARN-4465) SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled

2016-12-27 Thread Ying Zhang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782147#comment-15782147 ] Ying Zhang edited comment on YARN-4465 at 12/28/16 6:00 AM: Tha

[jira] [Comment Edited] (YARN-4465) SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled

2016-12-27 Thread Ying Zhang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782151#comment-15782151 ] Ying Zhang edited comment on YARN-4465 at 12/28/16 5:58 AM: Yes

[jira] [Commented] (YARN-4465) SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled

2016-12-27 Thread Ying Zhang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782151#comment-15782151 ] Ying Zhang commented on YARN-4465: -- Yes, you're right. Please see the repro steps in YARN-

[jira] [Commented] (YARN-4465) SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled

2016-12-27 Thread Ying Zhang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782147#comment-15782147 ] Ying Zhang commented on YARN-4465: -- Thanks [~leftnoteasy], [~sunilg] and [~bibinchundatt].

[jira] [Commented] (YARN-6021) When your allocated minShare of all queue`s added up exceed cluster capacity you can get some queue for 0 fairshare

2016-12-27 Thread Feng Yuan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782149#comment-15782149 ] Feng Yuan commented on YARN-6021: - [~kasha] Thanks your detailedness reply.I think your exp

[jira] [Updated] (YARN-6031) Application recovery failed after disabling node label

2016-12-27 Thread Ying Zhang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ying Zhang updated YARN-6031: - Description: Here is the repro steps: Enable node label, restart RM, configure CS properly, and run some jo

[jira] [Updated] (YARN-6031) Application recovery failed after disabling node label

2016-12-27 Thread Ying Zhang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ying Zhang updated YARN-6031: - Description: Here is the repro steps: Enable node label, restart RM, configure CS properly, and run some jo

[jira] [Commented] (YARN-5756) Add state-machine implementation for scheduler queues

2016-12-27 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782136#comment-15782136 ] Hudson commented on YARN-5756: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #1104

[jira] [Updated] (YARN-6031) Application recovery failed after disabling node label

2016-12-27 Thread Ying Zhang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ying Zhang updated YARN-6031: - Priority: Minor (was: Major) > Application recovery failed after disabling node label > --

[jira] [Updated] (YARN-6031) Application recovery failed after disabling node label

2016-12-27 Thread Ying Zhang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ying Zhang updated YARN-6031: - Description: Here is the repro steps: Enable node label, restart RM, configure it properly, and run some jo

[jira] [Updated] (YARN-6031) Application recovery failed after disabling node label

2016-12-27 Thread Ying Zhang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ying Zhang updated YARN-6031: - Description: Here is the repro steps: Enable node label, restart RM, configure it properly, and run some jo

[jira] [Created] (YARN-6031) Application recovery failed after disabling node label

2016-12-27 Thread Ying Zhang (JIRA)
Ying Zhang created YARN-6031: Summary: Application recovery failed after disabling node label Key: YARN-6031 URL: https://issues.apache.org/jira/browse/YARN-6031 Project: Hadoop YARN Issue Type:

[jira] [Updated] (YARN-5906) Update AppSchedulingInfo to use SchedulingPlacementSet

2016-12-27 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-5906: - Attachment: YARN-5906.5.patch Attached ver.5 patch, rebased to latest trunk. > Update AppSchedulingInfo t

[jira] [Commented] (YARN-6024) Capacity Scheduler continuous reservation looking doesn't work when queue's used+reserved = max

2016-12-27 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782109#comment-15782109 ] Wangda Tan commented on YARN-6024: -- [~Ying Zhang], Gotcha, thanks for elaborate. Yeah it

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2016-12-27 Thread Li Lu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782102#comment-15782102 ] Li Lu commented on YARN-6029: - Thanks [~wangda]! bq. But it could cause inconsistency read dat

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2016-12-27 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782098#comment-15782098 ] Wangda Tan commented on YARN-6029: -- Thanks all for comments, [~Tao Yang] / [~gtCarrera9].

[jira] [Comment Edited] (YARN-6024) Capacity Scheduler continuous reservation looking doesn't work when queue's used+reserved = max

2016-12-27 Thread Ying Zhang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782092#comment-15782092 ] Ying Zhang edited comment on YARN-6024 at 12/28/16 5:25 AM: Sor

[jira] [Commented] (YARN-6024) Capacity Scheduler continuous reservation looking doesn't work when queue's used+reserved = max

2016-12-27 Thread Ying Zhang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782092#comment-15782092 ] Ying Zhang commented on YARN-6024: -- Sorry for the confusion, [~leftnoteasy] and [~sunilg]

[jira] [Commented] (YARN-5756) Add state-machine implementation for scheduler queues

2016-12-27 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782087#comment-15782087 ] Wangda Tan commented on YARN-5756: -- Committed to trunk / branch-2, thanks [~xgong] for wor

[jira] [Updated] (YARN-5756) Add state-machine implementation for scheduler queues

2016-12-27 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-5756: - Fix Version/s: 3.0.0-alpha2 2.9.0 > Add state-machine implementation for scheduler queue

[jira] [Updated] (YARN-5756) Add state-machine implementation for scheduler queues

2016-12-27 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-5756: - Summary: Add state-machine implementation for scheduler queues (was: Add state-machine implementation for

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2016-12-27 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782067#comment-15782067 ] Tao Yang commented on YARN-6029: Thanks [~gtCarrera9] for correcting me. There is something

[jira] [Commented] (YARN-220) NM should limit number of applications who's logs are being aggregated

2016-12-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782062#comment-15782062 ] Wilfred Spiegelenburg commented on YARN-220: Should this be marked as fixed now

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2016-12-27 Thread Li Lu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782007#comment-15782007 ] Li Lu commented on YARN-6029: - I'm not a scheduler expert, but "not affecting any data structur

[jira] [Commented] (YARN-5830) Avoid preempting AM containers

2016-12-27 Thread Karthik Kambatla (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781991#comment-15781991 ] Karthik Kambatla commented on YARN-5830: [~yufeigu], thanks for working on this.

[jira] [Commented] (YARN-6027) Support fromId for flows/flowrun apps

2016-12-27 Thread Varun Saxena (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781976#comment-15781976 ] Varun Saxena commented on YARN-6027: [~sunilg] bq. However if same flow is ran multiple

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2016-12-27 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781969#comment-15781969 ] Tao Yang commented on YARN-6029: Thanks [~Naganarasimha] [~djp] [~leftnoteasy] for your sug

[jira] [Comment Edited] (YARN-4465) SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled

2016-12-27 Thread Bibin A Chundatt (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781914#comment-15781914 ] Bibin A Chundatt edited comment on YARN-4465 at 12/28/16 3:20 AM: ---

[jira] [Commented] (YARN-4465) SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled

2016-12-27 Thread Bibin A Chundatt (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781914#comment-15781914 ] Bibin A Chundatt commented on YARN-4465: Thank you [~Ying Zhang] Apologies for mis

[jira] [Updated] (YARN-5830) Avoid preempting AM containers

2016-12-27 Thread Karthik Kambatla (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-5830: --- Description: While considering containers for preemption, avoid AM containers unless absolutely

[jira] [Commented] (YARN-5831) Propagate allowPreemptionFrom flag all the way down to the app

2016-12-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781770#comment-15781770 ] Hadoop QA commented on YARN-5831: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (YARN-5831) Propagate allowPreemptionFrom flag all the way down to the app

2016-12-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781718#comment-15781718 ] Hadoop QA commented on YARN-5831: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2016-12-27 Thread Naganarasimha G R (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781644#comment-15781644 ] Naganarasimha G R commented on YARN-6029: - Thanks [~djp] & [~wangda], for correctin

[jira] [Updated] (YARN-5831) Propagate allowPreemptionFrom flag all the way down to the app

2016-12-27 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-5831: --- Attachment: YARN-5831.002.patch Found another unnecessary recursion in function {{updatePreemptionVariables}}.

[jira] [Updated] (YARN-5831) Propagate allowPreemptionFrom flag all the way down to the app

2016-12-27 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-5831: --- Attachment: YARN-5831.001.patch The logic is covered by {{TestFairSchedulerPreemption.testNoPreemptionFromDisa

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2016-12-27 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781593#comment-15781593 ] Wangda Tan commented on YARN-6029: -- Thanks [~Tao Yang] for reporting this issue. [~Nagana

[jira] [Issue Comment Deleted] (YARN-4882) Change the log level to DEBUG for recovering completed applications

2016-12-27 Thread Robert Kanter (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kanter updated YARN-4882: Comment: was deleted (was: +1) > Change the log level to DEBUG for recovering completed applications

[jira] [Commented] (YARN-4882) Change the log level to DEBUG for recovering completed applications

2016-12-27 Thread Robert Kanter (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781579#comment-15781579 ] Robert Kanter commented on YARN-4882: - +1 > Change the log level to DEBUG for recoveri

[jira] [Comment Edited] (YARN-5831) Propagate allowPreemptionFrom flag all the way down to the app

2016-12-27 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781573#comment-15781573 ] Yufei Gu edited comment on YARN-5831 at 12/27/16 11:59 PM: --- There

[jira] [Commented] (YARN-5831) Propagate allowPreemptionFrom flag all the way down to the app

2016-12-27 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781573#comment-15781573 ] Yufei Gu commented on YARN-5831: There is an assumption from the original code: if the pare

[jira] [Commented] (YARN-4882) Change the log level to DEBUG for recovering completed applications

2016-12-27 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781529#comment-15781529 ] Daniel Templeton commented on YARN-4882: Test failures are unrelated, and the lack

[jira] [Commented] (YARN-5658) YARN should have a hook to delete a path from HDFS when an application ends

2016-12-27 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781518#comment-15781518 ] Jian He commented on YARN-5658: --- [~templedf], not just HDFS, allowing deleting a path from ZK

[jira] [Commented] (YARN-5709) Cleanup leader election configs and pluggability

2016-12-27 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781487#comment-15781487 ] Jian He commented on YARN-5709: --- Could you re-submit the patch with your change and retry ?

[jira] [Commented] (YARN-4882) Change the log level to DEBUG for recovering completed applications

2016-12-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781489#comment-15781489 ] Hadoop QA commented on YARN-4882: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (YARN-5709) Cleanup leader election configs and pluggability

2016-12-27 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781465#comment-15781465 ] Daniel Templeton commented on YARN-5709: Hmmm... Looks like {{ActiveStandbyElector

[jira] [Commented] (YARN-5257) Fix unreleased resources and null dereferences

2016-12-27 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781454#comment-15781454 ] Hudson commented on YARN-5257: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #1104

[jira] [Resolved] (YARN-5992) revert the visibility of interface AllocationFileLoaderService.Listener to public for outside usage

2016-12-27 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved YARN-5992. Resolution: Duplicate YARN-6000 already resolves the issue, even though this patch was posted

[jira] [Updated] (YARN-5995) Add RMStateStore metrics to monitor all RMStateStoreEventTypeTransition performance

2016-12-27 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated YARN-5995: --- Assignee: zhangyubiao > Add RMStateStore metrics to monitor all RMStateStoreEventTypeTransition

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2016-12-27 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781407#comment-15781407 ] Junping Du commented on YARN-6029: -- Thanks [~Tao Yang] for reporting the issue. I think th

[jira] [Updated] (YARN-5257) Fix unreleased resources and null dereferences

2016-12-27 Thread Robert Kanter (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kanter updated YARN-5257: Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2 2.9.0 Thanks [~yufeigu].

[jira] [Commented] (YARN-5257) Fix bad practices

2016-12-27 Thread Robert Kanter (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781383#comment-15781383 ] Robert Kanter commented on YARN-5257: - +1 > Fix bad practices > - > >

[jira] [Updated] (YARN-5257) Fix unreleased resources and null dereferences

2016-12-27 Thread Robert Kanter (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kanter updated YARN-5257: Summary: Fix unreleased resources and null dereferences (was: Fix bad practices) > Fix unreleased r

[jira] [Commented] (YARN-5962) Spelling errors in logging and exceptions for resource manager code

2016-12-27 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781378#comment-15781378 ] Hudson commented on YARN-5962: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #1104

[jira] [Created] (YARN-6030) Eliminate timelineServiceV2 boolean flag in TimelineClientImpl

2016-12-27 Thread Li Lu (JIRA)
Li Lu created YARN-6030: --- Summary: Eliminate timelineServiceV2 boolean flag in TimelineClientImpl Key: YARN-6030 URL: https://issues.apache.org/jira/browse/YARN-6030 Project: Hadoop YARN Issue Type: S

[jira] [Commented] (YARN-5709) Cleanup leader election configs and pluggability

2016-12-27 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781367#comment-15781367 ] Jian He commented on YARN-5709: --- I'm not sure which part of the patch is causing javadoc fail

[jira] [Commented] (YARN-5709) Cleanup leader election configs and pluggability

2016-12-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781360#comment-15781360 ] Hadoop QA commented on YARN-5709: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Updated] (YARN-4882) Change the log level to DEBUG for recovering completed applications

2016-12-27 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated YARN-4882: --- Attachment: YARN-4882.004.patch Fixed a typo. > Change the log level to DEBUG for recovering c

[jira] [Commented] (YARN-5962) Spelling errors in logging and exceptions for resource manager code

2016-12-27 Thread Robert Kanter (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781299#comment-15781299 ] Robert Kanter commented on YARN-5962: - +1 I'll commit this momentarily. > Spelling err

[jira] [Commented] (YARN-5969) FairShareComparator: Cache value of getResourceUsage for better performance

2016-12-27 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781275#comment-15781275 ] Hudson commented on YARN-5969: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #1104

[jira] [Commented] (YARN-5938) Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes

2016-12-27 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781274#comment-15781274 ] Hudson commented on YARN-5938: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #1104

[jira] [Commented] (YARN-5529) Create new DiskValidator class with metrics

2016-12-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781271#comment-15781271 ] Hadoop QA commented on YARN-5529: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote |

[jira] [Updated] (YARN-6022) Revert changes of AbstractResourceRequest

2016-12-27 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-6022: - Target Version/s: 2.9.0, 3.0.0-alpha2 (was: 2.8.0, 3.0.0-alpha2) > Revert changes of AbstractResourceReque

[jira] [Commented] (YARN-6022) Revert changes of AbstractResourceRequest

2016-12-27 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781255#comment-15781255 ] Junping Du commented on YARN-6022: -- Remove 2.8 from target version given YARN-5774 was not

[jira] [Updated] (YARN-5938) Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes

2016-12-27 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-5938: -- Summary: Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and o

[jira] [Commented] (YARN-5938) Refactoring OpportunisticContainerAllocator use SchedulerRequestKey instead of Priority

2016-12-27 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781216#comment-15781216 ] Arun Suresh commented on YARN-5938: --- The failed test case runs locally and the checkstyle

[jira] [Updated] (YARN-5529) Create new DiskValidator class with metrics

2016-12-27 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-5529: --- Attachment: YARN-5529.003.patch Thanks [~rkanter]'s review. I've uploaded the new patch for all your comments.

[jira] [Updated] (YARN-5969) FairShareComparator: Cache value of getResourceUsage for better performance

2016-12-27 Thread Karthik Kambatla (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-5969: --- Summary: FairShareComparator: Cache value of getResourceUsage for better performance (was: Fai

[jira] [Commented] (YARN-5969) FairShareComparator getResourceUsage poor performance

2016-12-27 Thread Karthik Kambatla (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781174#comment-15781174 ] Karthik Kambatla commented on YARN-5969: +1. Checking this in. > FairShareCompara

[jira] [Commented] (YARN-6025) Few issues in synchronization in CapacityScheduler & AbstractYarnScheduler

2016-12-27 Thread Naganarasimha G R (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781057#comment-15781057 ] Naganarasimha G R commented on YARN-6025: - Ok in that case will consider only remov

[jira] [Commented] (YARN-6029) CapacityScheduler deadlock when ParentQueue#getQueueUserAclInfo is called by Thread_A at the moment that Thread_B calls LeafQueue#assignContainers to release a reserved

2016-12-27 Thread Naganarasimha G R (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781054#comment-15781054 ] Naganarasimha G R commented on YARN-6029: - Thanks for working on the patch [~Tao Ya

[jira] [Commented] (YARN-6025) Few issues in synchronization in CapacityScheduler & AbstractYarnScheduler

2016-12-27 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781024#comment-15781024 ] Wangda Tan commented on YARN-6025: -- bq. But just one more doubt In my mind, methods in

[jira] [Updated] (YARN-5709) Cleanup leader election configs and pluggability

2016-12-27 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-5709: -- Attachment: yarn-5709-branch-2.8.02.patch > Cleanup leader election configs and pluggability > --

[jira] [Commented] (YARN-6024) Capacity Scheduler continuous reservation looking doesn't work when queue's used+reserved = max

2016-12-27 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781013#comment-15781013 ] Wangda Tan commented on YARN-6024: -- Thanks for review, [~sunilg] / [~Ying Zhang]. [~Ying

[jira] [Commented] (YARN-5969) FairShareComparator getResourceUsage poor performance

2016-12-27 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15780987#comment-15780987 ] Yufei Gu commented on YARN-5969: Thanks [~zsl2007]'s new patch. LGTM. +1(non-binding). Wou

[jira] [Commented] (YARN-4465) SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled

2016-12-27 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15780983#comment-15780983 ] Wangda Tan commented on YARN-4465: -- Nice catch, thanks [~Ying Zhang]! I think to solve t

[jira] [Commented] (YARN-3955) Support for priority ACLs in CapacityScheduler

2016-12-27 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15780979#comment-15780979 ] Sunil G commented on YARN-3955: --- java doc error is not related. Its a known failure from hado

[jira] [Commented] (YARN-6012) Remove node label (removeFromClusterNodeLabels) document is missing

2016-12-27 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15780911#comment-15780911 ] Sunil G commented on YARN-6012: --- Could you please upload a patch. I can help to review. > Re

[jira] [Commented] (YARN-5899) A small fix for printing debug info inside function canAssignToThisQueue()

2016-12-27 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15780909#comment-15780909 ] Sunil G commented on YARN-5899: --- Thanks for the patch. I ll take a look today. > A small fi

[jira] [Commented] (YARN-6021) When your allocated minShare of all queue`s added up exceed cluster capacity you can get some queue for 0 fairshare

2016-12-27 Thread Karthik Kambatla (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15780867#comment-15780867 ] Karthik Kambatla commented on YARN-6021: Excuse the long-winded response. I believ

[jira] [Assigned] (YARN-5991) Yarn Distributed Shell does not print throwable t to App Master When failed to start container

2016-12-27 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton reassigned YARN-5991: -- Assignee: Daniel Templeton > Yarn Distributed Shell does not print throwable t to App Ma

[jira] [Comment Edited] (YARN-5991) Yarn Distributed Shell does not print throwable t to App Master When failed to start container

2016-12-27 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15741409#comment-15741409 ] Daniel Templeton edited comment on YARN-5991 at 12/27/16 5:00 PM: ---

[jira] [Commented] (YARN-6020) Resource.add exceed Int boundary,when compute queue demand in FairScheduler

2016-12-27 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15780686#comment-15780686 ] Daniel Templeton commented on YARN-6020: It means that the git apply of your patch

[jira] [Commented] (YARN-4423) Cleanup lint warnings in resource mananger

2016-12-27 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15780600#comment-15780600 ] Daniel Templeton commented on YARN-4423: That's what happens when it sits for a yea

  1   2   >