[jira] [Reopened] (YARN-1115) Provide optional means for a scheduler to check real user ACLs

2018-11-01 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne reopened YARN-1115: -- This has come up again as a requirement for us. > Provide optional means for a scheduler to check real user

[jira] [Commented] (YARN-7225) Add queue and partition info to RM audit log

2018-10-31 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670827#comment-16670827 ] Eric Payne commented on YARN-7225: -- Thanks [~jhung]. The behavior is slightly different d

[jira] [Commented] (YARN-7225) Add queue and partition info to RM audit log

2018-10-23 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16661440#comment-16661440 ] Eric Payne commented on YARN-7225: -- Thanks a lot for the review, [~jhung]. I have loaded

[jira] [Updated] (YARN-7225) Add queue and partition info to RM audit log

2018-10-23 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7225: - Attachment: YARN-7225.005.patch > Add queue and partition info to RM audit log > -

[jira] [Commented] (YARN-8647) Add a flag to disable move app between queues

2018-10-16 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652189#comment-16652189 ] Eric Payne commented on YARN-8647: -- {quote}{quote}Addition flag to disable required {quot

[jira] [Updated] (YARN-7274) Ability to disable elasticity at leaf queue level

2018-10-11 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7274: - Fix Version/s: 3.0.4 2.8.5 2.9.2 2.10.0 > Ability

[jira] [Commented] (YARN-7274) Ability to disable elasticity at leaf queue level

2018-10-11 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16646624#comment-16646624 ] Eric Payne commented on YARN-7274: -- this backports cleanly all the way back to 2.8. I wil

[jira] [Commented] (YARN-7225) Add queue and partition info to RM audit log

2018-10-11 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16646487#comment-16646487 ] Eric Payne commented on YARN-7225: -- The ASF license warning is not related to this patch.

[jira] [Commented] (YARN-7225) Add queue and partition info to RM audit log

2018-10-10 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16645402#comment-16645402 ] Eric Payne commented on YARN-7225: -- I re-uploaded the branch-2.8 patch to kick off the 2.

[jira] [Updated] (YARN-7225) Add queue and partition info to RM audit log

2018-10-10 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7225: - Attachment: YARN-7225.branch-2.8.001.patch > Add queue and partition info to RM audit log > --

[jira] [Updated] (YARN-7225) Add queue and partition info to RM audit log

2018-10-10 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7225: - Attachment: (was: YARN-7225.branch-2.8.001.patch) > Add queue and partition info to RM audit log > ---

[jira] [Commented] (YARN-7225) Add queue and partition info to RM audit log

2018-10-10 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16645136#comment-16645136 ] Eric Payne commented on YARN-7225: -- Attached 004 version of the trunk patch to address th

[jira] [Updated] (YARN-7225) Add queue and partition info to RM audit log

2018-10-10 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7225: - Attachment: YARN-7225.004.patch > Add queue and partition info to RM audit log > -

[jira] [Updated] (YARN-7225) Add queue and partition info to RM audit log

2018-10-09 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7225: - Attachment: YARN-7225.branch-2.8.001.patch > Add queue and partition info to RM audit log > --

[jira] [Updated] (YARN-7225) Add queue and partition info to RM audit log

2018-10-08 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7225: - Affects Version/s: 2.9.1 2.8.4 3.0.2 3

[jira] [Updated] (YARN-7225) Add queue and partition info to RM audit log

2018-10-08 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7225: - Attachment: YARN-7225.003.patch > Add queue and partition info to RM audit log > -

[jira] [Commented] (YARN-7225) Add queue and partition info to RM audit log

2018-10-08 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642125#comment-16642125 ] Eric Payne commented on YARN-7225: -- bq. I think we can remove these right? Since this wil

[jira] [Commented] (YARN-7225) Add queue and partition info to RM audit log

2018-10-04 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16638556#comment-16638556 ] Eric Payne commented on YARN-7225: -- {quote} - In the capacity scheduler, the AM Released

[jira] [Updated] (YARN-7225) Add queue and partition info to RM audit log

2018-10-04 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7225: - Attachment: YARN-7225.002.patch > Add queue and partition info to RM audit log > -

[jira] [Commented] (YARN-7225) Add queue and partition info to RM audit log

2018-10-03 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16637142#comment-16637142 ] Eric Payne commented on YARN-7225: -- After more manual testing on 001, I discovered the fo

[jira] [Commented] (YARN-7225) Add queue and partition info to RM audit log

2018-10-02 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16636171#comment-16636171 ] Eric Payne commented on YARN-7225: -- The pre-commit failed to build: {code} [INFO] Buildin

[jira] [Updated] (YARN-7225) Add queue and partition info to RM audit log

2018-10-02 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-7225: - Attachment: YARN-7225.001.patch > Add queue and partition info to RM audit log > -

[jira] [Assigned] (YARN-7225) Add queue and partition info to RM audit log

2018-10-02 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne reassigned YARN-7225: Assignee: Eric Payne > Add queue and partition info to RM audit log > -

[jira] [Commented] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-28 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16631969#comment-16631969 ] Eric Payne commented on YARN-8774: -- Thanks [~Tao Yang] for the branch-2 and branch-2.8 pa

[jira] [Commented] (YARN-7225) Add queue and partition info to RM audit log

2018-09-20 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622458#comment-16622458 ] Eric Payne commented on YARN-7225: -- Thanks [~jhung] for raising this issue. This came up

[jira] [Commented] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-20 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622323#comment-16622323 ] Eric Payne commented on YARN-8774: -- [~Tao Yang], The patch and UT look good to me. Howeve

[jira] [Commented] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-19 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621254#comment-16621254 ] Eric Payne commented on YARN-8774: -- Thanks [~Tao Yang] for reporting this issue and provi

[jira] [Commented] (YARN-8709) CS preemption monitor always fails since one under-served queue was deleted

2018-09-10 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16609499#comment-16609499 ] Eric Payne commented on YARN-8709: -- Thanks [~Tao Yang]. +1. Will commit shortly. > CS pr

[jira] [Updated] (YARN-8709) CS preemption monitor always fails since one under-served queue was deleted

2018-09-10 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-8709: - Summary: CS preemption monitor always fails since one under-served queue was deleted (was: intra-queue pr

[jira] [Commented] (YARN-8709) intra-queue preemption checker always fail since one under-served queue was deleted

2018-09-07 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16607172#comment-16607172 ] Eric Payne commented on YARN-8709: -- [~Tao Yang], thanks for the patch. The changes look g

[jira] [Commented] (YARN-8730) TestRMWebServiceAppsNodelabel#testAppsRunning fails

2018-09-05 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604634#comment-16604634 ] Eric Payne commented on YARN-8730: -- -1 overall |Vote|Subsystem|Runtime|Comment| |0|findbu

[jira] [Updated] (YARN-8730) TestRMWebServiceAppsNodelabel#testAppsRunning fails

2018-09-04 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-8730: - Attachment: YARN-8730.001.branch-2.8.patch > TestRMWebServiceAppsNodelabel#testAppsRunning fails > ---

[jira] [Assigned] (YARN-8730) TestRMWebServiceAppsNodelabel#testAppsRunning fails

2018-08-30 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne reassigned YARN-8730: Assignee: Eric Payne > TestRMWebServiceAppsNodelabel#testAppsRunning fails > --

[jira] [Commented] (YARN-8051) TestRMEmbeddedElector#testCallbackSynchronization is flakey

2018-08-29 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596905#comment-16596905 ] Eric Payne commented on YARN-8051: -- {quote} -1 unit336m 4s hadoop-yarn-s

[jira] [Commented] (YARN-8051) TestRMEmbeddedElector#testCallbackSynchronization is flakey

2018-08-29 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596901#comment-16596901 ] Eric Payne commented on YARN-8051: -- +1. Committing to branch-2, branch-2.9, and branch-2.

[jira] [Commented] (YARN-8509) Total pending resource calculation in preemption should use user-limit factor instead of minimum-user-limit-percent

2018-08-28 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16595610#comment-16595610 ] Eric Payne commented on YARN-8509: -- bq. I'll comment on the updates once I have some prog

[jira] [Commented] (YARN-8709) intra-queue preemption checker always fail since one under-served queue was deleted

2018-08-28 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16595277#comment-16595277 ] Eric Payne commented on YARN-8709: -- [~Tao Yang], thanks for raising this issue and sugges

[jira] [Commented] (YARN-8509) Total pending resource calculation in preemption should use user-limit factor instead of minimum-user-limit-percent

2018-08-22 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16589226#comment-16589226 ] Eric Payne commented on YARN-8509: -- {code:title=UsersManager#computeUserLimit} -Resou

[jira] [Commented] (YARN-8509) Total pending resource calculation in preemption should use user-limit factor instead of minimum-user-limit-percent

2018-08-13 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16578867#comment-16578867 ] Eric Payne commented on YARN-8509: -- [~Zian Chen], thanks for your reply. Let's take a st

[jira] [Comment Edited] (YARN-8509) Total pending resource calculation in preemption should use user-limit factor instead of minimum-user-limit-percent

2018-08-10 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16576342#comment-16576342 ] Eric Payne edited comment on YARN-8509 at 8/10/18 6:12 PM: --- [~Zi

[jira] [Commented] (YARN-8509) Total pending resource calculation in preemption should use user-limit factor instead of minimum-user-limit-percent

2018-08-10 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16576342#comment-16576342 ] Eric Payne commented on YARN-8509: -- [~Zian Chen], can I please get a couple of clarificat

[jira] [Commented] (YARN-8634) Implement yarn cmd for killing all applications in the cluster

2018-08-08 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573861#comment-16573861 ] Eric Payne commented on YARN-8634: --  [~BilwaST], I tam also unsure whether or not this fe

[jira] [Commented] (YARN-8509) Fix UserLimit calculation for preemption to balance scenario after queue satisfied

2018-08-03 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16568441#comment-16568441 ] Eric Payne commented on YARN-8509: -- [~Zian Chen], [~sunilg], and others: Can you please c

[jira] [Commented] (YARN-8509) Fix UserLimit calculation for preemption to balance scenario after queue satisfied

2018-08-03 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16568258#comment-16568258 ] Eric Payne commented on YARN-8509: -- [~Zian Chen]. Unfortunately, this code cannot go in a

[jira] [Commented] (YARN-8509) Fix UserLimit calculation for preemption to balance scenario after queue satisfied

2018-08-02 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16567405#comment-16567405 ] Eric Payne commented on YARN-8509: -- I am very sorry for coming in late here. Please give

[jira] [Commented] (YARN-4606) CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps

2018-07-23 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16553038#comment-16553038 ] Eric Payne commented on YARN-4606: -- +1 Thanks for all of your work on this JIRA, [~manira

[jira] [Commented] (YARN-4606) CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps

2018-07-19 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16549772#comment-16549772 ] Eric Payne commented on YARN-4606: -- bq. Will take care of #2, #3 & #4. [~maniraj...@gmail

[jira] [Commented] (YARN-4606) CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps

2018-07-17 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547161#comment-16547161 ] Eric Payne commented on YARN-4606: -- Thank you, [~maniraj...@gmail.com], for the latest pa

[jira] [Commented] (YARN-8421) when moving app, activeUsers is increased, even though app does not have outstanding request

2018-07-13 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16543675#comment-16543675 ] Eric Payne commented on YARN-8421: -- Thanks [~kyungwan nam] for the 2.8 update. [~sunilg],

[jira] [Commented] (YARN-8421) when moving app, activeUsers is increased, even though app does not have outstanding request

2018-07-12 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541802#comment-16541802 ] Eric Payne commented on YARN-8421: -- Yes, I agree. Changes LGTM. +1 If this is a problem i

[jira] [Commented] (YARN-8421) when moving app, activeUsers is increased, even though app does not have outstanding request

2018-07-10 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539269#comment-16539269 ] Eric Payne commented on YARN-8421: -- [~kyungwan nam], Thank you for providing the fix for

[jira] [Commented] (YARN-8421) when moving app, activeUsers is increased, even though app does not have outstanding request

2018-07-09 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537517#comment-16537517 ] Eric Payne commented on YARN-8421: -- I am concerned about the findbugs and shadedclient wa

[jira] [Commented] (YARN-4606) CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps

2018-07-09 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537497#comment-16537497 ] Eric Payne commented on YARN-4606: -- {quote} So, after move app operation and if there is

[jira] [Commented] (YARN-8379) Add an option to allow Capacity Scheduler preemption to balance satisfied queues

2018-06-26 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523691#comment-16523691 ] Eric Payne commented on YARN-8379: -- [~Zian Chen] , I probably won't be able to get to thi

[jira] [Commented] (YARN-4606) CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps

2018-06-25 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522811#comment-16522811 ] Eric Payne commented on YARN-4606: -- {quote}At the same time, this patch is less "strict"

[jira] [Commented] (YARN-4606) CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps

2018-06-22 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16520822#comment-16520822 ] Eric Payne commented on YARN-4606: -- [~maniraj...@gmail.com], we can fix the queue applica

[jira] [Updated] (YARN-4606) CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps

2018-06-22 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-4606: - Attachment: YARN-4606.POC.3.patch > CapacityScheduler: applications could get starved because computation

[jira] [Commented] (YARN-8444) NodeResourceMonitor crashes on bad swapFree value

2018-06-22 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16520597#comment-16520597 ] Eric Payne commented on YARN-8444: -- Thanks [~Jim_Brennan] for the work on this JIRA. +1.

[jira] [Commented] (YARN-4606) CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps

2018-06-21 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16519562#comment-16519562 ] Eric Payne commented on YARN-4606: -- [~maniraj...@gmail.com], I am fine with using this JI

[jira] [Commented] (YARN-4606) CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps

2018-06-19 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16517557#comment-16517557 ] Eric Payne commented on YARN-4606: -- I put this Jira into PATCH AVAILABLE mode so that it

[jira] [Resolved] (YARN-8425) Yarn container getting killed due to running beyond physical memory limits

2018-06-14 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne resolved YARN-8425. -- Resolution: Not A Bug > Yarn container getting killed due to running beyond physical memory limits > ---

[jira] [Commented] (YARN-8379) Add an option to allow Capacity Scheduler preemption to balance satisfied queues

2018-06-13 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511694#comment-16511694 ] Eric Payne commented on YARN-8379: -- [~Zian Chen], I attached the confs I used to create m

[jira] [Updated] (YARN-8379) Add an option to allow Capacity Scheduler preemption to balance satisfied queues

2018-06-13 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-8379: - Attachment: ericpayne.confs.tgz > Add an option to allow Capacity Scheduler preemption to balance satisfie

[jira] [Commented] (YARN-8379) Add an option to allow Capacity Scheduler preemption to balance satisfied queues

2018-06-13 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511469#comment-16511469 ] Eric Payne commented on YARN-8379: -- [~Zian Chen], thank you for the work on this issue an

[jira] [Commented] (YARN-8379) Add an option to allow Capacity Scheduler preemption to balance satisfied queues

2018-06-12 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16509992#comment-16509992 ] Eric Payne commented on YARN-8379: -- [~Zian Chen], I will review the patch but it may take

[jira] [Commented] (YARN-8379) Add an option to allow Capacity Scheduler preemption to balance satisfied queues

2018-06-07 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504630#comment-16504630 ] Eric Payne commented on YARN-8379: -- {quote} Why is this a preemption only concept? To avo

[jira] [Commented] (YARN-8379) Add an option to allow Capacity Scheduler preemption to balance satisfied queues

2018-06-06 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503282#comment-16503282 ] Eric Payne commented on YARN-8379: -- Thanks [~Zian Chen] for working on this and providing

[jira] [Commented] (YARN-4781) Support intra-queue preemption for fairness ordering policy.

2018-06-01 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16498517#comment-16498517 ] Eric Payne commented on YARN-4781: -- bq. Thank you. I ll commit this to branch-2 shortly.

[jira] [Commented] (YARN-4606) CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps

2018-05-31 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497232#comment-16497232 ] Eric Payne commented on YARN-4606: -- Thanks [~maniraj...@gmail.com] for the updated patch.

[jira] [Commented] (YARN-8379) Add an option to allow Capacity Scheduler preemption to balance satisfied queues

2018-05-31 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496715#comment-16496715 ] Eric Payne commented on YARN-8379: -- Thanks [~leftnoteasy] for bringing this up. Yes, our

[jira] [Commented] (YARN-4606) CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps

2018-05-30 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16495656#comment-16495656 ] Eric Payne commented on YARN-4606: -- [~maniraj...@gmail.com], do you have a status on upda

[jira] [Commented] (YARN-8377) Javadoc build failed in hadoop-yarn-server-nodemanager

2018-05-30 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16495410#comment-16495410 ] Eric Payne commented on YARN-8377: -- Thanks a lot, [~tasanuma0829] for tracking this all d

[jira] [Commented] (YARN-4781) Support intra-queue preemption for fairness ordering policy.

2018-05-29 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493761#comment-16493761 ] Eric Payne commented on YARN-4781: -- Thanks a lot [~sunilg]! I attached patch {{YARN-4781.

[jira] [Updated] (YARN-4781) Support intra-queue preemption for fairness ordering policy.

2018-05-29 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-4781: - Attachment: YARN-4781.005.branch-2.patch > Support intra-queue preemption for fairness ordering policy. >

[jira] [Commented] (YARN-8292) Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative

2018-05-25 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16490886#comment-16490886 ] Eric Payne commented on YARN-8292: -- +1. Unless there are addition comments, I[ll commit to

[jira] [Commented] (YARN-8292) Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative

2018-05-24 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16489070#comment-16489070 ] Eric Payne commented on YARN-8292: -- [~leftnoteasy], thanks for the new patch. - In the fol

[jira] [Commented] (YARN-4781) Support intra-queue preemption for fairness ordering policy.

2018-05-23 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16487902#comment-16487902 ] Eric Payne commented on YARN-4781: -- bq. FairOrdering policy could be used with weights? [

[jira] [Commented] (YARN-8292) Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative

2018-05-22 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16484257#comment-16484257 ] Eric Payne commented on YARN-8292: -- {quote}Actually this is required after the change. {qu

[jira] [Commented] (YARN-8179) Preemption does not happen due to natural_termination_factor when DRF is used

2018-05-21 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483046#comment-16483046 ] Eric Payne commented on YARN-8179: -- Committed to trunk, branch-3.1, and branch-3.0. Thanks

[jira] [Commented] (YARN-8179) Preemption does not happen due to natural_termination_factor when DRF is used

2018-05-21 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482855#comment-16482855 ] Eric Payne commented on YARN-8179: -- {quote}Can we please change the {{newInstance}} call t

[jira] [Comment Edited] (YARN-8179) Preemption does not happen due to natural_termination_factor when DRF is used

2018-05-21 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482841#comment-16482841 ] Eric Payne edited comment on YARN-8179 at 5/21/18 6:22 PM: --- [~kyu

[jira] [Commented] (YARN-8179) Preemption does not happen due to natural_termination_factor when DRF is used

2018-05-21 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482841#comment-16482841 ] Eric Payne commented on YARN-8179: -- [~kyungwan nam], I am really sorry for the long delay,

[jira] [Commented] (YARN-8292) Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative

2018-05-21 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482689#comment-16482689 ] Eric Payne commented on YARN-8292: -- Thanks [~leftnoteasy] for your work on this issue. -

[jira] [Commented] (YARN-8292) Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative

2018-05-17 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16479599#comment-16479599 ] Eric Payne commented on YARN-8292: -- Thanks for the updated patch, [~leftnoteasy]. The cha

[jira] [Commented] (YARN-4781) Support intra-queue preemption for fairness ordering policy.

2018-05-16 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478184#comment-16478184 ] Eric Payne commented on YARN-4781: -- Hi [~sunilg]. Will you have an opportunity to review t

[jira] [Commented] (YARN-8292) Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative

2018-05-16 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478157#comment-16478157 ] Eric Payne commented on YARN-8292: -- bq. you can check org.apache.hadoop.yarn.server.resou

[jira] [Commented] (YARN-8292) Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative

2018-05-16 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478000#comment-16478000 ] Eric Payne commented on YARN-8292: -- {quote} ||Queue||Guaranteed||Used ||Pending|| |A|<20,

[jira] [Commented] (YARN-8292) Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative

2018-05-16 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16477674#comment-16477674 ] Eric Payne commented on YARN-8292: -- Thanks [~ssath...@hortonworks.com] and [~leftnoteasy].

[jira] [Commented] (YARN-4606) CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps

2018-05-07 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466541#comment-16466541 ] Eric Payne commented on YARN-4606: -- {code:title=AppSchedulingInfo#updatePendingResources}

[jira] [Updated] (YARN-8248) Job hangs when queue is specified and that queue has 0 capability of a resource

2018-05-07 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-8248: - Component/s: yarn fairscheduler > Job hangs when queue is specified and that queue has 0 c

[jira] [Commented] (YARN-4606) CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps

2018-05-07 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466136#comment-16466136 ] Eric Payne commented on YARN-4606: -- Thanks [~maniraj...@gmail.com] for your consistent and

[jira] [Commented] (YARN-8179) Preemption does not happen due to natural_termination_factor when DRF is used

2018-05-03 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16463041#comment-16463041 ] Eric Payne commented on YARN-8179: -- [~sunilg], if there is no objection, I'll commit this

[jira] [Comment Edited] (YARN-4781) Support intra-queue preemption for fairness ordering policy.

2018-05-01 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459919#comment-16459919 ] Eric Payne edited comment on YARN-4781 at 5/1/18 6:04 PM: -- I misse

[jira] [Commented] (YARN-4781) Support intra-queue preemption for fairness ordering policy.

2018-05-01 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459919#comment-16459919 ] Eric Payne commented on YARN-4781: -- I missed one of the two places that needed a check for

[jira] [Updated] (YARN-4781) Support intra-queue preemption for fairness ordering policy.

2018-05-01 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-4781: - Attachment: YARN-4781.005.patch > Support intra-queue preemption for fairness ordering policy. > --

[jira] [Commented] (YARN-8179) Preemption does not happen due to natural_termination_factor when DRF is used

2018-04-30 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458804#comment-16458804 ] Eric Payne commented on YARN-8179: -- [~kyungwan nam], thanks for the new patch. +1 > Pree

[jira] [Commented] (YARN-4781) Support intra-queue preemption for fairness ordering policy.

2018-04-27 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456511#comment-16456511 ] Eric Payne commented on YARN-4781: -- bq. Sorry for the delay here Eric Payne. I will check

[jira] [Commented] (YARN-8004) Add unit tests for inter queue preemption for dominant resource calculator

2018-04-27 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456369#comment-16456369 ] Eric Payne commented on YARN-8004: -- bq. could i backport this to branch-2.9/2.8 as well? [

[jira] [Commented] (YARN-8004) Add unit tests for inter queue preemption for dominant resource calculator

2018-04-24 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451226#comment-16451226 ] Eric Payne commented on YARN-8004: -- bq. So when we set the last parameter to true, we will

[jira] [Commented] (YARN-8179) Preemption does not happen due to natural_termination_factor when DRF is used

2018-04-20 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446001#comment-16446001 ] Eric Payne commented on YARN-8179: -- Can {{qT.toBePreempted}} ever be <0,0>? If so, I don't

<    2   3   4   5   6   7   8   9   10   11   >