[jira] [Commented] (YARN-2026) Fair scheduler : Fair share for inactive queues causes unfair allocation in some scenarios

[ https://issues.apache.org/jira/browse/YARN-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038029#comment-14038029 ] Sandy Ryza commented on YARN-2026: -- I think it might be simpler to just: * Create

[jira] [Commented] (YARN-2176) CapacityScheduler loops over all running applications rather than actively requesting apps

[ https://issues.apache.org/jira/browse/YARN-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038045#comment-14038045 ] Sandy Ryza commented on YARN-2176: -- Can we merge the ActiveUsersManager stuff into an

[jira] [Commented] (YARN-2176) CapacityScheduler loops over all running applications rather than actively requesting apps

[ https://issues.apache.org/jira/browse/YARN-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14035902#comment-14035902 ] Sandy Ryza commented on YARN-2176: -- The Fair Scheduler should probably avoid this as well.

[jira] [Commented] (YARN-2176) CapacityScheduler loops over all running applications rather than actively requesting apps

[ https://issues.apache.org/jira/browse/YARN-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14035925#comment-14035925 ] Sandy Ryza commented on YARN-2176: -- Ah, good point, you're right. In that case what do

[jira] [Commented] (YARN-2176) CapacityScheduler loops over all running applications rather than actively requesting apps

[ https://issues.apache.org/jira/browse/YARN-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14036130#comment-14036130 ] Sandy Ryza commented on YARN-2176: -- Without the ActivationCallback, the ActiveUsersManager

[jira] [Commented] (YARN-2072) RM/NM UIs and webservices are missing vcore information

[ https://issues.apache.org/jira/browse/YARN-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14036145#comment-14036145 ] Sandy Ryza commented on YARN-2072: -- In the Fair Scheduler web page we report every

[jira] [Commented] (YARN-2026) Fair scheduler : Fair share for inactive queues causes unfair allocation in some scenarios

[ https://issues.apache.org/jira/browse/YARN-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14036890#comment-14036890 ] Sandy Ryza commented on YARN-2026: -- [~ashwinshankar77], can we implement this as a

[jira] [Commented] (YARN-409) Allow apps to be killed via the RM REST API

[ https://issues.apache.org/jira/browse/YARN-409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14033950#comment-14033950 ] Sandy Ryza commented on YARN-409: - definitely. will close this because there seems to be

[jira] [Resolved] (YARN-409) Allow apps to be killed via the RM REST API

[ https://issues.apache.org/jira/browse/YARN-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved YARN-409. - Resolution: Duplicate Allow apps to be killed via the RM REST API

[jira] [Commented] (YARN-2140) Add support for network IO isolation/scheduling for containers

[ https://issues.apache.org/jira/browse/YARN-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026675#comment-14026675 ] Sandy Ryza commented on YARN-2140: -- +1 to a design sketch with alternatives Add support

[jira] [Commented] (YARN-2139) Add support for disk IO isolation/scheduling for containers

[ https://issues.apache.org/jira/browse/YARN-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14027359#comment-14027359 ] Sandy Ryza commented on YARN-2139: -- With short-circuit local reads, reads on local data do

[jira] [Commented] (YARN-1913) With Fair Scheduler, cluster can logjam when all resources are consumed by AMs

[ https://issues.apache.org/jira/browse/YARN-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14015518#comment-14015518 ] Sandy Ryza commented on YARN-1913: -- This is looking good. A small things.

[jira] [Commented] (YARN-2026) Fair scheduler : Fair share for inactive queues causes unfair allocation in some scenarios

[ https://issues.apache.org/jira/browse/YARN-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14016155#comment-14016155 ] Sandy Ryza commented on YARN-2026: -- Hi Ashwin, have been busy with other stuff and

[jira] [Commented] (YARN-1913) With Fair Scheduler, cluster can logjam when all resources are consumed by AMs

[ https://issues.apache.org/jira/browse/YARN-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14014716#comment-14014716 ] Sandy Ryza commented on YARN-1913: -- The primary benefit of the logic in

[jira] [Commented] (YARN-2010) RM can't transition to active if it can't recover an app attempt

[ https://issues.apache.org/jira/browse/YARN-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14013398#comment-14013398 ] Sandy Ryza commented on YARN-2010: -- Agree with Vinod that non-secure cluster to secure

[jira] [Commented] (YARN-1913) With Fair Scheduler, cluster can logjam when all resources are consumed by AMs

[ https://issues.apache.org/jira/browse/YARN-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14014304#comment-14014304 ] Sandy Ryza commented on YARN-1913: -- Thanks for the updated patch Wei. For queues,

[jira] [Commented] (YARN-2054) Poor defaults for YARN ZK configs for retries and retry-inteval

[ https://issues.apache.org/jira/browse/YARN-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14012674#comment-14012674 ] Sandy Ryza commented on YARN-2054: -- +1 Poor defaults for YARN ZK configs for retries and

[jira] [Commented] (YARN-1368) Common work to re-populate containers’ state into scheduler

[ https://issues.apache.org/jira/browse/YARN-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14013013#comment-14013013 ] Sandy Ryza commented on YARN-1368: -- Regardless of the technical approach taken in any

[jira] [Commented] (YARN-2091) Add ContainerExitStatus.KILL_EXCEEDED_MEMORY and pass it to app masters

[ https://issues.apache.org/jira/browse/YARN-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14013119#comment-14013119 ] Sandy Ryza commented on YARN-2091: -- ContainerExitStatus should stay an int. While

[jira] [Commented] (YARN-1474) Make schedulers services

[ https://issues.apache.org/jira/browse/YARN-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011507#comment-14011507 ] Sandy Ryza commented on YARN-1474: -- My opinion is that it's ok to change these semantics,

[jira] [Resolved] (YARN-2068) FairScheduler uses the same ResourceCalculator for all policies

[ https://issues.apache.org/jira/browse/YARN-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved YARN-2068. -- Resolution: Invalid Closing this as invalid. Obviously feel free to reopen if I'm missing something.

[jira] [Commented] (YARN-2026) Fair scheduler : Fair share for inactive queues causes unfair allocation in some scenarios

[ https://issues.apache.org/jira/browse/YARN-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011523#comment-14011523 ] Sandy Ryza commented on YARN-2026: -- The nice thing about fair share currently is that it's

[jira] [Created] (YARN-2111) In FairScheduler.attemptScheduling, we won't count containers as assigned if they have 0 memory but non-zero cores

Sandy Ryza created YARN-2111: Summary: In FairScheduler.attemptScheduling, we won't count containers as assigned if they have 0 memory but non-zero cores Key: YARN-2111 URL:

[jira] [Assigned] (YARN-2111) In FairScheduler.attemptScheduling, we won't count containers as assigned if they have 0 memory but non-zero cores

[ https://issues.apache.org/jira/browse/YARN-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned YARN-2111: Assignee: Sandy Ryza In FairScheduler.attemptScheduling, we won't count containers as assigned if

[jira] [Updated] (YARN-2111) In FairScheduler.attemptScheduling, we won't count containers as assigned if they have 0 memory but non-zero cores

[ https://issues.apache.org/jira/browse/YARN-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-2111: - Attachment: YARN-2111.patch In FairScheduler.attemptScheduling, we won't count containers as assigned if

[jira] [Commented] (YARN-596) In fair scheduler, intra-application container priorities affect inter-application preemption decisions

[ https://issues.apache.org/jira/browse/YARN-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011540#comment-14011540 ] Sandy Ryza commented on YARN-596: - +1 In fair scheduler, intra-application container

[jira] [Commented] (YARN-596) In fair scheduler, intra-application container priorities affect inter-application preemption decisions

[ https://issues.apache.org/jira/browse/YARN-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011542#comment-14011542 ] Sandy Ryza commented on YARN-596: - (pending Jenkins) In fair scheduler, intra-application

[jira] [Commented] (YARN-1913) With Fair Scheduler, cluster can logjam when all resources are consumed by AMs

[ https://issues.apache.org/jira/browse/YARN-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011587#comment-14011587 ] Sandy Ryza commented on YARN-1913: -- I think we should avoid doing approximate calculation

[jira] [Updated] (YARN-2111) In FairScheduler.attemptScheduling, we don't count containers as assigned if they have 0 memory but non-zero cores

[ https://issues.apache.org/jira/browse/YARN-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-2111: - Summary: In FairScheduler.attemptScheduling, we don't count containers as assigned if they have 0 memory

[jira] [Commented] (YARN-596) In fair scheduler, intra-application container priorities affect inter-application preemption decisions

[ https://issues.apache.org/jira/browse/YARN-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14012030#comment-14012030 ] Sandy Ryza commented on YARN-596: - +1 In fair scheduler, intra-application container

[jira] [Updated] (YARN-596) Use scheduling policies throughout the hierarchy to decide which containers to preempt

[ https://issues.apache.org/jira/browse/YARN-596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-596: Summary: Use scheduling policies throughout the hierarchy to decide which containers to preempt (was: In

[jira] [Updated] (YARN-596) Use scheduling policies throughout the queue hierarchy to decide which containers to preempt

[ https://issues.apache.org/jira/browse/YARN-596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-596: Summary: Use scheduling policies throughout the queue hierarchy to decide which containers to preempt (was:

[jira] [Commented] (YARN-596) Use scheduling policies throughout the queue hierarchy to decide which containers to preempt

[ https://issues.apache.org/jira/browse/YARN-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14012038#comment-14012038 ] Sandy Ryza commented on YARN-596: - I just committed this to trunk and branch-2. Thanks Wei

[jira] [Resolved] (YARN-2106) TestFairScheduler in trunk is failing

[ https://issues.apache.org/jira/browse/YARN-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved YARN-2106. -- Resolution: Duplicate TestFairScheduler in trunk is failing -

[jira] [Commented] (YARN-2105) Three TestFairScheduler tests fail in trunk

[ https://issues.apache.org/jira/browse/YARN-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010497#comment-14010497 ] Sandy Ryza commented on YARN-2105: -- +1. Thanks for the quick turnaround on this Ashwin.

[jira] [Updated] (YARN-2108) Show minShare on RM Fair Scheduler page

[ https://issues.apache.org/jira/browse/YARN-2108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-2108: - Summary: Show minShare on RM Fair Scheduler page (was: Show minShare on RM Scheduler page) Show

[jira] [Updated] (YARN-2105) Fix TestFairScheduler after YARN-2012

[ https://issues.apache.org/jira/browse/YARN-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-2105: - Summary: Fix TestFairScheduler after YARN-2012 (was: Three TestFairScheduler tests fail in trunk) Fix

[jira] [Commented] (YARN-596) In fair scheduler, intra-application container priorities affect inter-application preemption decisions

[ https://issues.apache.org/jira/browse/YARN-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010791#comment-14010791 ] Sandy Ryza commented on YARN-596: - Thanks Wei. Getting close - a few more comments. {code}

[jira] [Commented] (YARN-2083) In fair scheduler, Queue should not been assigned more containers when its usedResource had reach the maxResource limit

[ https://issues.apache.org/jira/browse/YARN-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007786#comment-14007786 ] Sandy Ryza commented on YARN-2083: -- I think it would be better to solve this by making

[jira] [Commented] (YARN-2012) Fair Scheduler : Default rule in queue placement policy can take a queue as an optional attribute

[ https://issues.apache.org/jira/browse/YARN-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007797#comment-14007797 ] Sandy Ryza commented on YARN-2012: -- +1 Fair Scheduler : Default rule in queue placement

[jira] [Updated] (YARN-2012) Fair Scheduler: allow default queue placement rule to take an arbitrary queue

[ https://issues.apache.org/jira/browse/YARN-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-2012: - Summary: Fair Scheduler: allow default queue placement rule to take an arbitrary queue (was: Fair

[jira] [Commented] (YARN-2012) Fair Scheduler: allow default queue placement rule to take an arbitrary queue

[ https://issues.apache.org/jira/browse/YARN-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007807#comment-14007807 ] Sandy Ryza commented on YARN-2012: -- I committed this to trunk and branch-2. Thanks

[jira] [Commented] (YARN-2073) FairScheduler starts preempting resources even with free resources on the cluster

[ https://issues.apache.org/jira/browse/YARN-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007821#comment-14007821 ] Sandy Ryza commented on YARN-2073: -- +1 FairScheduler starts preempting resources even

[jira] [Updated] (YARN-2073) Fair Scheduler: Add a utilization threshold to prevent preempting resources when cluster is free

[ https://issues.apache.org/jira/browse/YARN-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-2073: - Summary: Fair Scheduler: Add a utilization threshold to prevent preempting resources when cluster is free

[jira] [Commented] (YARN-2099) Preemption in fair scheduler should consider app priorities

[ https://issues.apache.org/jira/browse/YARN-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007841#comment-14007841 ] Sandy Ryza commented on YARN-2099: -- I think the desired behavior here should be that a

[jira] [Commented] (YARN-596) In fair scheduler, intra-application container priorities affect inter-application preemption decisions

[ https://issues.apache.org/jira/browse/YARN-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14006609#comment-14006609 ] Sandy Ryza commented on YARN-596: - Ah, I see what you're saying. Good point. In that case

[jira] [Commented] (YARN-596) In fair scheduler, intra-application container priorities affect inter-application preemption decisions

[ https://issues.apache.org/jira/browse/YARN-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14006641#comment-14006641 ] Sandy Ryza commented on YARN-596: - Sounds good In fair scheduler, intra-application

[jira] [Commented] (YARN-596) In fair scheduler, intra-application container priorities affect inter-application preemption decisions

[ https://issues.apache.org/jira/browse/YARN-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14006676#comment-14006676 ] Sandy Ryza commented on YARN-596: - The current patch uses the queue's policy to

[jira] [Commented] (YARN-2073) FairScheduler starts preempting resources even with free resources on the cluster

[ https://issues.apache.org/jira/browse/YARN-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14006682#comment-14006682 ] Sandy Ryza commented on YARN-2073: -- {code} + /** Preemption related variables */ {code}

[jira] [Commented] (YARN-2089) FairScheduler: QueuePlacementPolicy and QueuePlacementRule are missing audience annotations

[ https://issues.apache.org/jira/browse/YARN-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14005206#comment-14005206 ] Sandy Ryza commented on YARN-2089: -- +1 FairScheduler: QueuePlacementPolicy and

[jira] [Updated] (YARN-2091) Add ContainerExitStatus.KILL_EXCEEDED_MEMORY and pass it to app masters

[ https://issues.apache.org/jira/browse/YARN-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-2091: - Summary: Add ContainerExitStatus.KILL_EXCEEDED_MEMORY and pass it to app masters (was: Add

[jira] [Commented] (YARN-2093) Fair Scheduler IllegalStateException after upgrade from 2.2.0 to 2.4.1-SNAP

[ https://issues.apache.org/jira/browse/YARN-2093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14005397#comment-14005397 ] Sandy Ryza commented on YARN-2093: -- Thanks for reporting this Jon. Did this occur in an

[jira] [Commented] (YARN-2012) Fair Scheduler : Default rule in queue placement policy can take a queue as an optional attribute

[ https://issues.apache.org/jira/browse/YARN-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14005409#comment-14005409 ] Sandy Ryza commented on YARN-2012: -- My thinking is that

[jira] [Commented] (YARN-2073) FairScheduler starts preempting resources even with free resources on the cluster

[ https://issues.apache.org/jira/browse/YARN-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004290#comment-14004290 ] Sandy Ryza commented on YARN-2073: -- There are some situations where preemption with free

[jira] [Commented] (YARN-2012) Fair Scheduler : Default rule in queue placement policy can take a queue as an optional attribute

[ https://issues.apache.org/jira/browse/YARN-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004307#comment-14004307 ] Sandy Ryza commented on YARN-2012: -- {code} + defaultQueueName = root. +

[jira] [Assigned] (YARN-2068) FairScheduler uses the same ResourceCalculator for all policies

[ https://issues.apache.org/jira/browse/YARN-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned YARN-2068: Assignee: Sandy Ryza (was: Karthik Kambatla) FairScheduler uses the same ResourceCalculator for

[jira] [Commented] (YARN-2068) FairScheduler uses the same ResourceCalculator for all policies

[ https://issues.apache.org/jira/browse/YARN-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14000854#comment-14000854 ] Sandy Ryza commented on YARN-2068: -- I think this is not the case. The code is slightly

[jira] [Updated] (YARN-2068) FairScheduler uses the same ResourceCalculator for all policies

[ https://issues.apache.org/jira/browse/YARN-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-2068: - Assignee: Karthik Kambatla (was: Sandy Ryza) FairScheduler uses the same ResourceCalculator for all

[jira] [Commented] (YARN-2054) Poor defaults for YARN ZK configs for retries and retry-inteval

[ https://issues.apache.org/jira/browse/YARN-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13998337#comment-13998337 ] Sandy Ryza commented on YARN-2054: -- +1 Poor defaults for YARN ZK configs for retries and

[jira] [Updated] (YARN-1986) In Fifo Scheduler, node heartbeat in between creating app and attempt causes NPE

[ https://issues.apache.org/jira/browse/YARN-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1986: - Summary: In Fifo Scheduler, node heartbeat in between creating app and attempt causes NPE (was: After

[jira] [Commented] (YARN-2027) YARN ignores host-specific resource requests

[ https://issues.apache.org/jira/browse/YARN-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13997750#comment-13997750 ] Sandy Ryza commented on YARN-2027: -- Including a rack in your request will allow containers

[jira] [Commented] (YARN-1986) After upgrade from 2.2.0 to 2.4.0, NPE on first job start.

[ https://issues.apache.org/jira/browse/YARN-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13997224#comment-13997224 ] Sandy Ryza commented on YARN-1986: -- Sorry for being so slow on this. +1 to the change. I

[jira] [Assigned] (YARN-1986) After upgrade from 2.2.0 to 2.4.0, NPE on first job start.

[ https://issues.apache.org/jira/browse/YARN-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned YARN-1986: Assignee: Sandy Ryza (was: Hong Zhiguo) After upgrade from 2.2.0 to 2.4.0, NPE on first job

[jira] [Commented] (YARN-2017) Merge some of the common lib code in schedulers

[ https://issues.apache.org/jira/browse/YARN-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13995898#comment-13995898 ] Sandy Ryza commented on YARN-2017: -- Thanks for working on this Jian. A couple questions:

[jira] [Updated] (YARN-1986) In Fifo Scheduler, node heartbeat in between creating app and attempt causes NPE

[ https://issues.apache.org/jira/browse/YARN-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1986: - Assignee: Hong Zhiguo (was: Sandy Ryza) In Fifo Scheduler, node heartbeat in between creating app and

[jira] [Commented] (YARN-2042) String shouldn't be compared using == in QueuePlacementRule#NestedUserQueue#getQueueForApp()

[ https://issues.apache.org/jira/browse/YARN-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13997334#comment-13997334 ] Sandy Ryza commented on YARN-2042: -- +1 String shouldn't be compared using == in

[jira] [Commented] (YARN-2027) YARN ignores host-specific resource requests

[ https://issues.apache.org/jira/browse/YARN-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13994652#comment-13994652 ] Sandy Ryza commented on YARN-2027: -- YARN doesn't guarantee any node-locality unless you

[jira] [Updated] (YARN-1864) Fair Scheduler Dynamic Hierarchical User Queues

[ https://issues.apache.org/jira/browse/YARN-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1864: - Assignee: Ashwin Shankar Fair Scheduler Dynamic Hierarchical User Queues

[jira] [Commented] (YARN-1899) Capacity Scheduler Usage History

[ https://issues.apache.org/jira/browse/YARN-1899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13991062#comment-13991062 ] Sandy Ryza commented on YARN-1899: -- By megabyte-seconds and vcore-seconds, do you mean the

[jira] [Commented] (YARN-1864) Fair Scheduler Dynamic Hierarchical User Queues

[ https://issues.apache.org/jira/browse/YARN-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13991168#comment-13991168 ] Sandy Ryza commented on YARN-1864: -- Thanks Ashwin, it's looking good, a few more comments

[jira] [Commented] (YARN-1864) Fair Scheduler Dynamic Hierarchical User Queues

[ https://issues.apache.org/jira/browse/YARN-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13991303#comment-13991303 ] Sandy Ryza commented on YARN-1864: -- Your reasoning about isTerminal makes sense to me.

[jira] [Commented] (YARN-1864) Fair Scheduler Dynamic Hierarchical User Queues

[ https://issues.apache.org/jira/browse/YARN-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13991419#comment-13991419 ] Sandy Ryza commented on YARN-1864: -- +1 pending Jenkins Fair Scheduler Dynamic

[jira] [Commented] (YARN-1961) Fair scheduler preemption doesn't work for non-leaf queues

[ https://issues.apache.org/jira/browse/YARN-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988518#comment-13988518 ] Sandy Ryza commented on YARN-1961: -- Hey Ashwin, The history is only that we haven't yet

[jira] [Commented] (YARN-1368) Common work to re-populate containers’ state into scheduler

[ https://issues.apache.org/jira/browse/YARN-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987182#comment-13987182 ] Sandy Ryza commented on YARN-1368: -- +1 to sharing common code between schedulers where

[jira] [Commented] (YARN-1963) Support priorities across applications within the same queue

[ https://issues.apache.org/jira/browse/YARN-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985193#comment-13985193 ] Sandy Ryza commented on YARN-1963: -- Thanks for picking this up Sunil. Can we separate

[jira] [Commented] (YARN-1864) Fair Scheduler Dynamic Hierarchical User Queues

[ https://issues.apache.org/jira/browse/YARN-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981589#comment-13981589 ] Sandy Ryza commented on YARN-1864: -- I think it's better to leave out case 4. The right

[jira] [Commented] (YARN-483) Improve documentation on log aggregation in yarn-default.xml

[ https://issues.apache.org/jira/browse/YARN-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981596#comment-13981596 ] Sandy Ryza commented on YARN-483: - +1 Improve documentation on log aggregation in

[jira] [Commented] (YARN-1864) Fair Scheduler Dynamic Hierarchical User Queues

[ https://issues.apache.org/jira/browse/YARN-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980479#comment-13980479 ] Sandy Ryza commented on YARN-1864: -- Your argument for checking whether the input to the

[jira] [Updated] (YARN-1969) Earliest Deadline First Scheduling in the Fair Scheduler

[ https://issues.apache.org/jira/browse/YARN-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1969: - Summary: Earliest Deadline First Scheduling in the Fair Scheduler (was: Earliest Deadline First

[jira] [Commented] (YARN-1796) container-executor shouldn't require o-r permissions

[ https://issues.apache.org/jira/browse/YARN-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13976282#comment-13976282 ] Sandy Ryza commented on YARN-1796: -- +1 container-executor shouldn't require o-r

[jira] [Created] (YARN-1959) Fix headroom calculation in Fair Scheduler

Sandy Ryza created YARN-1959: Summary: Fix headroom calculation in Fair Scheduler Key: YARN-1959 URL: https://issues.apache.org/jira/browse/YARN-1959 Project: Hadoop YARN Issue Type: Bug

[jira] [Updated] (YARN-1959) Fix headroom calculation in Fair Scheduler

[ https://issues.apache.org/jira/browse/YARN-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1959: - Description: The Fair Scheduler currently always sets the headroom to 0. Fix headroom calculation in

[jira] [Commented] (YARN-1959) Fix headroom calculation in Fair Scheduler

[ https://issues.apache.org/jira/browse/YARN-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13973869#comment-13973869 ] Sandy Ryza commented on YARN-1959: -- The headroom for an app should be set to the min(app's

[jira] [Commented] (YARN-1959) Fix headroom calculation in Fair Scheduler

[ https://issues.apache.org/jira/browse/YARN-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13974456#comment-13974456 ] Sandy Ryza commented on YARN-1959: -- One thing I don't understand from reading the Capacity

[jira] [Commented] (YARN-1864) Fair Scheduler Dynamic Hierarchical User Queues

[ https://issues.apache.org/jira/browse/YARN-1864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13974497#comment-13974497 ] Sandy Ryza commented on YARN-1864: -- Thanks for working on this Ashwin. I like the

[jira] [Commented] (YARN-1959) Fix headroom calculation in Fair Scheduler

[ https://issues.apache.org/jira/browse/YARN-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13974541#comment-13974541 ] Sandy Ryza commented on YARN-1959: -- Ah, ok, thanks Jason. With your formula, assuming no

[jira] [Assigned] (YARN-1959) Fix headroom calculation in Fair Scheduler

[ https://issues.apache.org/jira/browse/YARN-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned YARN-1959: Assignee: Sandy Ryza Fix headroom calculation in Fair Scheduler

[jira] [Commented] (YARN-1959) Fix headroom calculation in Fair Scheduler

[ https://issues.apache.org/jira/browse/YARN-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13974557#comment-13974557 ] Sandy Ryza commented on YARN-1959: -- Cool, wanted to make sure I understood how it worked.

[jira] [Updated] (YARN-1941) Yarn scheduler ACL improvement

[ https://issues.apache.org/jira/browse/YARN-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1941: - Assignee: Gordon Wang Yarn scheduler ACL improvement --

[jira] [Commented] (YARN-1941) Yarn scheduler ACL improvement

[ https://issues.apache.org/jira/browse/YARN-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13972312#comment-13972312 ] Sandy Ryza commented on YARN-1941: -- bq. Currently, if user want to make ACL works

[jira] [Commented] (YARN-1941) Yarn scheduler ACL improvement

[ https://issues.apache.org/jira/browse/YARN-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969665#comment-13969665 ] Sandy Ryza commented on YARN-1941: -- Hi Gordon. (1) seems like a bug that we should

[jira] [Created] (YARN-1948) Expose utility methods in Apps.java publically

Sandy Ryza created YARN-1948: Summary: Expose utility methods in Apps.java publically Key: YARN-1948 URL: https://issues.apache.org/jira/browse/YARN-1948 Project: Hadoop YARN Issue Type:

[jira] [Commented] (YARN-1931) Private API change in YARN-1824 in 2.4 broke compatibility with previous releases

[ https://issues.apache.org/jira/browse/YARN-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13970223#comment-13970223 ] Sandy Ryza commented on YARN-1931: -- Filed YARN-1948 for providing public versions of the

[jira] [Updated] (YARN-1931) Private API change in YARN-1824 in 2.4 broke compatibility with previous releases

[ https://issues.apache.org/jira/browse/YARN-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1931: - Attachment: YARN-1931-2.patch Updated patch adds comments to clarify what's going on. Private API

[jira] [Commented] (YARN-1931) YARN-1824 in 2.4 broke compatibility with previous releases

[ https://issues.apache.org/jira/browse/YARN-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13966856#comment-13966856 ] Sandy Ryza commented on YARN-1931: -- +1 to adding the old API back in. The old API isn't

[jira] [Updated] (YARN-1931) YARN-1824 in 2.4 broke compatibility with previous releases

[ https://issues.apache.org/jira/browse/YARN-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1931: - Attachment: YARN-1931.patch YARN-1824 in 2.4 broke compatibility with previous releases

[jira] [Commented] (YARN-1931) YARN-1824 in 2.4 broke compatibility with previous releases

[ https://issues.apache.org/jira/browse/YARN-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13966898#comment-13966898 ] Sandy Ryza commented on YARN-1931: -- Attached a patch that adds back the old method as

[jira] [Updated] (YARN-1931) YARN-1824 in 2.4 broke compatibility with previous releases

[ https://issues.apache.org/jira/browse/YARN-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1931: - Attachment: YARN-1931-1.patch YARN-1824 in 2.4 broke compatibility with previous releases

[jira] [Commented] (YARN-1931) YARN-1824 in 2.4 broke compatibility with previous releases

[ https://issues.apache.org/jira/browse/YARN-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13966932#comment-13966932 ] Sandy Ryza commented on YARN-1931: -- Oops, updated patch includes setEnvFromInputString as

[jira] [Commented] (YARN-1931) Private API change in YARN-1824 in 2.4 broke compatibility with previous releases

[ https://issues.apache.org/jira/browse/YARN-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13967224#comment-13967224 ] Sandy Ryza commented on YARN-1931: -- bq. It'll be great to also put a class level javadoc

<    1   2   3   4   5   6   7   8   9   10   >