[ https://issues.apache.org/jira/browse/YARN-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15976956#comment-15976956 ]
Carlo Curino commented on YARN-2113: ------------------------------------ [~wangda] I lack context here, as I come in late in the conversation. I don't understand why simple dead-zones cannot avoid oscillations. Even if you change the "target values" afterwards. This is equivalent of changing your thermostat target temperature, something like an hysteresis (somewhat akin to a dead-zone) typically takes care of it. Regarding the idea of a directed graph, it seems tricky because you would have to maintain some form of windowing semantics as well, since not all cycles appear instantaneously, but also over time some preemption cycles will eventually happen (and if far apart enough they are not a problem, just naturally changing cluster conditions). Tuning this windowing semantics seems very tricky, and I am not sure I even get what are the precise semantics it enforces. Overall I would stick to the simplest solution possible. More generally, I would ask you ([~sunilg]?) to provide very crisp description of the semantics that you target, and most importantly a crisp explanation of how this plays with the rest of the scheduler features (e.g., node-labels / non-preemptible queues / delay-scheduling). It is ok to say feature A doesn't work with feature B, because they break each-other invariants, but this should be captured clearly. I suggest to look at YARN-6451, and you use those ideas plus SLS to build tests that ensure that whatever invariant you have in mind is enforced and maintained by the implementation. > Add cross-user preemption within CapacityScheduler's leaf-queue > --------------------------------------------------------------- > > Key: YARN-2113 > URL: https://issues.apache.org/jira/browse/YARN-2113 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler > Reporter: Vinod Kumar Vavilapalli > Assignee: Sunil G > Attachments: > TestNoIntraQueuePreemptionIfBelowUserLimitAndDifferentPrioritiesWithExtraUsers.txt, > YARN-2113.0001.patch, YARN-2113.0002.patch, YARN-2113.0003.patch, > YARN-2113.0004.patch, YARN-2113.0005.patch, YARN-2113.0006.patch, > YARN-2113.0007.patch, YARN-2113.v0.patch > > > Preemption today only works across queues and moves around resources across > queues per demand and usage. We should also have user-level preemption within > a queue, to balance capacity across users in a predictable manner. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org