[jira] [Updated] (YARN-10622) Fix preemption policy to exclude childless ParentQueues
[
https://issues.apache.org/jira/browse/YARN-10622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shilun Fan updated YARN-10622:
--
Component/s: capacity scheduler
Target Version/s: 3.4.0
Affects Version/s: 3.4.0
> Fix preemption policy to exclude childless ParentQueues
> ---
>
> Key: YARN-10622
> URL: https://issues.apache.org/jira/browse/YARN-10622
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
>Affects Versions: 3.4.0
>Reporter: Andras Gyori
>Assignee: Andras Gyori
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: YARN-10622.001.patch
>
>
> ProportionalCapacityPreemptionPolicy selects the potential LeafQueues to be
> preempted by this logic:
> {code:java}
> private Set getLeafQueueNames(TempQueuePerPartition q) {
> // If its a ManagedParentQueue, it might not have any children
> if ((q.children == null || q.children.isEmpty())
> && !(q.parentQueue instanceof ManagedParentQueue)) {
> return ImmutableSet.of(q.queueName);
> }
> Set leafQueueNames = new HashSet<>();
> for (TempQueuePerPartition child : q.children) {
> leafQueueNames.addAll(getLeafQueueNames(child));
> }
> return leafQueueNames;
> }
> {code}
> This, however does not take childless ParentQueues (which was introduced in
> YARN-10596) into account.
> A childless ParentQueue will throw a NPE in
> FifoCandidatesSelector#selectCandidates:
> {code:java}
> LeafQueue leafQueue = preemptionContext.getQueueByPartition(queueName,
> RMNodeLabelsManager.NO_LABEL).leafQueue;
> {code}
> TempQueuePerPartition has a leafQueue member variable, which is null, if the
> queue is not a LeafQueue. In case of childless ParentQueue, it is null, but
> its name is present in the leafQueueNames as stated before.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10622) Fix preemption policy to exclude childless ParentQueues
[
https://issues.apache.org/jira/browse/YARN-10622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szilard Nemeth updated YARN-10622:
--
Fix Version/s: 3.4.0
> Fix preemption policy to exclude childless ParentQueues
> ---
>
> Key: YARN-10622
> URL: https://issues.apache.org/jira/browse/YARN-10622
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Andras Gyori
>Assignee: Andras Gyori
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: YARN-10622.001.patch
>
>
> ProportionalCapacityPreemptionPolicy selects the potential LeafQueues to be
> preempted by this logic:
> {code:java}
> private Set getLeafQueueNames(TempQueuePerPartition q) {
> // If its a ManagedParentQueue, it might not have any children
> if ((q.children == null || q.children.isEmpty())
> && !(q.parentQueue instanceof ManagedParentQueue)) {
> return ImmutableSet.of(q.queueName);
> }
> Set leafQueueNames = new HashSet<>();
> for (TempQueuePerPartition child : q.children) {
> leafQueueNames.addAll(getLeafQueueNames(child));
> }
> return leafQueueNames;
> }
> {code}
> This, however does not take childless ParentQueues (which was introduced in
> YARN-10596) into account.
> A childless ParentQueue will throw a NPE in
> FifoCandidatesSelector#selectCandidates:
> {code:java}
> LeafQueue leafQueue = preemptionContext.getQueueByPartition(queueName,
> RMNodeLabelsManager.NO_LABEL).leafQueue;
> {code}
> TempQueuePerPartition has a leafQueue member variable, which is null, if the
> queue is not a LeafQueue. In case of childless ParentQueue, it is null, but
> its name is present in the leafQueueNames as stated before.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10622) Fix preemption policy to exclude childless ParentQueues
[
https://issues.apache.org/jira/browse/YARN-10622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Gyori updated YARN-10622:
Attachment: YARN-10622.001.patch
> Fix preemption policy to exclude childless ParentQueues
> ---
>
> Key: YARN-10622
> URL: https://issues.apache.org/jira/browse/YARN-10622
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Andras Gyori
>Assignee: Andras Gyori
>Priority: Major
> Attachments: YARN-10622.001.patch
>
>
> ProportionalCapacityPreemptionPolicy selects the potential LeafQueues to be
> preempted by this logic:
> {code:java}
> private Set getLeafQueueNames(TempQueuePerPartition q) {
> // If its a ManagedParentQueue, it might not have any children
> if ((q.children == null || q.children.isEmpty())
> && !(q.parentQueue instanceof ManagedParentQueue)) {
> return ImmutableSet.of(q.queueName);
> }
> Set leafQueueNames = new HashSet<>();
> for (TempQueuePerPartition child : q.children) {
> leafQueueNames.addAll(getLeafQueueNames(child));
> }
> return leafQueueNames;
> }
> {code}
> This, however does not take childless ParentQueues (which was introduced in
> YARN-10596) into account.
> A childless ParentQueue will throw a NPE in
> FifoCandidatesSelector#selectCandidates:
> {code:java}
> LeafQueue leafQueue = preemptionContext.getQueueByPartition(queueName,
> RMNodeLabelsManager.NO_LABEL).leafQueue;
> {code}
> TempQueuePerPartition has a leafQueue member variable, which is null, if the
> queue is not a LeafQueue. In case of childless ParentQueue, it is null, but
> its name is present in the leafQueueNames as stated before.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10622) Fix preemption policy to exclude childless ParentQueues
[
https://issues.apache.org/jira/browse/YARN-10622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Gyori updated YARN-10622:
Attachment: YARN-10622.001.patch
> Fix preemption policy to exclude childless ParentQueues
> ---
>
> Key: YARN-10622
> URL: https://issues.apache.org/jira/browse/YARN-10622
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Andras Gyori
>Assignee: Andras Gyori
>Priority: Major
> Attachments: YARN-10622.001.patch
>
>
> ProportionalCapacityPreemptionPolicy selects the potential LeafQueues to be
> preempted by this logic:
> {code:java}
> private Set getLeafQueueNames(TempQueuePerPartition q) {
> // If its a ManagedParentQueue, it might not have any children
> if ((q.children == null || q.children.isEmpty())
> && !(q.parentQueue instanceof ManagedParentQueue)) {
> return ImmutableSet.of(q.queueName);
> }
> Set leafQueueNames = new HashSet<>();
> for (TempQueuePerPartition child : q.children) {
> leafQueueNames.addAll(getLeafQueueNames(child));
> }
> return leafQueueNames;
> }
> {code}
> This, however does not take childless ParentQueues (which was introduced in
> YARN-10596) into account.
> A childless ParentQueue will throw a NPE in
> FifoCandidatesSelector#selectCandidates:
> {code:java}
> LeafQueue leafQueue = preemptionContext.getQueueByPartition(queueName,
> RMNodeLabelsManager.NO_LABEL).leafQueue;
> {code}
> TempQueuePerPartition has a leafQueue member variable, which is null, if the
> queue is not a LeafQueue. In case of childless ParentQueue, it is null, but
> its name is present in the leafQueueNames as stated before.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10622) Fix preemption policy to exclude childless ParentQueues
[
https://issues.apache.org/jira/browse/YARN-10622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Gyori updated YARN-10622:
Attachment: (was: YARN-10622.001.patch)
> Fix preemption policy to exclude childless ParentQueues
> ---
>
> Key: YARN-10622
> URL: https://issues.apache.org/jira/browse/YARN-10622
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Andras Gyori
>Assignee: Andras Gyori
>Priority: Major
> Attachments: YARN-10622.001.patch
>
>
> ProportionalCapacityPreemptionPolicy selects the potential LeafQueues to be
> preempted by this logic:
> {code:java}
> private Set getLeafQueueNames(TempQueuePerPartition q) {
> // If its a ManagedParentQueue, it might not have any children
> if ((q.children == null || q.children.isEmpty())
> && !(q.parentQueue instanceof ManagedParentQueue)) {
> return ImmutableSet.of(q.queueName);
> }
> Set leafQueueNames = new HashSet<>();
> for (TempQueuePerPartition child : q.children) {
> leafQueueNames.addAll(getLeafQueueNames(child));
> }
> return leafQueueNames;
> }
> {code}
> This, however does not take childless ParentQueues (which was introduced in
> YARN-10596) into account.
> A childless ParentQueue will throw a NPE in
> FifoCandidatesSelector#selectCandidates:
> {code:java}
> LeafQueue leafQueue = preemptionContext.getQueueByPartition(queueName,
> RMNodeLabelsManager.NO_LABEL).leafQueue;
> {code}
> TempQueuePerPartition has a leafQueue member variable, which is null, if the
> queue is not a LeafQueue. In case of childless ParentQueue, it is null, but
> its name is present in the leafQueueNames as stated before.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10622) Fix preemption policy to exclude childless ParentQueues
[
https://issues.apache.org/jira/browse/YARN-10622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Gyori updated YARN-10622:
Attachment: (was: YARN-10622.001.patch)
> Fix preemption policy to exclude childless ParentQueues
> ---
>
> Key: YARN-10622
> URL: https://issues.apache.org/jira/browse/YARN-10622
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Andras Gyori
>Assignee: Andras Gyori
>Priority: Major
> Attachments: YARN-10622.001.patch
>
>
> ProportionalCapacityPreemptionPolicy selects the potential LeafQueues to be
> preempted by this logic:
> {code:java}
> private Set getLeafQueueNames(TempQueuePerPartition q) {
> // If its a ManagedParentQueue, it might not have any children
> if ((q.children == null || q.children.isEmpty())
> && !(q.parentQueue instanceof ManagedParentQueue)) {
> return ImmutableSet.of(q.queueName);
> }
> Set leafQueueNames = new HashSet<>();
> for (TempQueuePerPartition child : q.children) {
> leafQueueNames.addAll(getLeafQueueNames(child));
> }
> return leafQueueNames;
> }
> {code}
> This, however does not take childless ParentQueues (which was introduced in
> YARN-10596) into account.
> A childless ParentQueue will throw a NPE in
> FifoCandidatesSelector#selectCandidates:
> {code:java}
> LeafQueue leafQueue = preemptionContext.getQueueByPartition(queueName,
> RMNodeLabelsManager.NO_LABEL).leafQueue;
> {code}
> TempQueuePerPartition has a leafQueue member variable, which is null, if the
> queue is not a LeafQueue. In case of childless ParentQueue, it is null, but
> its name is present in the leafQueueNames as stated before.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10622) Fix preemption policy to exclude childless ParentQueues
[
https://issues.apache.org/jira/browse/YARN-10622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Gyori updated YARN-10622:
Attachment: YARN-10622.001.patch
> Fix preemption policy to exclude childless ParentQueues
> ---
>
> Key: YARN-10622
> URL: https://issues.apache.org/jira/browse/YARN-10622
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Andras Gyori
>Assignee: Andras Gyori
>Priority: Major
> Attachments: YARN-10622.001.patch
>
>
> ProportionalCapacityPreemptionPolicy selects the potential LeafQueues to be
> preempted by this logic:
> {code:java}
> private Set getLeafQueueNames(TempQueuePerPartition q) {
> // If its a ManagedParentQueue, it might not have any children
> if ((q.children == null || q.children.isEmpty())
> && !(q.parentQueue instanceof ManagedParentQueue)) {
> return ImmutableSet.of(q.queueName);
> }
> Set leafQueueNames = new HashSet<>();
> for (TempQueuePerPartition child : q.children) {
> leafQueueNames.addAll(getLeafQueueNames(child));
> }
> return leafQueueNames;
> }
> {code}
> This, however does not take childless ParentQueues (which was introduced in
> YARN-10596) into account.
> A childless ParentQueue will throw a NPE in
> FifoCandidatesSelector#selectCandidates:
> {code:java}
> LeafQueue leafQueue = preemptionContext.getQueueByPartition(queueName,
> RMNodeLabelsManager.NO_LABEL).leafQueue;
> {code}
> TempQueuePerPartition has a leafQueue member variable, which is null, if the
> queue is not a LeafQueue. In case of childless ParentQueue, it is null, but
> its name is present in the leafQueueNames as stated before.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10622) Fix preemption policy to exclude childless ParentQueues
[
https://issues.apache.org/jira/browse/YARN-10622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Gyori updated YARN-10622:
Attachment: (was: YARN-10622.001.patch)
> Fix preemption policy to exclude childless ParentQueues
> ---
>
> Key: YARN-10622
> URL: https://issues.apache.org/jira/browse/YARN-10622
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Andras Gyori
>Assignee: Andras Gyori
>Priority: Major
> Attachments: YARN-10622.001.patch
>
>
> ProportionalCapacityPreemptionPolicy selects the potential LeafQueues to be
> preempted by this logic:
> {code:java}
> private Set getLeafQueueNames(TempQueuePerPartition q) {
> // If its a ManagedParentQueue, it might not have any children
> if ((q.children == null || q.children.isEmpty())
> && !(q.parentQueue instanceof ManagedParentQueue)) {
> return ImmutableSet.of(q.queueName);
> }
> Set leafQueueNames = new HashSet<>();
> for (TempQueuePerPartition child : q.children) {
> leafQueueNames.addAll(getLeafQueueNames(child));
> }
> return leafQueueNames;
> }
> {code}
> This, however does not take childless ParentQueues (which was introduced in
> YARN-10596) into account.
> A childless ParentQueue will throw a NPE in
> FifoCandidatesSelector#selectCandidates:
> {code:java}
> LeafQueue leafQueue = preemptionContext.getQueueByPartition(queueName,
> RMNodeLabelsManager.NO_LABEL).leafQueue;
> {code}
> TempQueuePerPartition has a leafQueue member variable, which is null, if the
> queue is not a LeafQueue. In case of childless ParentQueue, it is null, but
> its name is present in the leafQueueNames as stated before.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10622) Fix preemption policy to exclude childless ParentQueues
[
https://issues.apache.org/jira/browse/YARN-10622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Gyori updated YARN-10622:
Attachment: YARN-10622.001.patch
> Fix preemption policy to exclude childless ParentQueues
> ---
>
> Key: YARN-10622
> URL: https://issues.apache.org/jira/browse/YARN-10622
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Andras Gyori
>Assignee: Andras Gyori
>Priority: Major
> Attachments: YARN-10622.001.patch
>
>
> ProportionalCapacityPreemptionPolicy selects the potential LeafQueues to be
> preempted by this logic:
> {code:java}
> private Set getLeafQueueNames(TempQueuePerPartition q) {
> // If its a ManagedParentQueue, it might not have any children
> if ((q.children == null || q.children.isEmpty())
> && !(q.parentQueue instanceof ManagedParentQueue)) {
> return ImmutableSet.of(q.queueName);
> }
> Set leafQueueNames = new HashSet<>();
> for (TempQueuePerPartition child : q.children) {
> leafQueueNames.addAll(getLeafQueueNames(child));
> }
> return leafQueueNames;
> }
> {code}
> This, however does not take childless ParentQueues (which was introduced in
> YARN-10596) into account.
> A childless ParentQueue will throw a NPE in
> FifoCandidatesSelector#selectCandidates:
> {code:java}
> LeafQueue leafQueue = preemptionContext.getQueueByPartition(queueName,
> RMNodeLabelsManager.NO_LABEL).leafQueue;
> {code}
> TempQueuePerPartition has a leafQueue member variable, which is null, if the
> queue is not a LeafQueue. In case of childless ParentQueue, it is null, but
> its name is present in the leafQueueNames as stated before.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10622) Fix preemption policy to exclude childless ParentQueues
[
https://issues.apache.org/jira/browse/YARN-10622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Gyori updated YARN-10622:
Summary: Fix preemption policy to exclude childless ParentQueues (was: Fix
preemption policy to exclude childless ParentQueus)
> Fix preemption policy to exclude childless ParentQueues
> ---
>
> Key: YARN-10622
> URL: https://issues.apache.org/jira/browse/YARN-10622
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Andras Gyori
>Assignee: Andras Gyori
>Priority: Major
>
> ProportionalCapacityPreemptionPolicy selects the potential LeafQueues to be
> preempted by this logic:
> {code:java}
> private Set getLeafQueueNames(TempQueuePerPartition q) {
> // If its a ManagedParentQueue, it might not have any children
> if ((q.children == null || q.children.isEmpty())
> && !(q.parentQueue instanceof ManagedParentQueue)) {
> return ImmutableSet.of(q.queueName);
> }
> Set leafQueueNames = new HashSet<>();
> for (TempQueuePerPartition child : q.children) {
> leafQueueNames.addAll(getLeafQueueNames(child));
> }
> return leafQueueNames;
> }
> {code}
> This, however does not take childless ParentQueues (which was introduced in
> YARN-10596) into account.
> A childless ParentQueue will throw a NPE in
> FifoCandidatesSelector#selectCandidates:
> {code:java}
> LeafQueue leafQueue = preemptionContext.getQueueByPartition(queueName,
> RMNodeLabelsManager.NO_LABEL).leafQueue;
> {code}
> TempQueuePerPartition has a leafQueue member variable, which is null, if the
> queue is not a LeafQueue. In case of childless ParentQueue, it is null, but
> its name is present in the leafQueueNames as stated before.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
