[jira] [Updated] (YARN-10139) ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB

2020-05-22 Thread Prabhu Joseph (Jira) 


 [ 
https://issues.apache.org/jira/browse/YARN-10139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prabhu Joseph updated YARN-10139:
-
Parent: YARN-5734
Issue Type: Sub-task  (was: Bug)

> ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > 
> default 8GB
> 
>
> Key: YARN-10139
> URL: https://issues.apache.org/jira/browse/YARN-10139
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Fix For: 3.3.0, 3.2.2, 3.1.4
>
> Attachments: YARN-10139-001.patch, YARN-10139-002.patch
>
>
> ValidateAndGetSchedulerConfiguration fails when the cluster max allocation 
> (yarn.scheduler.maximum-allocation-mb) is set to resource (eg: 16GB) > 
> default 8GB in yarn-site.xml.
> As part of validation API, there are two configuration used - 
> CapacitySchedulerConfiguration and Configuration (yarn-site.xml). When 
> CapacityScheduler is initialized with CapacitySchedulerConfiguration, as part 
> of queues initialization, it checks the queue maximum allocation which is not 
> present and so checks cluster max allocation which is not present (it is 
> present only in YarnConfiguration) and defaults to 8GB. This will fail as 
> queue max allocation 8GB is decreased from previous 16GB.
> {code}
> 2020-02-14 07:38:46,087 WARN 
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices: 
> CapacityScheduler configuration validation failed:java.io.IOException: Failed 
> to re-init queues : Trying to reinitialize root.default.c1.c3 the maximum 
> allocation size can not be decreased! Current setting:  vCores:88>, trying to set it to: 
> {code}
> CapacityScheduler initialize code reads a yarn config from 
> CapacitySchedulerConfiguration causing the issue. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-10139) ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB

2020-02-17 Thread Prabhu Joseph (Jira) 


 [ 
https://issues.apache.org/jira/browse/YARN-10139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prabhu Joseph updated YARN-10139:
-
Attachment: YARN-10139-002.patch

> ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > 
> default 8GB
> 
>
> Key: YARN-10139
> URL: https://issues.apache.org/jira/browse/YARN-10139
> Project: Hadoop YARN
>  Issue Type: Bug
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-10139-001.patch, YARN-10139-002.patch
>
>
> ValidateAndGetSchedulerConfiguration fails when the cluster max allocation 
> (yarn.scheduler.maximum-allocation-mb) is set to resource (eg: 16GB) > 
> default 8GB in yarn-site.xml.
> As part of validation API, there are two configuration used - 
> CapacitySchedulerConfiguration and Configuration (yarn-site.xml). When 
> CapacityScheduler is initialized with CapacitySchedulerConfiguration, as part 
> of queues initialization, it checks the queue maximum allocation which is not 
> present and so checks cluster max allocation which is not present (it is 
> present only in YarnConfiguration) and defaults to 8GB. This will fail as 
> queue max allocation 8GB is decreased from previous 16GB.
> {code}
> 2020-02-14 07:38:46,087 WARN 
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices: 
> CapacityScheduler configuration validation failed:java.io.IOException: Failed 
> to re-init queues : Trying to reinitialize root.default.c1.c3 the maximum 
> allocation size can not be decreased! Current setting:  vCores:88>, trying to set it to: 
> {code}
> CapacityScheduler initialize code reads a yarn config from 
> CapacitySchedulerConfiguration causing the issue. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-10139) ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB

2020-02-17 Thread Prabhu Joseph (Jira) 


 [ 
https://issues.apache.org/jira/browse/YARN-10139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prabhu Joseph updated YARN-10139:
-
Attachment: (was: YARN-10139-002.patch)

> ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > 
> default 8GB
> 
>
> Key: YARN-10139
> URL: https://issues.apache.org/jira/browse/YARN-10139
> Project: Hadoop YARN
>  Issue Type: Bug
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-10139-001.patch
>
>
> ValidateAndGetSchedulerConfiguration fails when the cluster max allocation 
> (yarn.scheduler.maximum-allocation-mb) is set to resource (eg: 16GB) > 
> default 8GB in yarn-site.xml.
> As part of validation API, there are two configuration used - 
> CapacitySchedulerConfiguration and Configuration (yarn-site.xml). When 
> CapacityScheduler is initialized with CapacitySchedulerConfiguration, as part 
> of queues initialization, it checks the queue maximum allocation which is not 
> present and so checks cluster max allocation which is not present (it is 
> present only in YarnConfiguration) and defaults to 8GB. This will fail as 
> queue max allocation 8GB is decreased from previous 16GB.
> {code}
> 2020-02-14 07:38:46,087 WARN 
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices: 
> CapacityScheduler configuration validation failed:java.io.IOException: Failed 
> to re-init queues : Trying to reinitialize root.default.c1.c3 the maximum 
> allocation size can not be decreased! Current setting:  vCores:88>, trying to set it to: 
> {code}
> CapacityScheduler initialize code reads a yarn config from 
> CapacitySchedulerConfiguration causing the issue. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-10139) ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB

2020-02-17 Thread Prabhu Joseph (Jira) 


 [ 
https://issues.apache.org/jira/browse/YARN-10139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prabhu Joseph updated YARN-10139:
-
Attachment: YARN-10139-002.patch

> ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > 
> default 8GB
> 
>
> Key: YARN-10139
> URL: https://issues.apache.org/jira/browse/YARN-10139
> Project: Hadoop YARN
>  Issue Type: Bug
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-10139-001.patch, YARN-10139-002.patch
>
>
> ValidateAndGetSchedulerConfiguration fails when the cluster max allocation 
> (yarn.scheduler.maximum-allocation-mb) is set to resource (eg: 16GB) > 
> default 8GB in yarn-site.xml.
> As part of validation API, there are two configuration used - 
> CapacitySchedulerConfiguration and Configuration (yarn-site.xml). When 
> CapacityScheduler is initialized with CapacitySchedulerConfiguration, as part 
> of queues initialization, it checks the queue maximum allocation which is not 
> present and so checks cluster max allocation which is not present (it is 
> present only in YarnConfiguration) and defaults to 8GB. This will fail as 
> queue max allocation 8GB is decreased from previous 16GB.
> {code}
> 2020-02-14 07:38:46,087 WARN 
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices: 
> CapacityScheduler configuration validation failed:java.io.IOException: Failed 
> to re-init queues : Trying to reinitialize root.default.c1.c3 the maximum 
> allocation size can not be decreased! Current setting:  vCores:88>, trying to set it to: 
> {code}
> CapacityScheduler initialize code reads a yarn config from 
> CapacitySchedulerConfiguration causing the issue. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-10139) ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB

2020-02-14 Thread Prabhu Joseph (Jira) 


 [ 
https://issues.apache.org/jira/browse/YARN-10139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prabhu Joseph updated YARN-10139:
-
Attachment: YARN-10139-001.patch

> ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > 
> default 8GB
> 
>
> Key: YARN-10139
> URL: https://issues.apache.org/jira/browse/YARN-10139
> Project: Hadoop YARN
>  Issue Type: Bug
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-10139-001.patch
>
>
> ValidateAndGetSchedulerConfiguration fails when the cluster max allocation 
> (yarn.scheduler.maximum-allocation-mb) is set to resource (eg: 16GB) > 
> default 8GB in yarn-site.xml.
> As part of validation API, there are two configuration used - 
> CapacitySchedulerConfiguration and Configuration (yarn-site.xml). When 
> CapacityScheduler is initialized with CapacitySchedulerConfiguration, as part 
> of queues initialization, it checks the queue maximum allocation which is not 
> present and so checks cluster max allocation which is not present (it is 
> present only in YarnConfiguration) and defaults to 8GB. This will fail as 
> queue max allocation 8GB is decreased from previous 16GB.
> {code}
> 2020-02-14 07:38:46,087 WARN 
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices: 
> CapacityScheduler configuration validation failed:java.io.IOException: Failed 
> to re-init queues : Trying to reinitialize root.default.c1.c3 the maximum 
> allocation size can not be decreased! Current setting:  vCores:88>, trying to set it to: 
> {code}
> CapacityScheduler initialize code reads a yarn config from 
> CapacitySchedulerConfiguration causing the issue. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org