[jira] [Resolved] (YARN-6279) Scheduler rest api JSON is not providing all child queues names
[
https://issues.apache.org/jira/browse/YARN-6279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Doneriya resolved YARN-6279.
---
Fix Version/s: 2.8.0
3.0.0-alpha1
Release Note: Marking issue as resolved as the original issue (YARN-2336)
has been resolved
Resolution: Fixed
> Scheduler rest api JSON is not providing all child queues names
> ---
>
> Key: YARN-6279
> URL: https://issues.apache.org/jira/browse/YARN-6279
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api, fairscheduler, scheduler
>Affects Versions: 2.4.1
> Environment: Ubuntu 14.04, 7.7 GiB, i5, 3.4GHz x 4, 64-bit
>Reporter: Ashish Doneriya
>Priority: Major
> Fix For: 3.0.0-alpha1, 2.8.0
>
>
> When I hit rest api /ws/v1/cluster/scheduler to get the JSON file. Its gave
> me all child queues information, But it didn't gave me all information about
> child queues of child queues. It displays information of only one sub child
> queue. While in xml format there is no such problem.
> I'm providing the xml and json outputs..
>
> {"scheduler":{"schedulerInfo":{"type":"fairScheduler","rootQueue":{"maxApps":2147483647,"minResources":{"memory":0,"vCores":0},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":8192,"vCores":8},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root","schedulingPolicy":"fair","childQueues":[{"maxApps":20,"minResources":{"memory":1024,"vCores":1},"maxResources":{"memory":5283,"vCores":2},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":5283,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.Engineering","schedulingPolicy":"fair","childQueues":{"type":["fairSchedulerLeafQueueInfo"],"maxApps":2147483647,"minResources":{"memory":1024,"vCores":1},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":2642,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.Engineering.Development","schedulingPolicy":"fair","numPendingApps":0,"numActiveApps":0},"childQueues":{"type":"fairSchedulerLeafQueueInfo","maxApps":2147483647,"minResources":{"memory":1024,"vCores":1},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":2642,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.Engineering.TESTING","schedulingPolicy":"fair","numPendingApps":0,"numActiveApps":0}},{"type":"fairSchedulerLeafQueueInfo","maxApps":2147483647,"minResources":{"memory":0,"vCores":0},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":2909,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.default","schedulingPolicy":"fair","numPendingApps":0,"numActiveApps":0}]
>
>
> http://www.w3.org/2001/XMLSchema-instance;
> xsi:type="fairScheduler">
>
> 2147483647
>
> 0
> 0
>
>
> 8192
> 8
>
>
> 0
> 0
>
>
> 8192
> 8
>
>
> 8192
> 8
>
> root
> fair
>
> 20
>
> 1024
> 1
>
>
> 5283
> 2
>
>
> 0
> 0
>
>
> 5283
> 0
>
>
> 8192
> 8
>
> root.Engineering
> fair
>xsi:type="fairSchedulerLeafQueueInfo">
> 2147483647
>
> 1024
>
[jira] [Commented] (YARN-6279) Scheduler rest api JSON is not providing all child queues names
[
https://issues.apache.org/jira/browse/YARN-6279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929662#comment-15929662
]
Ashish Doneriya commented on YARN-6279:
---
Sorry Guys, It seems like there was already a bug filed
https://issues.apache.org/jira/browse/YARN-2336. In the json file there are
duplicate keys ('childQueues') 'root.Engineering' that should not be
childQueues : [
"queueName":"root.Engineering",
"childQueues":{
..
"queueName":"root.Engineering.Development"
..
},
"childQueues":{
..
"queueName":"root.Engineering.Testing"
..
}
]
I tested this on apache 2.4.1 and cloudera 5.9
> Scheduler rest api JSON is not providing all child queues names
> ---
>
> Key: YARN-6279
> URL: https://issues.apache.org/jira/browse/YARN-6279
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api, fairscheduler, scheduler
>Affects Versions: 2.4.1
> Environment: Ubuntu 14.04, 7.7 GiB, i5, 3.4GHz x 4, 64-bit
>Reporter: Ashish Doneriya
>
> When I hit rest api /ws/v1/cluster/scheduler to get the JSON file. Its gave
> me all child queues information, But it didn't gave me all information about
> child queues of child queues. It displays information of only one sub child
> queue. While in xml format there is no such problem.
> I'm providing the xml and json outputs..
>
> {"scheduler":{"schedulerInfo":{"type":"fairScheduler","rootQueue":{"maxApps":2147483647,"minResources":{"memory":0,"vCores":0},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":8192,"vCores":8},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root","schedulingPolicy":"fair","childQueues":[{"maxApps":20,"minResources":{"memory":1024,"vCores":1},"maxResources":{"memory":5283,"vCores":2},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":5283,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.Engineering","schedulingPolicy":"fair","childQueues":{"type":["fairSchedulerLeafQueueInfo"],"maxApps":2147483647,"minResources":{"memory":1024,"vCores":1},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":2642,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.Engineering.Development","schedulingPolicy":"fair","numPendingApps":0,"numActiveApps":0},"childQueues":{"type":"fairSchedulerLeafQueueInfo","maxApps":2147483647,"minResources":{"memory":1024,"vCores":1},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":2642,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.Engineering.TESTING","schedulingPolicy":"fair","numPendingApps":0,"numActiveApps":0}},{"type":"fairSchedulerLeafQueueInfo","maxApps":2147483647,"minResources":{"memory":0,"vCores":0},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":2909,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.default","schedulingPolicy":"fair","numPendingApps":0,"numActiveApps":0}]
>
>
> http://www.w3.org/2001/XMLSchema-instance;
> xsi:type="fairScheduler">
>
> 2147483647
>
> 0
> 0
>
>
> 8192
> 8
>
>
> 0
> 0
>
>
> 8192
> 8
>
>
> 8192
> 8
>
> root
> fair
>
> 20
>
> 1024
> 1
>
>
> 5283
> 2
>
>
> 0
> 0
>
>
> 5283
> 0
>
>
> 8192
> 8
>
> root.Engineering
>
[jira] [Issue Comment Deleted] (YARN-6279) Scheduler rest api JSON is not providing all child queues names
[
https://issues.apache.org/jira/browse/YARN-6279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Doneriya updated YARN-6279:
--
Comment: was deleted
(was: There are two child queues of 'root.Engineering' that are
'root.Engineering.Development' and 'root.Engineering.TESTING'. XML version is
showing both these queues but json version is showing only one
'root.Engineering.TESTING'.)
> Scheduler rest api JSON is not providing all child queues names
> ---
>
> Key: YARN-6279
> URL: https://issues.apache.org/jira/browse/YARN-6279
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api, fairscheduler, scheduler
>Affects Versions: 2.4.1
> Environment: Ubuntu 14.04, 7.7 GiB, i5, 3.4GHz x 4, 64-bit
>Reporter: Ashish Doneriya
>
> When I hit rest api /ws/v1/cluster/scheduler to get the JSON file. Its gave
> me all child queues information, But it didn't gave me all information about
> child queues of child queues. It displays information of only one sub child
> queue. While in xml format there is no such problem.
> I'm providing the xml and json outputs..
>
> {"scheduler":{"schedulerInfo":{"type":"fairScheduler","rootQueue":{"maxApps":2147483647,"minResources":{"memory":0,"vCores":0},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":8192,"vCores":8},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root","schedulingPolicy":"fair","childQueues":[{"maxApps":20,"minResources":{"memory":1024,"vCores":1},"maxResources":{"memory":5283,"vCores":2},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":5283,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.Engineering","schedulingPolicy":"fair","childQueues":{"type":["fairSchedulerLeafQueueInfo"],"maxApps":2147483647,"minResources":{"memory":1024,"vCores":1},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":2642,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.Engineering.Development","schedulingPolicy":"fair","numPendingApps":0,"numActiveApps":0},"childQueues":{"type":"fairSchedulerLeafQueueInfo","maxApps":2147483647,"minResources":{"memory":1024,"vCores":1},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":2642,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.Engineering.TESTING","schedulingPolicy":"fair","numPendingApps":0,"numActiveApps":0}},{"type":"fairSchedulerLeafQueueInfo","maxApps":2147483647,"minResources":{"memory":0,"vCores":0},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":2909,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.default","schedulingPolicy":"fair","numPendingApps":0,"numActiveApps":0}]
>
>
> http://www.w3.org/2001/XMLSchema-instance;
> xsi:type="fairScheduler">
>
> 2147483647
>
> 0
> 0
>
>
> 8192
> 8
>
>
> 0
> 0
>
>
> 8192
> 8
>
>
> 8192
> 8
>
> root
> fair
>
> 20
>
> 1024
> 1
>
>
> 5283
> 2
>
>
> 0
> 0
>
>
> 5283
> 0
>
>
> 8192
> 8
>
> root.Engineering
> fair
>xsi:type="fairSchedulerLeafQueueInfo">
> 2147483647
>
>
[jira] [Commented] (YARN-6279) Scheduler rest api JSON is not providing all child queues names
[
https://issues.apache.org/jira/browse/YARN-6279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929634#comment-15929634
]
Ashish Doneriya commented on YARN-6279:
---
There are two child queues of 'root.Engineering' that are
'root.Engineering.Development' and 'root.Engineering.TESTING'. XML version is
showing both these queues but json version is showing only one
'root.Engineering.TESTING'.
> Scheduler rest api JSON is not providing all child queues names
> ---
>
> Key: YARN-6279
> URL: https://issues.apache.org/jira/browse/YARN-6279
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api, fairscheduler, scheduler
>Affects Versions: 2.4.1
> Environment: Ubuntu 14.04, 7.7 GiB, i5, 3.4GHz x 4, 64-bit
>Reporter: Ashish Doneriya
>
> When I hit rest api /ws/v1/cluster/scheduler to get the JSON file. Its gave
> me all child queues information, But it didn't gave me all information about
> child queues of child queues. It displays information of only one sub child
> queue. While in xml format there is no such problem.
> I'm providing the xml and json outputs..
>
> {"scheduler":{"schedulerInfo":{"type":"fairScheduler","rootQueue":{"maxApps":2147483647,"minResources":{"memory":0,"vCores":0},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":8192,"vCores":8},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root","schedulingPolicy":"fair","childQueues":[{"maxApps":20,"minResources":{"memory":1024,"vCores":1},"maxResources":{"memory":5283,"vCores":2},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":5283,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.Engineering","schedulingPolicy":"fair","childQueues":{"type":["fairSchedulerLeafQueueInfo"],"maxApps":2147483647,"minResources":{"memory":1024,"vCores":1},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":2642,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.Engineering.Development","schedulingPolicy":"fair","numPendingApps":0,"numActiveApps":0},"childQueues":{"type":"fairSchedulerLeafQueueInfo","maxApps":2147483647,"minResources":{"memory":1024,"vCores":1},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":2642,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.Engineering.TESTING","schedulingPolicy":"fair","numPendingApps":0,"numActiveApps":0}},{"type":"fairSchedulerLeafQueueInfo","maxApps":2147483647,"minResources":{"memory":0,"vCores":0},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":2909,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.default","schedulingPolicy":"fair","numPendingApps":0,"numActiveApps":0}]
>
>
> http://www.w3.org/2001/XMLSchema-instance;
> xsi:type="fairScheduler">
>
> 2147483647
>
> 0
> 0
>
>
> 8192
> 8
>
>
> 0
> 0
>
>
> 8192
> 8
>
>
> 8192
> 8
>
> root
> fair
>
> 20
>
> 1024
> 1
>
>
> 5283
> 2
>
>
> 0
> 0
>
>
> 5283
> 0
>
>
> 8192
> 8
>
> root.Engineering
> fair
>xsi:type="fairSchedulerLeafQueueInfo">
> 2147483647
>
>
[jira] [Created] (YARN-6279) Scheduler rest api JSON is not providing all child queues names
Ashish Doneriya created YARN-6279:
-
Summary: Scheduler rest api JSON is not providing all child queues
names
Key: YARN-6279
URL: https://issues.apache.org/jira/browse/YARN-6279
Project: Hadoop YARN
Issue Type: Bug
Components: api, fairscheduler, scheduler
Affects Versions: 2.4.1
Environment: Ubuntu 14.04, 7.7 GiB, i5, 3.4GHz x 4, 64-bit
Reporter: Ashish Doneriya
When I hit rest api /ws/v1/cluster/scheduler to get the JSON file. Its gave me
all child queues information, But it didn't gave me all information about child
queues of child queues. It displays information of only one sub child queue.
While in xml format there is no such problem.
I'm providing the xml and json outputs..
{"scheduler":{"schedulerInfo":{"type":"fairScheduler","rootQueue":{"maxApps":2147483647,"minResources":{"memory":0,"vCores":0},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":8192,"vCores":8},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root","schedulingPolicy":"fair","childQueues":[{"maxApps":20,"minResources":{"memory":1024,"vCores":1},"maxResources":{"memory":5283,"vCores":2},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":5283,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.Engineering","schedulingPolicy":"fair","childQueues":{"type":["fairSchedulerLeafQueueInfo"],"maxApps":2147483647,"minResources":{"memory":1024,"vCores":1},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":2642,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.Engineering.Development","schedulingPolicy":"fair","numPendingApps":0,"numActiveApps":0},"childQueues":{"type":"fairSchedulerLeafQueueInfo","maxApps":2147483647,"minResources":{"memory":1024,"vCores":1},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":2642,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.Engineering.TESTING","schedulingPolicy":"fair","numPendingApps":0,"numActiveApps":0}},{"type":"fairSchedulerLeafQueueInfo","maxApps":2147483647,"minResources":{"memory":0,"vCores":0},"maxResources":{"memory":8192,"vCores":8},"usedResources":{"memory":0,"vCores":0},"fairResources":{"memory":2909,"vCores":0},"clusterResources":{"memory":8192,"vCores":8},"queueName":"root.default","schedulingPolicy":"fair","numPendingApps":0,"numActiveApps":0}]
http://www.w3.org/2001/XMLSchema-instance;
xsi:type="fairScheduler">
2147483647
0
0
8192
8
0
0
8192
8
8192
8
root
fair
20
1024
1
5283
2
0
0
5283
0
8192
8
root.Engineering
fair
2147483647
1024
1
8192
8
0
0
