[jira] [Comment Edited] (YARN-7451) Resources Types should be visible in the Cluster Apps API "resourceRequests" section
[
https://issues.apache.org/jira/browse/YARN-7451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16501854#comment-16501854
]
Szilard Nemeth edited comment on YARN-7451 at 6/7/18 4:50 PM:
--
Hi [~templedf]!
Tested with latest upstream, resourceRequests shows custom resources and the
scheduler endpoint also exposes custom resources for queues.
Looks like this has been fixed with
https://issues.apache.org/jira/browse/YARN-7817
As per our offline discussion, I will modify the description and title so that
this issue will only cover adding missing unit tests for YARN-7817
was (Author: snemeth):
Hi [~templedf]!
Tested with latest upstream, resourceRequests shows custom resources and the
scheduler endpoint also exposes custom resources for queues.
Looks like this has been fixed with
https://issues.apache.org/jira/browse/YARN-7817
Do you think it's worth to keep unit tests and open another Jira task to add
them as 7817 haven't added any tests and close this as a duplicate?
Thanks in advance!
> Resources Types should be visible in the Cluster Apps API "resourceRequests"
> section
>
>
> Key: YARN-7451
> URL: https://issues.apache.org/jira/browse/YARN-7451
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, restapi
>Affects Versions: 3.0.0
>Reporter: Grant Sohn
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: YARN-7451.001.patch, YARN-7451.002.patch,
> YARN-7451.003.patch, YARN-7451.004.patch, YARN-7451.005.patch,
> YARN-7451.006.patch, YARN-7451.007.patch, YARN-7451.008.patch,
> YARN-7451.009.patch, YARN-7451.010.patch, YARN-7451.011.patch,
> YARN-7451.012.patch,
> YARN-7451__Expose_custom_resource_types_on_RM_scheduler_API_as_flattened_map01_02.patch
>
>
> When running jobs that request resource types the RM Cluster Apps API should
> include this in the "resourceRequests" object.
> Additionally, when calling the RM scheduler API it returns:
> {noformat}
> "childQueues": {
> "queue": [
> {
> "allocatedContainers": 101,
> "amMaxResources": {
> "memory": 320390,
> "vCores": 192
> },
> "amUsedResources": {
> "memory": 1024,
> "vCores": 1
> },
> "clusterResources": {
> "memory": 640779,
> "vCores": 384
> },
> "demandResources": {
> "memory": 103424,
> "vCores": 101
> },
> "fairResources": {
> "memory": 640779,
> "vCores": 384
> },
> "maxApps": 2147483647,
> "maxResources": {
> "memory": 640779,
> "vCores": 384
> },
> "minResources": {
> "memory": 0,
> "vCores": 0
> },
> "numActiveApps": 1,
> "numPendingApps": 0,
> "preemptable": true,
> "queueName": "root.users.systest",
> "reservedContainers": 0,
> "reservedResources": {
> "memory": 0,
> "vCores": 0
> },
> "schedulingPolicy": "fair",
> "steadyFairResources": {
> "memory": 320390,
> "vCores": 192
> },
> "type":
[jira] [Comment Edited] (YARN-7451) Resources Types should be visible in the Cluster Apps API "resourceRequests" section
[
https://issues.apache.org/jira/browse/YARN-7451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16501854#comment-16501854
]
Szilard Nemeth edited comment on YARN-7451 at 6/7/18 4:50 PM:
--
Hi [~templedf]!
Tested with latest upstream, resourceRequests shows custom resources and the
scheduler endpoint also exposes custom resources for queues.
Looks like this has been fixed with
https://issues.apache.org/jira/browse/YARN-7817
As per our offline discussion, I will modify the description and title so that
this issue will only cover adding missing unit tests for YARN-7817
was (Author: snemeth):
Hi [~templedf]!
Tested with latest upstream, resourceRequests shows custom resources and the
scheduler endpoint also exposes custom resources for queues.
Looks like this has been fixed with
https://issues.apache.org/jira/browse/YARN-7817
As per our offline discussion, I will modify the description and title so that
this issue will only cover adding missing unit tests for YARN-7817
> Resources Types should be visible in the Cluster Apps API "resourceRequests"
> section
>
>
> Key: YARN-7451
> URL: https://issues.apache.org/jira/browse/YARN-7451
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, restapi
>Affects Versions: 3.0.0
>Reporter: Grant Sohn
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: YARN-7451.001.patch, YARN-7451.002.patch,
> YARN-7451.003.patch, YARN-7451.004.patch, YARN-7451.005.patch,
> YARN-7451.006.patch, YARN-7451.007.patch, YARN-7451.008.patch,
> YARN-7451.009.patch, YARN-7451.010.patch, YARN-7451.011.patch,
> YARN-7451.012.patch,
> YARN-7451__Expose_custom_resource_types_on_RM_scheduler_API_as_flattened_map01_02.patch
>
>
> When running jobs that request resource types the RM Cluster Apps API should
> include this in the "resourceRequests" object.
> Additionally, when calling the RM scheduler API it returns:
> {noformat}
> "childQueues": {
> "queue": [
> {
> "allocatedContainers": 101,
> "amMaxResources": {
> "memory": 320390,
> "vCores": 192
> },
> "amUsedResources": {
> "memory": 1024,
> "vCores": 1
> },
> "clusterResources": {
> "memory": 640779,
> "vCores": 384
> },
> "demandResources": {
> "memory": 103424,
> "vCores": 101
> },
> "fairResources": {
> "memory": 640779,
> "vCores": 384
> },
> "maxApps": 2147483647,
> "maxResources": {
> "memory": 640779,
> "vCores": 384
> },
> "minResources": {
> "memory": 0,
> "vCores": 0
> },
> "numActiveApps": 1,
> "numPendingApps": 0,
> "preemptable": true,
> "queueName": "root.users.systest",
> "reservedContainers": 0,
> "reservedResources": {
> "memory": 0,
> "vCores": 0
> },
> "schedulingPolicy": "fair",
> "steadyFairResources": {
> "memory": 320390,
> "vCores": 192
> },
> "type": "fairSchedulerLeafQueueInfo",
>
[jira] [Comment Edited] (YARN-7451) Resources Types should be visible in the Cluster Apps API "resourceRequests" section
[
https://issues.apache.org/jira/browse/YARN-7451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16501854#comment-16501854
]
Szilard Nemeth edited comment on YARN-7451 at 6/5/18 2:20 PM:
--
Hi [~templedf]!
Tested with latest upstream, resourceRequests shows custom resources and the
scheduler endpoint also exposes custom resources for queues.
Looks like this has been fixed with
https://issues.apache.org/jira/browse/YARN-7817
Do you think it's worth to keep unit tests and open another Jira task to add
them as 7817 haven't added any tests and close this as a duplicate?
Thanks in advance!
was (Author: snemeth):
Hi [~templedf]!
Tested with latest upstream, resourceRequests shows custom resources and the
scheduler endpoint also exposes custom resources for queues.
Looks like this has been fixed with
https://issues.apache.org/jira/browse/YARN-7817
Do you think it's worth to keep unit tests and open another Jira task to add
them as 7817 haven't add any tests?
Thanks in advance!
> Resources Types should be visible in the Cluster Apps API "resourceRequests"
> section
>
>
> Key: YARN-7451
> URL: https://issues.apache.org/jira/browse/YARN-7451
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, restapi
>Affects Versions: 3.0.0
>Reporter: Grant Sohn
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: YARN-7451.001.patch, YARN-7451.002.patch,
> YARN-7451.003.patch, YARN-7451.004.patch, YARN-7451.005.patch,
> YARN-7451.006.patch, YARN-7451.007.patch, YARN-7451.008.patch,
> YARN-7451.009.patch, YARN-7451.010.patch, YARN-7451.011.patch,
> YARN-7451.012.patch,
> YARN-7451__Expose_custom_resource_types_on_RM_scheduler_API_as_flattened_map01_02.patch
>
>
> When running jobs that request resource types the RM Cluster Apps API should
> include this in the "resourceRequests" object.
> Additionally, when calling the RM scheduler API it returns:
> {noformat}
> "childQueues": {
> "queue": [
> {
> "allocatedContainers": 101,
> "amMaxResources": {
> "memory": 320390,
> "vCores": 192
> },
> "amUsedResources": {
> "memory": 1024,
> "vCores": 1
> },
> "clusterResources": {
> "memory": 640779,
> "vCores": 384
> },
> "demandResources": {
> "memory": 103424,
> "vCores": 101
> },
> "fairResources": {
> "memory": 640779,
> "vCores": 384
> },
> "maxApps": 2147483647,
> "maxResources": {
> "memory": 640779,
> "vCores": 384
> },
> "minResources": {
> "memory": 0,
> "vCores": 0
> },
> "numActiveApps": 1,
> "numPendingApps": 0,
> "preemptable": true,
> "queueName": "root.users.systest",
> "reservedContainers": 0,
> "reservedResources": {
> "memory": 0,
> "vCores": 0
> },
> "schedulingPolicy": "fair",
> "steadyFairResources": {
> "memory": 320390,
> "vCores": 192
> },
> "type": "fairSchedulerLeafQueueInfo",
>
[jira] [Comment Edited] (YARN-7451) Resources Types should be visible in the Cluster Apps API "resourceRequests" section
[
https://issues.apache.org/jira/browse/YARN-7451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16326301#comment-16326301
]
Szilard Nemeth edited comment on YARN-7451 at 1/16/18 7:53 AM:
---
Thanks [~grepas] for your comments.
Please see my updated patch (v9) for the fixes for your comments.
# Good idea, I implemented a test class for this.
# fixed
# I added javadoc for most of the classes where it was reasonable.
was (Author: snemeth):
Thanks [~grepas] for your comments.
Please see my updated patch (v9) for the fixes for your comments.
# Good idea, I implemented a test class for this.
# fixed
# I added javadoc for most of the classes that were reasonable.
> Resources Types should be visible in the Cluster Apps API "resourceRequests"
> section
>
>
> Key: YARN-7451
> URL: https://issues.apache.org/jira/browse/YARN-7451
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, restapi
>Affects Versions: 3.0.0
>Reporter: Grant Sohn
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: YARN-7451.001.patch, YARN-7451.002.patch,
> YARN-7451.003.patch, YARN-7451.004.patch, YARN-7451.005.patch,
> YARN-7451.006.patch, YARN-7451.007.patch, YARN-7451.008.patch,
> YARN-7451.009.patch,
> YARN-7451__Expose_custom_resource_types_on_RM_scheduler_API_as_flattened_map01_02.patch
>
>
> When running jobs that request resource types the RM Cluster Apps API should
> include this in the "resourceRequests" object.
> Additionally, when calling the RM scheduler API it returns:
> {noformat}
> "childQueues": {
> "queue": [
> {
> "allocatedContainers": 101,
> "amMaxResources": {
> "memory": 320390,
> "vCores": 192
> },
> "amUsedResources": {
> "memory": 1024,
> "vCores": 1
> },
> "clusterResources": {
> "memory": 640779,
> "vCores": 384
> },
> "demandResources": {
> "memory": 103424,
> "vCores": 101
> },
> "fairResources": {
> "memory": 640779,
> "vCores": 384
> },
> "maxApps": 2147483647,
> "maxResources": {
> "memory": 640779,
> "vCores": 384
> },
> "minResources": {
> "memory": 0,
> "vCores": 0
> },
> "numActiveApps": 1,
> "numPendingApps": 0,
> "preemptable": true,
> "queueName": "root.users.systest",
> "reservedContainers": 0,
> "reservedResources": {
> "memory": 0,
> "vCores": 0
> },
> "schedulingPolicy": "fair",
> "steadyFairResources": {
> "memory": 320390,
> "vCores": 192
> },
> "type": "fairSchedulerLeafQueueInfo",
> "usedResources": {
> "memory": 103424,
> "vCores": 101
> }
> }
> ]
> {noformat}
> However, the web UI shows resource types usage.
--
This message was sent
[jira] [Comment Edited] (YARN-7451) Resources Types should be visible in the Cluster Apps API "resourceRequests" section
[
https://issues.apache.org/jira/browse/YARN-7451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16326301#comment-16326301
]
Szilard Nemeth edited comment on YARN-7451 at 1/15/18 2:56 PM:
---
Thanks [~grepas] for your comments.
Please see my updated patch (v9) for the fixes for your comments.
# Good idea, I implemented a test class for this.
# fixed
# I added javadoc for most of the classes that were reasonable.
was (Author: snemeth):
Thanks [~grepas] for your comments.
Please see my updated patch (v8) for the fixes for your comments.
# Good idea, I implemented a test class for this.
# fixed
# I added javadoc for most of the classes that were reasonable.
> Resources Types should be visible in the Cluster Apps API "resourceRequests"
> section
>
>
> Key: YARN-7451
> URL: https://issues.apache.org/jira/browse/YARN-7451
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, restapi
>Affects Versions: 3.0.0
>Reporter: Grant Sohn
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: YARN-7451.001.patch, YARN-7451.002.patch,
> YARN-7451.003.patch, YARN-7451.004.patch, YARN-7451.005.patch,
> YARN-7451.006.patch, YARN-7451.007.patch, YARN-7451.008.patch,
> YARN-7451.009.patch,
> YARN-7451__Expose_custom_resource_types_on_RM_scheduler_API_as_flattened_map01_02.patch
>
>
> When running jobs that request resource types the RM Cluster Apps API should
> include this in the "resourceRequests" object.
> Additionally, when calling the RM scheduler API it returns:
> {noformat}
> "childQueues": {
> "queue": [
> {
> "allocatedContainers": 101,
> "amMaxResources": {
> "memory": 320390,
> "vCores": 192
> },
> "amUsedResources": {
> "memory": 1024,
> "vCores": 1
> },
> "clusterResources": {
> "memory": 640779,
> "vCores": 384
> },
> "demandResources": {
> "memory": 103424,
> "vCores": 101
> },
> "fairResources": {
> "memory": 640779,
> "vCores": 384
> },
> "maxApps": 2147483647,
> "maxResources": {
> "memory": 640779,
> "vCores": 384
> },
> "minResources": {
> "memory": 0,
> "vCores": 0
> },
> "numActiveApps": 1,
> "numPendingApps": 0,
> "preemptable": true,
> "queueName": "root.users.systest",
> "reservedContainers": 0,
> "reservedResources": {
> "memory": 0,
> "vCores": 0
> },
> "schedulingPolicy": "fair",
> "steadyFairResources": {
> "memory": 320390,
> "vCores": 192
> },
> "type": "fairSchedulerLeafQueueInfo",
> "usedResources": {
> "memory": 103424,
> "vCores": 101
> }
> }
> ]
> {noformat}
> However, the web UI shows resource types usage.
--
This message was sent by
[jira] [Comment Edited] (YARN-7451) Resources Types should be visible in the Cluster Apps API "resourceRequests" section
[
https://issues.apache.org/jira/browse/YARN-7451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16325674#comment-16325674
]
Szilard Nemeth edited comment on YARN-7451 at 1/14/18 7:54 PM:
---
New patch contains reflection calls to ServiceFinder, so we are not having
license issues on the original Jersey ServiceFinder anymore.
I looked around and found some questions and issues about migrating to Jersey 2
on stackoverflow, so I suppose it's not a trivial thing to implement.
Anyway, I will create a separate task for the upgrade because for the long term
it would be the way to go.
UPDATE: the yetus test result below seems unrelated to my change.
was (Author: snemeth):
New patch contains reflection calls to ServiceFinder, so we are not having
license issues on the original Jersey ServiceFinder anymore.
I looked around and found some questions and issues about migrating to Jersey 2
on stackoverflow, so I suppose it's not a trivial thing to implement.
Anyway, I will create a separate task for the upgrade because for the long term
it would be the way to go.
> Resources Types should be visible in the Cluster Apps API "resourceRequests"
> section
>
>
> Key: YARN-7451
> URL: https://issues.apache.org/jira/browse/YARN-7451
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, restapi
>Affects Versions: 3.0.0
>Reporter: Grant Sohn
>Assignee: Szilard Nemeth
> Attachments: YARN-7451.001.patch, YARN-7451.002.patch,
> YARN-7451.003.patch, YARN-7451.004.patch, YARN-7451.005.patch,
> YARN-7451.006.patch, YARN-7451.007.patch,
> YARN-7451__Expose_custom_resource_types_on_RM_scheduler_API_as_flattened_map01_02.patch
>
>
> When running jobs that request resource types the RM Cluster Apps API should
> include this in the "resourceRequests" object.
> Additionally, when calling the RM scheduler API it returns:
> {noformat}
> "childQueues": {
> "queue": [
> {
> "allocatedContainers": 101,
> "amMaxResources": {
> "memory": 320390,
> "vCores": 192
> },
> "amUsedResources": {
> "memory": 1024,
> "vCores": 1
> },
> "clusterResources": {
> "memory": 640779,
> "vCores": 384
> },
> "demandResources": {
> "memory": 103424,
> "vCores": 101
> },
> "fairResources": {
> "memory": 640779,
> "vCores": 384
> },
> "maxApps": 2147483647,
> "maxResources": {
> "memory": 640779,
> "vCores": 384
> },
> "minResources": {
> "memory": 0,
> "vCores": 0
> },
> "numActiveApps": 1,
> "numPendingApps": 0,
> "preemptable": true,
> "queueName": "root.users.systest",
> "reservedContainers": 0,
> "reservedResources": {
> "memory": 0,
> "vCores": 0
> },
> "schedulingPolicy": "fair",
> "steadyFairResources": {
> "memory": 320390,
> "vCores": 192
> },
> "type": "fairSchedulerLeafQueueInfo",
> "usedResources": {
>
[jira] [Comment Edited] (YARN-7451) Resources Types should be visible in the Cluster Apps API "resourceRequests" section
[
https://issues.apache.org/jira/browse/YARN-7451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16325674#comment-16325674
]
Szilard Nemeth edited comment on YARN-7451 at 1/14/18 4:58 PM:
---
New patch contains reflection calls to ServiceFinder, so we are not having
license issues on the original Jersey ServiceFinder anymore.
I looked around and found some questions and issues about migrating to Jersey 2
on stackoverflow, so I suppose it's not a trivial thing to implement.
Anyway, I will create a separate task for the upgrade because for the long term
it would be the way to go.
was (Author: snemeth):
New patch contains reflection calls to ServiceFinder, so we are not having
license issues on the original Jersey ServiceFinder anymore.
> Resources Types should be visible in the Cluster Apps API "resourceRequests"
> section
>
>
> Key: YARN-7451
> URL: https://issues.apache.org/jira/browse/YARN-7451
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, restapi
>Affects Versions: 3.0.0
>Reporter: Grant Sohn
>Assignee: Szilard Nemeth
> Attachments: YARN-7451.001.patch, YARN-7451.002.patch,
> YARN-7451.003.patch, YARN-7451.004.patch, YARN-7451.005.patch,
> YARN-7451.006.patch, YARN-7451.007.patch,
> YARN-7451__Expose_custom_resource_types_on_RM_scheduler_API_as_flattened_map01_02.patch
>
>
> When running jobs that request resource types the RM Cluster Apps API should
> include this in the "resourceRequests" object.
> Additionally, when calling the RM scheduler API it returns:
> {noformat}
> "childQueues": {
> "queue": [
> {
> "allocatedContainers": 101,
> "amMaxResources": {
> "memory": 320390,
> "vCores": 192
> },
> "amUsedResources": {
> "memory": 1024,
> "vCores": 1
> },
> "clusterResources": {
> "memory": 640779,
> "vCores": 384
> },
> "demandResources": {
> "memory": 103424,
> "vCores": 101
> },
> "fairResources": {
> "memory": 640779,
> "vCores": 384
> },
> "maxApps": 2147483647,
> "maxResources": {
> "memory": 640779,
> "vCores": 384
> },
> "minResources": {
> "memory": 0,
> "vCores": 0
> },
> "numActiveApps": 1,
> "numPendingApps": 0,
> "preemptable": true,
> "queueName": "root.users.systest",
> "reservedContainers": 0,
> "reservedResources": {
> "memory": 0,
> "vCores": 0
> },
> "schedulingPolicy": "fair",
> "steadyFairResources": {
> "memory": 320390,
> "vCores": 192
> },
> "type": "fairSchedulerLeafQueueInfo",
> "usedResources": {
> "memory": 103424,
> "vCores": 101
> }
> }
> ]
> {noformat}
> However, the web UI shows resource types usage.
--
This message was sent
[jira] [Comment Edited] (YARN-7451) Resources Types should be visible in the Cluster Apps API "resourceRequests" section
[
https://issues.apache.org/jira/browse/YARN-7451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324185#comment-16324185
]
Szilard Nemeth edited comment on YARN-7451 at 1/12/18 4:21 PM:
---
Unfortunately, I got an unfavorable response for the legal jira, see:
https://issues.apache.org/jira/browse/LEGAL-359, so I cannot include the
ServiceFinder as a source file, apparently.
[~rkanter] Any ideas how to resolve this situation?
thanks!
was (Author: snemeth):
Unfortunately, I got an unfavorable response for the legal jira, see:
https://issues.apache.org/jira/browse/LEGAL-359, so I cannot include the
ServiceFinder as a source file, apparently.
Any ideas how to resolve this?
thanks!
> Resources Types should be visible in the Cluster Apps API "resourceRequests"
> section
>
>
> Key: YARN-7451
> URL: https://issues.apache.org/jira/browse/YARN-7451
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, restapi
>Affects Versions: 3.0.0
>Reporter: Grant Sohn
>Assignee: Szilard Nemeth
> Attachments: YARN-7451.001.patch, YARN-7451.002.patch,
> YARN-7451.003.patch, YARN-7451.004.patch, YARN-7451.005.patch,
> YARN-7451.006.patch,
> YARN-7451__Expose_custom_resource_types_on_RM_scheduler_API_as_flattened_map01_02.patch
>
>
> When running jobs that request resource types the RM Cluster Apps API should
> include this in the "resourceRequests" object.
> Additionally, when calling the RM scheduler API it returns:
> {noformat}
> "childQueues": {
> "queue": [
> {
> "allocatedContainers": 101,
> "amMaxResources": {
> "memory": 320390,
> "vCores": 192
> },
> "amUsedResources": {
> "memory": 1024,
> "vCores": 1
> },
> "clusterResources": {
> "memory": 640779,
> "vCores": 384
> },
> "demandResources": {
> "memory": 103424,
> "vCores": 101
> },
> "fairResources": {
> "memory": 640779,
> "vCores": 384
> },
> "maxApps": 2147483647,
> "maxResources": {
> "memory": 640779,
> "vCores": 384
> },
> "minResources": {
> "memory": 0,
> "vCores": 0
> },
> "numActiveApps": 1,
> "numPendingApps": 0,
> "preemptable": true,
> "queueName": "root.users.systest",
> "reservedContainers": 0,
> "reservedResources": {
> "memory": 0,
> "vCores": 0
> },
> "schedulingPolicy": "fair",
> "steadyFairResources": {
> "memory": 320390,
> "vCores": 192
> },
> "type": "fairSchedulerLeafQueueInfo",
> "usedResources": {
> "memory": 103424,
> "vCores": 101
> }
> }
> ]
> {noformat}
> However, the web UI shows resource types usage.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (YARN-7451) Resources Types should be visible in the Cluster Apps API "resourceRequests" section
[
https://issues.apache.org/jira/browse/YARN-7451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323935#comment-16323935
]
Szilard Nemeth edited comment on YARN-7451 at 1/12/18 1:11 PM:
---
Thanks [~grepas] for your comments.
All of them are fixed except
org.apache.hadoop.yarn.server.resourcemanager.webapp.representationhelper.path.ArrayElementPathSegment#extractIndex
since it is already "handled".
See the line "return Integer.parseInt(matcher.group(1));" ,
the matcher group call would throw an IllegalStateException which is perfectly
reasonable in tests I suppose.
Also fixed a bunch of checkstyle warnings.
was (Author: snemeth):
Thanks [~grepas] for your comments.
All of them are fixed except
org.apache.hadoop.yarn.server.resourcemanager.webapp.representationhelper.path.ArrayElementPathSegment#extractIndex
since it is already "handled".
in the line "return Integer.parseInt(matcher.group(1));" the matcher group call
would throw an IllegalStateException which is perfectly reasonable in tests I
think.
Also fixed a bunch of checkstyle warnings.
> Resources Types should be visible in the Cluster Apps API "resourceRequests"
> section
>
>
> Key: YARN-7451
> URL: https://issues.apache.org/jira/browse/YARN-7451
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, restapi
>Affects Versions: 3.0.0
>Reporter: Grant Sohn
>Assignee: Szilard Nemeth
> Attachments: YARN-7451.001.patch, YARN-7451.002.patch,
> YARN-7451.003.patch, YARN-7451.004.patch, YARN-7451.005.patch,
> YARN-7451.006.patch,
> YARN-7451__Expose_custom_resource_types_on_RM_scheduler_API_as_flattened_map01_02.patch
>
>
> When running jobs that request resource types the RM Cluster Apps API should
> include this in the "resourceRequests" object.
> Additionally, when calling the RM scheduler API it returns:
> {noformat}
> "childQueues": {
> "queue": [
> {
> "allocatedContainers": 101,
> "amMaxResources": {
> "memory": 320390,
> "vCores": 192
> },
> "amUsedResources": {
> "memory": 1024,
> "vCores": 1
> },
> "clusterResources": {
> "memory": 640779,
> "vCores": 384
> },
> "demandResources": {
> "memory": 103424,
> "vCores": 101
> },
> "fairResources": {
> "memory": 640779,
> "vCores": 384
> },
> "maxApps": 2147483647,
> "maxResources": {
> "memory": 640779,
> "vCores": 384
> },
> "minResources": {
> "memory": 0,
> "vCores": 0
> },
> "numActiveApps": 1,
> "numPendingApps": 0,
> "preemptable": true,
> "queueName": "root.users.systest",
> "reservedContainers": 0,
> "reservedResources": {
> "memory": 0,
> "vCores": 0
> },
> "schedulingPolicy": "fair",
> "steadyFairResources": {
> "memory": 320390,
> "vCores": 192
> },
> "type": "fairSchedulerLeafQueueInfo",
> "usedResources": {
>
[jira] [Comment Edited] (YARN-7451) Resources Types should be visible in the Cluster Apps API "resourceRequests" section
[
https://issues.apache.org/jira/browse/YARN-7451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323935#comment-16323935
]
Szilard Nemeth edited comment on YARN-7451 at 1/12/18 1:10 PM:
---
Thanks [~grepas] for your comments.
All of them are fixed except
org.apache.hadoop.yarn.server.resourcemanager.webapp.representationhelper.path.ArrayElementPathSegment#extractIndex
since it is already "handled".
in the line "return Integer.parseInt(matcher.group(1));" the matcher group call
would throw an IllegalStateException which is perfectly reasonable in tests I
think.
Also fixed a bunch of checkstyle warnings.
was (Author: snemeth):
Thanks [~grepas] for your comments.
All of them are fixed except
org.apache.hadoop.yarn.server.resourcemanager.webapp.representationhelper.path.ArrayElementPathSegment#extractIndex
since it is already "handled".
in the line "return Integer.parseInt(matcher.group(1));" the matcher group call
would throw an IllegalStateException which is perfectly reasonable in tests I
think.
> Resources Types should be visible in the Cluster Apps API "resourceRequests"
> section
>
>
> Key: YARN-7451
> URL: https://issues.apache.org/jira/browse/YARN-7451
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, restapi
>Affects Versions: 3.0.0
>Reporter: Grant Sohn
>Assignee: Szilard Nemeth
> Attachments: YARN-7451.001.patch, YARN-7451.002.patch,
> YARN-7451.003.patch, YARN-7451.004.patch, YARN-7451.005.patch,
> YARN-7451.006.patch,
> YARN-7451__Expose_custom_resource_types_on_RM_scheduler_API_as_flattened_map01_02.patch
>
>
> When running jobs that request resource types the RM Cluster Apps API should
> include this in the "resourceRequests" object.
> Additionally, when calling the RM scheduler API it returns:
> {noformat}
> "childQueues": {
> "queue": [
> {
> "allocatedContainers": 101,
> "amMaxResources": {
> "memory": 320390,
> "vCores": 192
> },
> "amUsedResources": {
> "memory": 1024,
> "vCores": 1
> },
> "clusterResources": {
> "memory": 640779,
> "vCores": 384
> },
> "demandResources": {
> "memory": 103424,
> "vCores": 101
> },
> "fairResources": {
> "memory": 640779,
> "vCores": 384
> },
> "maxApps": 2147483647,
> "maxResources": {
> "memory": 640779,
> "vCores": 384
> },
> "minResources": {
> "memory": 0,
> "vCores": 0
> },
> "numActiveApps": 1,
> "numPendingApps": 0,
> "preemptable": true,
> "queueName": "root.users.systest",
> "reservedContainers": 0,
> "reservedResources": {
> "memory": 0,
> "vCores": 0
> },
> "schedulingPolicy": "fair",
> "steadyFairResources": {
> "memory": 320390,
> "vCores": 192
> },
> "type": "fairSchedulerLeafQueueInfo",
> "usedResources": {
> "memory": 103424,
>
