[
https://issues.apache.org/jira/browse/YARN-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tamas Domok updated YARN-11069:
-------------------------------
Description:
During the investigation YARN-10922 I found multiple issues with dynamic ACL
handling, no wonder it was documented as unsupported feature. But in some cases
it did work and some users utilised that behaviour.
There is a severe bug when using Flexible AQC and dynamic parent queues,
*anybody* can submit an app regardless of what ACLs are set (when the dynamic
parent queue is not exists yet) - see: YARN-11066.
After this dynamic ACL feature YARN-11060 should be fixed as well.
Examples scenarios:
Legacy AQC:
{code}
yarn.scheduler.capacity.root.managed.auto-create-child-queue.enabled: true
yarn.scheduler.capacity.root.managed.acl_submit_applications: nobody
yarn.scheduler.capacity.root.managed.acl_administer_queue: nobody
yarn.scheduler.capacity.root.managed.acl_submit_applications: user1
yarn.scheduler.capacity.root.managed.acl_administer_queue: admin1
yarn.scheduler.capacity.root.managed.leaf-queue-template.acl_submit_applications:
user2
yarn.scheduler.capacity.root.managed.leaf-queue-template.acl_administer_queue:
admin2
{code}
user2 can submit an application to root.managed.user2 even if the queue doesn't
exist yet due to template
Permissions:
* root: {nobody, nobody}
* root.managed: {user1, admin1}
* root.managed.auto: {user2, admin2}
Flexible AQC:
{code}
yarn.scheduler.capacity.root.auto-queue-creation-v2.enabled: true
yarn.scheduler.capacity.root.acl_submit_applications: user
yarn.scheduler.capacity.root.acl_administer_queue: admin
yarn.scheduler.capacity.root.auto-queue-creation-v2.parent-template.acl_submit_applications:
parentUser1
yarn.scheduler.capacity.root.auto-queue-creation-v2.parent-template.acl_administer_queue:
parentAdmin1
yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_submit_applications:
user1
yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_administer_queue:
admin1
{code}
user1 can submit an application to root.autoParent.user1 even if the queue
doesn't exist yet due to template
Permissions:
* root: {user, admin}
* root.autoParent: {parentUser1, parentAdmin1}
* root.autoParent.autoLeaf: {user1, admin1}
NOTE: the *.leaf-template* and *.parent-template* overwrites the *.template*
variant.
was:
During the investigation YARN-10922 I found multiple issues with dynamic ACL
handling, no wonder it was documented as unsupported feature. But in some cases
it did work and some users utilised that behaviour.
There is a severe bug when using Flexible AQC and dynamic parent queues,
*anybody* can submit an app regardless of what ACLs are set (when the dynamic
parent queue is not exists yet) - see: YARN-11066.
After this dynamic ACL feature YARN-11060 should be fixed as well.
Examples scenarios:
Legacy AQC:
{code}
yarn.scheduler.capacity.root.managed.auto-create-child-queue.enabled: true
yarn.scheduler.capacity.root.managed.acl_submit_applications: nobody
yarn.scheduler.capacity.root.managed.acl_administer_queue: nobody
yarn.scheduler.capacity.root.managed.acl_submit_applications: user1
yarn.scheduler.capacity.root.managed.acl_administer_queue: admin1
yarn.scheduler.capacity.root.managed.leaf-queue-template.acl_submit_applications:
user2
yarn.scheduler.capacity.root.managed.leaf-queue-template.acl_administer_queue:
admin2
{code}
user2 can submit an application to root.managed.user2 even if the queue doesn't
exist yet due to template
Permissions:
- root: {nobody, nobody}
- root.managed: {user1, admin1}
- root.managed.auto: {user2, admin2}
Flexible AQC:
{code}
yarn.scheduler.capacity.root.auto-queue-creation-v2.enabled: true
yarn.scheduler.capacity.root.acl_submit_applications: user
yarn.scheduler.capacity.root.acl_administer_queue: admin
yarn.scheduler.capacity.root.auto-queue-creation-v2.parent-template.acl_submit_applications:
parentUser1
yarn.scheduler.capacity.root.auto-queue-creation-v2.parent-template.acl_administer_queue:
parentAdmin1
yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_submit_applications:
user1
yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_administer_queue:
admin1
{code}
user1 can submit an application to root.autoParent.user1 even if the queue
doesn't exist yet due to template
Permissions:
- root: {user, admin}
- root.autoParent: {parentUser1, parentAdmin1}
- root.autoParent.autoLeaf: {user1, admin1}
Flexible AQC with overwritten leaf:
{code}
yarn.scheduler.capacity.root.auto-queue-creation-v2.enabled: true
yarn.scheduler.capacity.root.acl_submit_applications: user
yarn.scheduler.capacity.root.acl_administer_queue: admin
yarn.scheduler.capacity.root.auto-queue-creation-v2.template.acl_submit_applications:
commonUser1
yarn.scheduler.capacity.root.auto-queue-creation-v2.template.acl_administer_queue:
commonUser2
yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_submit_applications:
user1
yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_administer_queue:
admin1
{code}
user1 can submit an application to root.autoParent.user1 even if the queue
doesn't exist yet due to template
Permissions:
- root: {user, admin}
- root.autoParent: {commonUser1, commonUser2}
- root.managed.autoLeaf: {user1, admin1}
NOTE: the *.leaf-template* and *.parent-template* overwrites the *.template*
variant.
> Dynamic Queue ACL handling in Legacy and Flexible Auto Created Queues
> ---------------------------------------------------------------------
>
> Key: YARN-11069
> URL: https://issues.apache.org/jira/browse/YARN-11069
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: capacity scheduler, yarn
> Reporter: Tamas Domok
> Assignee: Tamas Domok
> Priority: Major
>
> During the investigation YARN-10922 I found multiple issues with dynamic ACL
> handling, no wonder it was documented as unsupported feature. But in some
> cases it did work and some users utilised that behaviour.
> There is a severe bug when using Flexible AQC and dynamic parent queues,
> *anybody* can submit an app regardless of what ACLs are set (when the dynamic
> parent queue is not exists yet) - see: YARN-11066.
> After this dynamic ACL feature YARN-11060 should be fixed as well.
> Examples scenarios:
> Legacy AQC:
> {code}
> yarn.scheduler.capacity.root.managed.auto-create-child-queue.enabled: true
> yarn.scheduler.capacity.root.managed.acl_submit_applications: nobody
> yarn.scheduler.capacity.root.managed.acl_administer_queue: nobody
> yarn.scheduler.capacity.root.managed.acl_submit_applications: user1
> yarn.scheduler.capacity.root.managed.acl_administer_queue: admin1
> yarn.scheduler.capacity.root.managed.leaf-queue-template.acl_submit_applications:
> user2
> yarn.scheduler.capacity.root.managed.leaf-queue-template.acl_administer_queue:
> admin2
> {code}
> user2 can submit an application to root.managed.user2 even if the queue
> doesn't exist yet due to template
> Permissions:
> * root: {nobody, nobody}
> * root.managed: {user1, admin1}
> * root.managed.auto: {user2, admin2}
> Flexible AQC:
> {code}
> yarn.scheduler.capacity.root.auto-queue-creation-v2.enabled: true
> yarn.scheduler.capacity.root.acl_submit_applications: user
> yarn.scheduler.capacity.root.acl_administer_queue: admin
> yarn.scheduler.capacity.root.auto-queue-creation-v2.parent-template.acl_submit_applications:
> parentUser1
> yarn.scheduler.capacity.root.auto-queue-creation-v2.parent-template.acl_administer_queue:
> parentAdmin1
> yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_submit_applications:
> user1
> yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_administer_queue:
> admin1
> {code}
> user1 can submit an application to root.autoParent.user1 even if the queue
> doesn't exist yet due to template
> Permissions:
> * root: {user, admin}
> * root.autoParent: {parentUser1, parentAdmin1}
> * root.autoParent.autoLeaf: {user1, admin1}
> NOTE: the *.leaf-template* and *.parent-template* overwrites the *.template*
> variant.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
