Re: [Openstack-operators] Dynamic Policy
As an Op, I've ran into this problem and keep running into it. I would very much like a solution. Its also quite related to the nova instance user issue I've been working on, that's needed by the App Catalog project. So, yes, please keep fighting the good fight. Thanks, Kevin From: Adam Young [ayo...@redhat.com] Sent: Wednesday, August 05, 2015 7:50 AM To: openstack-operators@lists.openstack.org Subject: [Openstack-operators] Dynamic Policy How do you delegate the ability to delegate? Lets say you are running a large cloud (purely hypothetical here) and you want to let a user manage their own project. They are admin but they should be able to invite or eject people. In order to do this, an ordinary user needs to be able to make a role assignment. However, Keystone does not support this today: if you are admin somewhere, you are admin everywhere: https://bugs.launchpad.net/keystone/+bug/968696 Access control in OpenStack is controlled by Policy. An informal survey of operators shows that most people run with the stock policies such as the Nova policy: http://git.openstack.org/cgit/openstack/nova/tree/etc/nova/policy.json In order to scope admin to the proejct, we would need to have rules that enforce this scoping: Evey rule should check that the project_id in the token provided matches the project_id of the resource of the API. If we manage to get the policy files rewritten this way, We then need a way to limit what roles a user can assign.The default mechanism would say that a user needs to have an administrative role on the project (or domain) that they want to assign the role on. I don't think anything I've written thus far is controversial. Then, why has it not happened yet? here are the list of problems we need to solve: 1. Operators expect the existing policy files to work as is. Changing them will break workflow. 2. If everything is scoped, we need a way to delete resources left orphan when a project is deleted from Keystone and the service does not get the notification (or know how to handle it). Of the two, I think the top one is the more difficult to solve. Scoping everything can be handled via one of two mechanism; either allow a power-admin user to get a token scoped to some random project (even if it has been deleted) or allow a token scoped to an administrative project to also delete resources for a random project. Dealing with the existing policy file issues is trickier, and that is the real reason for the Dynamic Policy approach: give the endpoints the ability to fetch their policy files from Keystone. If policy goes from being a configuration file to data, it is managed outside of the configuration management tools, and becomes much more fluid. This has risks, and should be an Opt-In mechanism. Fetching the policy files from Keystone also provides the start of a richer set of management for policy rules. Currently, each of the stock policy files exists only in their seperate git repos. There is no sharing of policy rules across projects. If the policy files were managed, rule by rule, from a centralized repository, rules could be shared, providing, among other things, the ability to enforce hierarchical roles, roles namespaced to a service, and other, richer policy management. Feedback on this approach from operators is greatly appreciated. I need to justify the effort that would go in to making this happen, so if you want it, speak up. If, on the other hand, you feel that this is needlessly complicated or would make deployments more difficult, that is important too, and please let me know. Finally, if you can see some alternative methods of implementing a more dynamic access control, please chime in. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Dynamic Policy
We ran into this as well. What we did is create an external to keystone api, that we expose to our end users via a UI. The api will let user create projects (with a specific defined quota) and also add users with the project admins role to the project. Those admins can add/remove users from the project and also delete the project. You can also be a member, where you have the ability to spin up vm's under the project but not add/remove users or remove the project. We also do some other stuff to clean up items in a project before its deleted. We are working to move this functionality out of the current external API and into keystone. I believe we were going to look at waffle-haus to add a paste filter to intercept the project create calls and do the needful. We also modified the policy.json files for the services that we care about to add the new roles that we created. Kris Lindgren Senior Linux Systems Engineer GoDaddy, LLC. On 8/5/15, 9:39 AM, Fox, Kevin M kevin@pnnl.gov wrote: As an Op, I've ran into this problem and keep running into it. I would very much like a solution. Its also quite related to the nova instance user issue I've been working on, that's needed by the App Catalog project. So, yes, please keep fighting the good fight. Thanks, Kevin From: Adam Young [ayo...@redhat.com] Sent: Wednesday, August 05, 2015 7:50 AM To: openstack-operators@lists.openstack.org Subject: [Openstack-operators] Dynamic Policy How do you delegate the ability to delegate? Lets say you are running a large cloud (purely hypothetical here) and you want to let a user manage their own project. They are admin but they should be able to invite or eject people. In order to do this, an ordinary user needs to be able to make a role assignment. However, Keystone does not support this today: if you are admin somewhere, you are admin everywhere: https://bugs.launchpad.net/keystone/+bug/968696 Access control in OpenStack is controlled by Policy. An informal survey of operators shows that most people run with the stock policies such as the Nova policy: http://git.openstack.org/cgit/openstack/nova/tree/etc/nova/policy.json In order to scope admin to the proejct, we would need to have rules that enforce this scoping: Evey rule should check that the project_id in the token provided matches the project_id of the resource of the API. If we manage to get the policy files rewritten this way, We then need a way to limit what roles a user can assign.The default mechanism would say that a user needs to have an administrative role on the project (or domain) that they want to assign the role on. I don't think anything I've written thus far is controversial. Then, why has it not happened yet? here are the list of problems we need to solve: 1. Operators expect the existing policy files to work as is. Changing them will break workflow. 2. If everything is scoped, we need a way to delete resources left orphan when a project is deleted from Keystone and the service does not get the notification (or know how to handle it). Of the two, I think the top one is the more difficult to solve. Scoping everything can be handled via one of two mechanism; either allow a power-admin user to get a token scoped to some random project (even if it has been deleted) or allow a token scoped to an administrative project to also delete resources for a random project. Dealing with the existing policy file issues is trickier, and that is the real reason for the Dynamic Policy approach: give the endpoints the ability to fetch their policy files from Keystone. If policy goes from being a configuration file to data, it is managed outside of the configuration management tools, and becomes much more fluid. This has risks, and should be an Opt-In mechanism. Fetching the policy files from Keystone also provides the start of a richer set of management for policy rules. Currently, each of the stock policy files exists only in their seperate git repos. There is no sharing of policy rules across projects. If the policy files were managed, rule by rule, from a centralized repository, rules could be shared, providing, among other things, the ability to enforce hierarchical roles, roles namespaced to a service, and other, richer policy management. Feedback on this approach from operators is greatly appreciated. I need to justify the effort that would go in to making this happen, so if you want it, speak up. If, on the other hand, you feel that this is needlessly complicated or would make deployments more difficult, that is important too, and please let me know. Finally, if you can see some alternative methods of implementing a more dynamic access control, please chime in. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman
Re: [Openstack-operators] Dynamic Policy
On 08/05/2015 12:01 PM, Kris G. Lindgren wrote: We ran into this as well. What we did is create an external to keystone api, that we expose to our end users via a UI. The api will let user create projects (with a specific defined quota) and also add users with the project admins role to the project. Those admins can add/remove users from the project and also delete the project. You can also be a member, where you have the ability to spin up vm's under the project but not add/remove users or remove the project. We also do some other stuff to clean up items in a project before its deleted. We are working to move this functionality out of the current external API and into keystone. I believe we were going to look at waffle-haus to add a paste filter to intercept the project create calls and do the needful. We also modified the policy.json files for the services that we care about to add the new roles that we created. Were you working around limitation by building an external system to Keystone to provide a means of delegating the ability to assign roles? Would you have wanted to synchronize the roles you defined in your Keytone instance with the policy files directly? Did you have to modify policy files beyond the Keystone policy file? Kris Lindgren Senior Linux Systems Engineer GoDaddy, LLC. On 8/5/15, 9:39 AM, Fox, Kevin M kevin@pnnl.gov wrote: As an Op, I've ran into this problem and keep running into it. I would very much like a solution. Its also quite related to the nova instance user issue I've been working on, that's needed by the App Catalog project. So, yes, please keep fighting the good fight. Thanks, Kevin From: Adam Young [ayo...@redhat.com] Sent: Wednesday, August 05, 2015 7:50 AM To: openstack-operators@lists.openstack.org Subject: [Openstack-operators] Dynamic Policy How do you delegate the ability to delegate? Lets say you are running a large cloud (purely hypothetical here) and you want to let a user manage their own project. They are admin but they should be able to invite or eject people. In order to do this, an ordinary user needs to be able to make a role assignment. However, Keystone does not support this today: if you are admin somewhere, you are admin everywhere: https://bugs.launchpad.net/keystone/+bug/968696 Access control in OpenStack is controlled by Policy. An informal survey of operators shows that most people run with the stock policies such as the Nova policy: http://git.openstack.org/cgit/openstack/nova/tree/etc/nova/policy.json In order to scope admin to the proejct, we would need to have rules that enforce this scoping: Evey rule should check that the project_id in the token provided matches the project_id of the resource of the API. If we manage to get the policy files rewritten this way, We then need a way to limit what roles a user can assign.The default mechanism would say that a user needs to have an administrative role on the project (or domain) that they want to assign the role on. I don't think anything I've written thus far is controversial. Then, why has it not happened yet? here are the list of problems we need to solve: 1. Operators expect the existing policy files to work as is. Changing them will break workflow. 2. If everything is scoped, we need a way to delete resources left orphan when a project is deleted from Keystone and the service does not get the notification (or know how to handle it). Of the two, I think the top one is the more difficult to solve. Scoping everything can be handled via one of two mechanism; either allow a power-admin user to get a token scoped to some random project (even if it has been deleted) or allow a token scoped to an administrative project to also delete resources for a random project. Dealing with the existing policy file issues is trickier, and that is the real reason for the Dynamic Policy approach: give the endpoints the ability to fetch their policy files from Keystone. If policy goes from being a configuration file to data, it is managed outside of the configuration management tools, and becomes much more fluid. This has risks, and should be an Opt-In mechanism. Fetching the policy files from Keystone also provides the start of a richer set of management for policy rules. Currently, each of the stock policy files exists only in their seperate git repos. There is no sharing of policy rules across projects. If the policy files were managed, rule by rule, from a centralized repository, rules could be shared, providing, among other things, the ability to enforce hierarchical roles, roles namespaced to a service, and other, richer policy management. Feedback on this approach from operators is greatly appreciated. I need to justify the effort that would go in to making this happen, so if you want it, speak up. If, on the other hand, you feel that this is needlessly
Re: [Openstack-operators] Dynamic Policy
Echoing what others have said, we too have an abstraction layer in the form of a custom UI to allow project owners to create/delete users. As for your questions Adam, having policy in the Keystone database as data seems like a no brainer. As you suggest it enables us to do so much more. For problem #2, that's already a problem today, so I don't see how it has an impact (other than the problem of giving the keys to end-users). In fact, I'd argue that it's an even bigger problem today as an admin (i.e admin everywhere) can delete a project with running resources. A project_admin role limited in scope could be delegated the rights to create/delete users but not projects. -m On Wed, 2015-08-05 at 18:15 +, Kris G. Lindgren wrote: See inline. Kris Lindgren Senior Linux Systems Engineer GoDaddy, LLC. On 8/5/15, 11:19 AM, Adam Young ayo...@redhat.com wrote: On 08/05/2015 12:01 PM, Kris G. Lindgren wrote: We ran into this as well. What we did is create an external to keystone api, that we expose to our end users via a UI. The api will let user create projects (with a specific defined quota) and also add users with the project admins role to the project. Those admins can add/remove users from the project and also delete the project. You can also be a member, where you have the ability to spin up vm's under the project but not add/remove users or remove the project. We also do some other stuff to clean up items in a project before its deleted. We are working to move this functionality out of the current external API and into keystone. I believe we were going to look at waffle-haus to add a paste filter to intercept the project create calls and do the needful. We also modified the policy.json files for the services that we care about to add the new roles that we created. Were you working around limitation by building an external system to Keystone to provide a means of delegating the ability to assign roles? Yes. Basically we wrapped a function that required admin permissions in an keystone API - so that non-admin users could do some admin level tasks. Also, we have ran into the admin on a project in keystone == admin everywhere problem. We were using a created project_admin role to get around that limitation. Would you have wanted to synchronize the roles you defined in your Keytone instance with the policy files directly? Did you have to modify policy files beyond the Keystone policy file? Yes. And Yes, we did modify other services policy files as well to handle the newly created project_admin role. Kris Lindgren Senior Linux Systems Engineer GoDaddy, LLC. On 8/5/15, 9:39 AM, Fox, Kevin M kevin@pnnl.gov wrote: As an Op, I've ran into this problem and keep running into it. I would very much like a solution. Its also quite related to the nova instance user issue I've been working on, that's needed by the App Catalog project. So, yes, please keep fighting the good fight. Thanks, Kevin From: Adam Young [ayo...@redhat.com] Sent: Wednesday, August 05, 2015 7:50 AM To: openstack-operators@lists.openstack.org Subject: [Openstack-operators] Dynamic Policy How do you delegate the ability to delegate? Lets say you are running a large cloud (purely hypothetical here) and you want to let a user manage their own project. They are admin but they should be able to invite or eject people. In order to do this, an ordinary user needs to be able to make a role assignment. However, Keystone does not support this today: if you are admin somewhere, you are admin everywhere: https://bugs.launchpad.net/keystone/+bug/968696 Access control in OpenStack is controlled by Policy. An informal survey of operators shows that most people run with the stock policies such as the Nova policy: http://git.openstack.org/cgit/openstack/nova/tree/etc/nova/policy.json In order to scope admin to the proejct, we would need to have rules that enforce this scoping: Evey rule should check that the project_id in the token provided matches the project_id of the resource of the API. If we manage to get the policy files rewritten this way, We then need a way to limit what roles a user can assign.The default mechanism would say that a user needs to have an administrative role on the project (or domain) that they want to assign the role on. I don't think anything I've written thus far is controversial. Then, why has it not happened yet? here are the list of problems we need to solve: 1. Operators expect the existing policy files to work as is. Changing them will break workflow. 2. If everything is scoped, we need a way to delete resources left orphan when a project is deleted from Keystone
Re: [Openstack-operators] Dynamic Policy
Jumping in with another us too here. We have some custom Horizon extensions that allow project owners to manage some of this stuff. On Wed, Aug 5, 2015 at 4:14 PM, Marc Heckmann marc.heckm...@ubisoft.com wrote: Echoing what others have said, we too have an abstraction layer in the form of a custom UI to allow project owners to create/delete users. As for your questions Adam, having policy in the Keystone database as data seems like a no brainer. As you suggest it enables us to do so much more. For problem #2, that's already a problem today, so I don't see how it has an impact (other than the problem of giving the keys to end-users). In fact, I'd argue that it's an even bigger problem today as an admin (i.e admin everywhere) can delete a project with running resources. A project_admin role limited in scope could be delegated the rights to create/delete users but not projects. -m On Wed, 2015-08-05 at 18:15 +, Kris G. Lindgren wrote: See inline. Kris Lindgren Senior Linux Systems Engineer GoDaddy, LLC. On 8/5/15, 11:19 AM, Adam Young ayo...@redhat.com wrote: On 08/05/2015 12:01 PM, Kris G. Lindgren wrote: We ran into this as well. What we did is create an external to keystone api, that we expose to our end users via a UI. The api will let user create projects (with a specific defined quota) and also add users with the project admins role to the project. Those admins can add/remove users from the project and also delete the project. You can also be a member, where you have the ability to spin up vm's under the project but not add/remove users or remove the project. We also do some other stuff to clean up items in a project before its deleted. We are working to move this functionality out of the current external API and into keystone. I believe we were going to look at waffle-haus to add a paste filter to intercept the project create calls and do the needful. We also modified the policy.json files for the services that we care about to add the new roles that we created. Were you working around limitation by building an external system to Keystone to provide a means of delegating the ability to assign roles? Yes. Basically we wrapped a function that required admin permissions in an keystone API - so that non-admin users could do some admin level tasks. Also, we have ran into the admin on a project in keystone == admin everywhere problem. We were using a created project_admin role to get around that limitation. Would you have wanted to synchronize the roles you defined in your Keytone instance with the policy files directly? Did you have to modify policy files beyond the Keystone policy file? Yes. And Yes, we did modify other services policy files as well to handle the newly created project_admin role. Kris Lindgren Senior Linux Systems Engineer GoDaddy, LLC. On 8/5/15, 9:39 AM, Fox, Kevin M kevin@pnnl.gov wrote: As an Op, I've ran into this problem and keep running into it. I would very much like a solution. Its also quite related to the nova instance user issue I've been working on, that's needed by the App Catalog project. So, yes, please keep fighting the good fight. Thanks, Kevin From: Adam Young [ayo...@redhat.com] Sent: Wednesday, August 05, 2015 7:50 AM To: openstack-operators@lists.openstack.org Subject: [Openstack-operators] Dynamic Policy How do you delegate the ability to delegate? Lets say you are running a large cloud (purely hypothetical here) and you want to let a user manage their own project. They are admin but they should be able to invite or eject people. In order to do this, an ordinary user needs to be able to make a role assignment. However, Keystone does not support this today: if you are admin somewhere, you are admin everywhere: https://bugs.launchpad.net/keystone/+bug/968696 Access control in OpenStack is controlled by Policy. An informal survey of operators shows that most people run with the stock policies such as the Nova policy: http://git.openstack.org/cgit/openstack/nova/tree/etc/nova/policy.json In order to scope admin to the proejct, we would need to have rules that enforce this scoping: Evey rule should check that the project_id in the token provided matches the project_id of the resource of the API. If we manage to get the policy files rewritten this way, We then need a way to limit what roles a user can assign.The default mechanism would say that a user needs to have an administrative role on the project (or domain) that they want to assign the role on. I don't think anything I've written thus far
Re: [Openstack-operators] Dynamic Policy
On 06/08/15 04:01, Kris G. Lindgren wrote: We ran into this as well. What we did is create an external to keystone api, that we expose to our end users via a UI. The api will let user create projects (with a specific defined quota) and also add users with the project admins role to the project. Those admins can add/remove users from the project and also delete the project. You can also be a member, where you have the ability to spin up vm's under the project but not add/remove users or remove the project. We also do some other stuff to clean up items in a project before its deleted. We are working to move this functionality out of the current external API and into keystone. I believe we were going to look at waffle-haus to add a paste filter to intercept the project create calls and do the needful. We're working on something similar, but haven't rolled it to production yet. Is your code available open-source somewhere? Ours will be once it's clean-ish and tested properly, but not yet lest we lead someone into pain and misery. One of the goals you didn't mention above, but I'm sure you also noted, was that changing passwords or setting an initial password isn't exactly clear - we're working on getting a one time link set that an initial user can be sent to be able to set their own first password. We also modified the policy.json files for the services that we care about to add the new roles that we created. Not the easiest task to either get right, or make sure that the files are distributed around in an HA setting. But absolutely necessary. Kris Lindgren Senior Linux Systems Engineer GoDaddy, LLC. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Dynamic Policy for Access Control
On 04/07/2015 11:36 AM, Marc Heckmann wrote: My apologies for not seeing this sooner as the topic is of great interest. My comments below inline.. On Mon, 2015-02-23 at 16:41 +, Tim Bell wrote: -Original Message- From: Adam Young [mailto:ayo...@redhat.com] Sent: 23 February 2015 16:45 To: openstack-operators@lists.openstack.org Subject: [Openstack-operators] Dynamic Policy for Access Control Admin can do everything! has been a common lament, heard for multiple summits. Its more than just a development issue. I'd like to fix that. I think we all would. I'm looking to get some Operator input on the Dynamic Policy issue. I wrote up a general overview last fall, after the Kilo summit: https://adam.younglogic.com/2014/11/dynamic-policy-in-keystone/ I agree with everything in that post. I would add the following comments: 1. I doubt this will change, but to be clear, we cannot lose the ability to create custom roles and limit the capabilities of the standard roles. For example, if I wanted to limit the ability to make images public or limit the ability to associate a floating IP. That is a baseline consideration. We are hoping to make custom roles the norm. 2. This work should not be done in vacuum. Ideally, Horizon support for assigning roles to users and editing policy should be released at the same time or not long after. I realize that this is easier said than done, but it will be important in order for the feature to get used. Role assignments will be done the same way they are now, as Horizon fetches the list of roles from Keystone. Editing policy will require a new UI. I don't see that happening in Horizon until the Keystone mechanism is finalized. Thanks for the response. We can carry on the conversation at the Summit. Some of what I am looking at is: what are the general roles that Operators would like to have by default when deploying OpenStack? As I described in http://openstack-in-production.blogspot.ch/2015/02/delegation-of-roles.html, we've got (mapped per-project to an AD group) - operator (start/stop/reboot/console) - accounting (read ceilometer data for reporting) I've submitted a talk about policy for the Summit: https://www.openstack.org/vote-vancouver/presentation/dynamic-policy-for- access-control If you want, please vote for it, but even if it does not get selected, I'd like to discuss Policy with the operators at the summit, as input to the Keystone development effort. Sounds like a good topic for the ops meetup track. Feedback greatly welcome. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Dynamic Policy for Access Control
My apologies for not seeing this sooner as the topic is of great interest. My comments below inline.. On Mon, 2015-02-23 at 16:41 +, Tim Bell wrote: -Original Message- From: Adam Young [mailto:ayo...@redhat.com] Sent: 23 February 2015 16:45 To: openstack-operators@lists.openstack.org Subject: [Openstack-operators] Dynamic Policy for Access Control Admin can do everything! has been a common lament, heard for multiple summits. Its more than just a development issue. I'd like to fix that. I think we all would. I'm looking to get some Operator input on the Dynamic Policy issue. I wrote up a general overview last fall, after the Kilo summit: https://adam.younglogic.com/2014/11/dynamic-policy-in-keystone/ I agree with everything in that post. I would add the following comments: 1. I doubt this will change, but to be clear, we cannot lose the ability to create custom roles and limit the capabilities of the standard roles. For example, if I wanted to limit the ability to make images public or limit the ability to associate a floating IP. 2. This work should not be done in vacuum. Ideally, Horizon support for assigning roles to users and editing policy should be released at the same time or not long after. I realize that this is easier said than done, but it will be important in order for the feature to get used. Some of what I am looking at is: what are the general roles that Operators would like to have by default when deploying OpenStack? As I described in http://openstack-in-production.blogspot.ch/2015/02/delegation-of-roles.html, we've got (mapped per-project to an AD group) - operator (start/stop/reboot/console) - accounting (read ceilometer data for reporting) I've submitted a talk about policy for the Summit: https://www.openstack.org/vote-vancouver/presentation/dynamic-policy-for- access-control If you want, please vote for it, but even if it does not get selected, I'd like to discuss Policy with the operators at the summit, as input to the Keystone development effort. Sounds like a good topic for the ops meetup track. Feedback greatly welcome. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
