Re: [Openstack] Where does Keystone middleware for Swift belong?
Hi John, On 2012-04-11, at 8:03 PM, John Dickinson wrote: > I do not think that these pieces of middleware belong in the core swift repo. Understood. > 1) Including them in swift would require swift to depend on keystone for full > testing. As I mentioned in my initial email, the modules in question depend on Keystone only at the protocol level, and the unit tests run without any dependence on Keystone. I'm presuming that by 'full testing' you mean the currently non-existent functional tests? > 2) (When the middleware was created) Keystone's API was in a state of > constant flux. Keystone has changed quite a bit since then, so this may not > be the case anymore. As of the essex release, the API and protocol used to communicate identity via the environment can be considered more or less stable. > 3) Swift does not require Keystone to run (and in fact many production > environments don't use keystone at all). [1] Definitely a good point. > 4) We have previously removed auth systems from swift's core code in order to > simplify the codebase and allow separate dev cycles. All that is included now > is the most basic auth system required for dev work, stand-alone tests, and > POC deployments. > Our thought thus far has been that auth systems providing auth for swift have > the responsibility to maintain their integration code. It's not swift's > responsibility to manage and maintain code for every auth system that wants > to provide an auth mechanism for swift. I understand - swift core doesn't want to take on responsibility for integrating with different auth systems. > That being said, there has been quite a bit of recent conversation about the > concept of *-contrib areas for each project that include optional > add-ons/extensions/plugins/etc or alternative APIs for each project. I expect > these conversations to continue next week in person at the summit. One option > is for the swift keystone middleware to be a part of such a contrib area. > > It sounds like there are good reasons for both projects to not want to > include these pieces of middleware in their respective core repos. Perhaps a > contrib area can be a compromise, but that does not answer who is responsible > for it (ie who maintains the code?). I think that's a separate, but related, > question. +1 for allowing non-core functionality to be included via a 'contrib'-rooted package rather than having to reside outside the repo it targets. Regardless of whether it is included as 'contrib' or not, would would you have any interest in my factoring out common functionality from tempauth/swauth and making it available for reuse? >From my understanding, a swift auth mechanism has 3 parts - authentication >(authn), token authentication (token_authn), and authorization (authz). Both >of the main implementations (tempauth and swauth) implement all 3 parts >independently, though the only thing that differs between them is the authn >backend. The swift_auth middleware from Keystone similarly duplicates the >authz functionality of tempauth and swauth. If the common stuff could be >extracted and made available as library modules or base classes, the cost of >maintaining any given auth mechanism would be greatly reduced, and creating a >new mechanism would be that much easier. Please let me know your thoughts. Cheers, Maru ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Where does Keystone middleware for Swift belong?
On Thu, Apr 12, 2012 at 5:03 AM, John Dickinson wrote: > 4) We have previously removed auth systems from swift's core code in order to > simplify the codebase and allow separate dev cycles. All that is included now > is the most basic auth system required for dev work, stand-alone tests, and > POC deployments. If swift doesn't want to be opinionated about a auth server I think this is a good reason to have it outside Swift and probably like you suggest in the contrib area if that something we can make happen. I like the idea of having the middleware in the *client as effectively those middleware are clients but that may not what a end-user expect. It may be another discussion but it would be nice to abstract the auth code in Swift as currently this is not so easy to implement all Swift features in Auth middleware. The best way currently to do so is to follow the commits in tempauth/swauth and each middleware maintainers implement the features on his own. Maru has been thinking about it and started to do an implementation in the swift_auth middleware in this review : https://review.openstack.org/#patch,sidebyside,5661,4,keystone/middleware/swift_auth.py I can't remember what was the outcome of the initial work done by Greg on authz in Swift but is Maru review can be something we can look forward to have in Swift so all the other middleware would only have to do the strict minimal of authentication/validation ? Cheers, Chmouel. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Where does Keystone middleware for Swift belong?
On 04/11/2012 11:03 PM, John Dickinson wrote: I do not think that these pieces of middleware belong in the core swift repo. 1) Including them in swift would require swift to depend on keystone for full testing. 2) (When the middleware was created) Keystone's API was in a state of constant flux. Keystone has changed quite a bit since then, so this may not be the case anymore. 3) Swift does not require Keystone to run (and in fact many production environments don't use keystone at all). [1] 4) We have previously removed auth systems from swift's core code in order to simplify the codebase and allow separate dev cycles. All that is included now is the most basic auth system required for dev work, stand-alone tests, and POC deployments. Our thought thus far has been that auth systems providing auth for swift have the responsibility to maintain their integration code. It's not swift's responsibility to manage and maintain code for every auth system that wants to provide an auth mechanism for swift. That being said, there has been quite a bit of recent conversation about the concept of *-contrib areas for each project that include optional add-ons/extensions/plugins/etc or alternative APIs for each project. I expect these conversations to continue next week in person at the summit. One option is for the swift keystone middleware to be a part of such a contrib area. It sounds like there are good reasons for both projects to not want to include these pieces of middleware in their respective core repos. Perhaps a contrib area can be a compromise, but that does not answer who is responsible for it (ie who maintains the code?). I think that's a separate, but related, question. I think this is an excellent solution -- the contrib area/project. And as far as who should be responsible for it, I believe that the responsibilities should be shared across the relevant projects that may be integrated. For instance, recently Chmouel did a great job working with the Keystone contributors to hammer out the Swift keystone middleware and worked with the Glance contributors to make requisite changes in the Glance Swift storage driver... so a number of affected parties shared responsibility. Best, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Where does Keystone middleware for Swift belong?
I do not think that these pieces of middleware belong in the core swift repo. 1) Including them in swift would require swift to depend on keystone for full testing. 2) (When the middleware was created) Keystone's API was in a state of constant flux. Keystone has changed quite a bit since then, so this may not be the case anymore. 3) Swift does not require Keystone to run (and in fact many production environments don't use keystone at all). [1] 4) We have previously removed auth systems from swift's core code in order to simplify the codebase and allow separate dev cycles. All that is included now is the most basic auth system required for dev work, stand-alone tests, and POC deployments. Our thought thus far has been that auth systems providing auth for swift have the responsibility to maintain their integration code. It's not swift's responsibility to manage and maintain code for every auth system that wants to provide an auth mechanism for swift. That being said, there has been quite a bit of recent conversation about the concept of *-contrib areas for each project that include optional add-ons/extensions/plugins/etc or alternative APIs for each project. I expect these conversations to continue next week in person at the summit. One option is for the swift keystone middleware to be a part of such a contrib area. It sounds like there are good reasons for both projects to not want to include these pieces of middleware in their respective core repos. Perhaps a contrib area can be a compromise, but that does not answer who is responsible for it (ie who maintains the code?). I think that's a separate, but related, question. [1] Likewise Keystone doesn't require swift and many keystone deployments don't use swift at all. --John On Apr 11, 2012, at 4:07 PM, Maru Newby wrote: > The Keystone repo currently contains the following Swift-specific wsgi > middleware modules: > > https://github.com/openstack/keystone/blob/master/keystone/middleware/s3_token.py > https://github.com/openstack/keystone/blob/master/keystone/middleware/swift_auth.py > > Neither module depends directly on Keystone. s3_token calls Keystone through > HTTP, and swift_auth retrieves Keystone identity data from the wsgi > environment. Both modules, however, depend directly on Swift, and this > forces the Keystone test suite to have to install Swift to run successfully. > > Separate from the dependency issue, both middleware modules need to ensure > that Swift-specific authorization requirements are met. It doesn't make > sense for the Keystone project to be responsible for this, since the Swift > team is the final arbiter of how Swift authorization should behave. > > All signs point to the Swift repo being the best place for these modules to > be maintained. Does this seem reasonable, or is there a better alternative? > Please chime in, especially if you are Swift core. > > Cheers, > > > Maru smime.p7s Description: S/MIME cryptographic signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Where does Keystone middleware for Swift belong?
Agreed that s3_token belongs in Swift, and as per Joshua, ec2_token probably belongs in Nova. Cheers, Maru On 2012-04-11, at 3:07 PM, Chmouel Boudjnah wrote: > On Thu, Apr 12, 2012 at 12:01 AM, Joshua Harlow > wrote: > This also seems to make sense for other items in that directory /middleware? > Should the EC2/S3 stuff be in nova? > > I'd say the s3 middle-ware if it has to be moved would probably be a better > fit in swift than nova since it has been designed to be used with swift's > swift3 middleware. > > Chnmouel. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Where does Keystone middleware for Swift belong?
On Thu, Apr 12, 2012 at 12:01 AM, Joshua Harlow wrote: > This also seems to make sense for other items in that directory > /middleware? > Should the EC2/S3 stuff be in nova? > I'd say the s3 middle-ware if it has to be moved would probably be a better fit in swift than nova since it has been designed to be used with swift's swift3 middleware. Chnmouel. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Where does Keystone middleware for Swift belong?
This also seems to make sense for other items in that directory /middleware? Should the EC2/S3 stuff be in nova? On 4/11/12 2:07 PM, "Maru Newby" wrote: The Keystone repo currently contains the following Swift-specific wsgi middleware modules: https://github.com/openstack/keystone/blob/master/keystone/middleware/s3_token.py https://github.com/openstack/keystone/blob/master/keystone/middleware/swift_auth.py Neither module depends directly on Keystone. s3_token calls Keystone through HTTP, and swift_auth retrieves Keystone identity data from the wsgi environment. Both modules, however, depend directly on Swift, and this forces the Keystone test suite to have to install Swift to run successfully. Separate from the dependency issue, both middleware modules need to ensure that Swift-specific authorization requirements are met. It doesn't make sense for the Keystone project to be responsible for this, since the Swift team is the final arbiter of how Swift authorization should behave. All signs point to the Swift repo being the best place for these modules to be maintained. Does this seem reasonable, or is there a better alternative? Please chime in, especially if you are Swift core. Cheers, Maru ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Where does Keystone middleware for Swift belong?
The Keystone repo currently contains the following Swift-specific wsgi middleware modules: https://github.com/openstack/keystone/blob/master/keystone/middleware/s3_token.py https://github.com/openstack/keystone/blob/master/keystone/middleware/swift_auth.py Neither module depends directly on Keystone. s3_token calls Keystone through HTTP, and swift_auth retrieves Keystone identity data from the wsgi environment. Both modules, however, depend directly on Swift, and this forces the Keystone test suite to have to install Swift to run successfully. Separate from the dependency issue, both middleware modules need to ensure that Swift-specific authorization requirements are met. It doesn't make sense for the Keystone project to be responsible for this, since the Swift team is the final arbiter of how Swift authorization should behave. All signs point to the Swift repo being the best place for these modules to be maintained. Does this seem reasonable, or is there a better alternative? Please chime in, especially if you are Swift core. Cheers, Maru ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
