On Thu, 2014-02-27 at 17:02 -0500, Adam Young wrote: > On 02/27/2014 12:52 PM, Jay Pipes wrote: > > On Thu, 2014-02-27 at 16:13 +0000, Henry Nash wrote: > >> So a couple of things about this: > >> > >> > >> 1) Today (and also true for Grizzly and Havana), the user can chose > >> what LDAP attribute should be returned as the user or group ID. So it > >> is NOT a safe assumption today (ignoring any support for > >> domain-specific LDAP support) that the format of a user or group ID is > >> a 32 char UUID. Quite often, I would think, that email address would > >> be chosen by a cloud provider as the LDAP id field, by default we use > >> the CN. Since we really don't want to ever change the user or group > >> ID we have given out from keystone for a particular entity, this means > >> we need to update nova (or anything else) that has made a 32 char > >> assumption. > > I don't believe this is correct. Keystone is the service that deals with > > authentication. As such, Keystone should be the one and only one service > > that should have any need whatsoever to need to understand a non-UUID > > value for a user ID. The only value that should ever be communicated > > *from* Keystone should be the UUID value of the user. > > We never said it was a UUID, its only been that in the "allocated by > Keystone" cases thus far.
Every user record in Keystone should have an "allocated by Keystone" user ID that is a UUID value. Why? Because every single other OpenStack service uses UUID values for identifiers (not names), and Keystone should be no different. > userids are blobs....well, character blocks, and are not expected to be > interprested by the end services, but also not confined to be 32 > characters long. I would argue that they absolutely *should* be confined to 32 characters (or technically, 16 bytes, which is the size of a UUID). > Most LDAP use cases do not allow us to write data back the Directory. > Federation has the same use cases. Frankly, I don't care :) Let's say Super-Duper Identity Driver has 100 character IDs natively. Does that mean that these IDs need be sent all over the OpenStack ecosystem? No. Have a translation table in Keystone and spread the UUID user ID throughout OpenStack. > That means that we are going to end > up with a slew of ephemeral data in Keystone for no real value. Not sure why you think this is ephemeral data... What is ephemeral about a non-UUID identifier from another auth system (federated or not) that is mapped to a Keystone user UUID? If you look at virtually every flexible authentication mechanism in virtually all web frameworks out there, you will see something very similar, where there is a table that maps external IDs to the IDs that are native to some platform. Best, -jay > > > > If the Keystone service uses LDAP or federation for alternative > > authentication schemes, then Keystone should have a mapping table that > > translates those elongated and non-UUID identifiers values (email > > addresses, LDAP CNs, etc) into the UUID value that is then communicated > > to all other OpenStack services. > The short of it is that we need to define a max userid length and stick > to it. Thus far, we've had 64 characters in Keystone, and I think we > can stick with that. Nova needs to expand its view to 64 chars. > > The rest is Keystone internals to deal with. > > > > > Best, > > -jay > > > >> 2) In oder to support the ability for service providers to be able to > >> have the identity part of keystone be satisfied by a customer LDAP > >> (i.e. for a given domain, have a specific LDAP), then, as has been > >> stated, we need to subsequently, when handed an API call with just a > >> user or group ID, be able to "route" this call to the correct LDAP. > >> Trying to keep true to the openstack design principles, we had > >> planned to encode a domain identifier into the user or group ID - i.e. > >> distribute the data to where it is needed, in ortherwords, the user > >> and group ID provide all the info we need to route the call to the > >> right place. Two implementations come to mind: > >> 2a) Simply concatenate the user/group ID with the domain_id, plus some > >> separator and make a composite public facing ID. e.g. > >> "user_entity_id@@UUID_of_domain". This would have a technical maximum > >> size of 64+2+64 (i.e. 130), although in reality since we control > >> domain_id and we know it is always 32 char UUID - in fact the max size > >> would be 98. This has the problem of increasing the size of the > >> public facing field beyond the existing 64. This is what we had > >> planned for IceHouse - and is currently in review. > >> 2b) Use a similar concatenation idea as 2a), but limit the total size > >> to the existing 64. Since we control domain_id, we could (internally > >> and not visibly to the outside world), create a domain_index, that was > >> used in place of domain_id in the publicly visible field, to minimize > >> the number of chars it requires. So the public facing composite ID > >> might be something like <up to 54 chars of entity_id>@@<8 chars of > >> domain_index>. There is a chance, of course, that the 54 char > >> restriction might be problematic for LDAP users....but I doubt it. We > >> would make that a restriction and if it really became a problem, we > >> could consider a field size increase at a later release > >> 3) The alternative to 2a and 2b is to have, as had been suggested, an > >> internal mapping table that maps external facing entity_ids to a > >> domain plus local entity ID. The problem with this idea is that: > >> - This could become a very big table (you will essentially have an > >> entry for every user in every corporate LDAP that has accessed a given > >> openstack) > >> - Since most LDAPs are RO, we will never see deletes...so we won't > >> know when (without some kind of garbage collection) to cull entries > >> - It obviously does not solve 1) - since existing LDAP support can > >> break the 32 char limit - and so it isn't true that this mapping table > >> causes all public facing entity IDs to be simple 32 char UUIDs > >> > >> > >> From a delivery into IceHouse point of view any of the above are > >> possible, since the actual mapping used is relatively small part of > >> the patch. I personally favor 2b), since it is simple, has "less > >> moving parts" and does not change any external facing requirements for > >> storage of user and group IDs (above and beyond what is true today). > >> > >> > >> Henry > >> On 27 Feb 2014, at 03:46, Adam Young <ayo...@redhat.com> wrote: > >> > >>> On 02/26/2014 08:25 AM, Dolph Mathews wrote: > >>> > >>>> On Tue, Feb 25, 2014 at 2:38 PM, Jay Pipes <jaypi...@gmail.com> > >>>> wrote: > >>>> On Tue, 2014-02-25 at 11:47 -0800, Morgan Fainberg wrote: > >>>> > For purposes of supporting multiple backends for > >>>> Identity (multiple > >>>> > LDAP, mix of LDAP and SQL, federation, etc) Keystone is > >>>> planning to > >>>> > increase the maximum size of the USER_ID field from an > >>>> upper limit of > >>>> > 64 to an upper limit of 255. This change would not > >>>> impact any > >>>> > currently assigned USER_IDs (they would remain in the > >>>> old simple UUID > >>>> > format), however, new USER_IDs would be increased to > >>>> include the IDP > >>>> > identifier (e.g. USER_ID@@IDP_IDENTIFIER). > >>>> > >>>> > >>>> -1 > >>>> > >>>> I think a better solution would be to have a simple > >>>> translation table > >>>> only in Keystone that would store this longer identifier > >>>> (for folks > >>>> using federation and/or LDAP) along with the Keystone user > >>>> UUID that is > >>>> used in foreign key relations and other mapping tables > >>>> through Keystone > >>>> and other projects. > >>>> > >>>> > >>>> Morgan and I talked this suggestion through last night and agreed > >>>> it's probably the best approach, and has the benefit of zero > >>>> impact on other services, which is something we're obviously > >>>> trying to avoid. I imagine it could be as simple as a user_id to > >>>> domain_id lookup table. All we really care about is "given a > >>>> globally unique user ID, which identity backend is the user from?" > >>>> > >>>> > >>>> On the downside, it would likely become bloated with unused > >>>> ephemeral user IDs, so we'll need enough metadata about the > >>>> mapping to implement a purging behavior down the line. > >>> UUIDs are 32 chars long. Its really just uuid@@uuid that pushes us > >>> over the 64 character limit. > >>> If we can shorten up the IDP_ID we can fit everything in 64 chars > >>> (which means only Nova needs to expand its column size) > >>> > >>> What if we enumerated IDPs by index, from 10000000 to 99999999 or > >>> something comparable, and then use the new domain_index (or prot > >>> domain id to not be a uuid). Then the above scheme would work and > >>> no migration would be required. > >>> > >>> > >>>> > >>>> > >>>> The only identifiers that would ever be communicated to > >>>> any non-Keystone > >>>> OpenStack endpoint would be the UUID user and tenant IDs. > >>>> > >>>> > There is the obvious concern that projects are utilizing > >>>> (and storing) > >>>> > the user_id in a field that cannot accommodate the > >>>> increased upper > >>>> > limit. Before this change is merged in, it is important > >>>> for the > >>>> > Keystone team to understand if there are any places that > >>>> would be > >>>> > overflowed by the increased size. > >>>> > >>>> > >>>> I would go so far as to say the user_id and tenant_id > >>>> fields should be > >>>> *reduced* in size to a fixed 16-char BINARY or 32-char > >>>> CHAR field for > >>>> performance reasons. Lengthening commonly-used and > >>>> frequently-joined > >>>> identifier fields is not a good option, IMO. > >>>> > >>>> Best, > >>>> -jay > >>>> > >>>> > The review that would implement this change in size > >>>> > is https://review.openstack.org/#/c/74214 and is > >>>> actively being worked > >>>> > on/reviewed. > >>>> > > >>>> > > >>>> > I have already spoken with the Nova team, and a single > >>>> instance has > >>>> > been identified that would require a migration (that > >>>> will have a fix > >>>> > proposed for the I3 timeline). > >>>> > > >>>> > > >>>> > If there are any other known locations that would have > >>>> issues with an > >>>> > increased USER_ID size, or any concerns with this change > >>>> to USER_ID > >>>> > format, please respond so that the issues/concerns can > >>>> be addressed. > >>>> > Again, the plan is not to change current USER_IDs but > >>>> that new ones > >>>> > could be up to 255 characters in length. > >>>> > > >>>> > > >>>> > Cheers, > >>>> > Morgan Fainberg > >>>> > — > >>>> > Morgan Fainberg > >>>> > Principal Software Engineer > >>>> > Core Developer, Keystone > >>>> > m...@metacloud.com > >>>> > > >>>> > > >>>> > >>>> > _______________________________________________ > >>>> > OpenStack-dev mailing list > >>>> > OpenStack-dev@lists.openstack.org > >>>> > > >>>> > >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>>> > >>>> > >>>> > >>>> _______________________________________________ > >>>> OpenStack-dev mailing list > >>>> OpenStack-dev@lists.openstack.org > >>>> > >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>>> > >>>> > >>>> > >>>> > >>>> _______________________________________________ > >>>> OpenStack-dev mailing list > >>>> OpenStack-dev@lists.openstack.org > >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>> > >>> _______________________________________________ > >>> OpenStack-dev mailing list > >>> OpenStack-dev@lists.openstack.org > >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > >> _______________________________________________ > >> OpenStack-dev mailing list > >> OpenStack-dev@lists.openstack.org > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > _______________________________________________ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev