[openstack-dev] [keystone] Case sensitivity & backend databases
Hi Do we specify somewhere whether text field matching in the API is case sensitive or in-sensitive? I'm thinking about filters, as well as user and domain names in authentication. I think our current implementation will always be case sensitive for filters (since we do that in python and do not, yet, pass the filters to the backends), while authentication will reflect the "case sensitivity or lack thereof" of the underlying database. I believe that MySQL is case in-sensitive by default, while Postgres, sqllite and others are case-sensitive by default. If using an LDAP backend, then I think this is case-sensitive. The above seems to be inconsistent. It might become even more so when we pass the filters to the backend. Given that other projects already pass filters to the backend, we may also have inter-project inconsistencies that bleed through to the user experience. Should we make at least a recommendation that the backend should case-sensitive (you can configure MySQL to be so)? Insist on it? Ignore it and keep things as they are? Henry ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Case sensitivity & backend databases
Resurrecting this thread. . . I think I'm misunderstanding where we landed on this issue. On the one hand, it seems like there are tests to assert that uniqueness of names is case-sensitive. On the other, some folks have identified reasons why they would want case-insensitivity on uniqueness checks for creating new users. Still others I think have wisely pointed out that we should probably get out of the business of creating users. Trying to incorporate all of these perspectives, I propose the following: 1) We add a configuration option to just the keystone sql identity driver to force case-sensitivity on uniqueness checks. I'm pretty sure there is a way to do this in sqlalchemy, basically whatever is equivalent to 'SELECT * FROM user WHERE BINARY name = %s'. This config option would only affect create_user and update_user. 2) We always force case-sensitive comparison for get_user_by_name, using a similar mechanism as above. By focusing on changes to queries we needn't bother with a migration and can make the behavior a deployer choice. Is this a bad goal or approach? IANADBA, markwash ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Case sensitivity & backend databases
On Wed, Sep 25, 2013 at 3:45 AM, Henry Nash wrote: > Hi > > Do we specify somewhere whether text field matching in the API is case > sensitive or in-sensitive? I'm thinking about filters, as well as user and > domain names in authentication. I think our current implementation will > always be case sensitive for filters (since we do that in python and do > not, yet, pass the filters to the backends), while authentication will > reflect the "case sensitivity or lack thereof" of the underlying database. > I believe that MySQL is case in-sensitive by default, while Postgres, > sqllite and others are case-sensitive by default. If using an LDAP > backend, then I think this is case-sensitive. > > The above seems to be inconsistent. It might become even more so when we > pass the filters to the backend. Given that other projects already pass > filters to the backend, we may also have inter-project inconsistencies that > bleed through to the user experience. Should we make at least a > recommendation that the backend should case-sensitive (you can configure > MySQL to be so)? Insist on it? Ignore it and keep things as they are? Relevant bug: https://bugs.launchpad.net/keystone/+bug/1229093 I've never heard anyone complain that "keystone is case sensitive but I expect it to be case-insensitive." There is a clear expectation / assumption that we are case sensitive, and I'd like to assert that to be true in general, across driver implementations. However, there are times when you might want to make case-insensitive comparisons, but that should be hidden from the user. For example: create_user(name='dolph', password='asdf') # should succeed create_user(name='Dolph' password='asdf') # should fail, because a case-insensitive equivalent already exists authenticate(name='Dolph', password='asdf') # should fail, because the user that exists was created with an lowercase name get_user_by_name(name='Dolph') # should fail, because the user that exists was created with an lowercase name (until we get into case-sensitivity hints for the driver) We don't provide for the behavior illustrated in the second line of keystone today, but I think it's worth considering (it's not trivial with UTF-8 support, though). > Henry > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Case sensitivity & backend databases
On Wed, Sep 25, 2013 at 09:45:32AM +0100, Henry Nash wrote:
> Hi
>
> Do we specify somewhere whether text field matching in the API is case
> sensitive or in-sensitive? I'm thinking about filters, as well as user and
> domain names in authentication. I think our current implementation will
> always be case sensitive for filters (since we do that in python and do not,
> yet, pass the filters to the backends), while authentication will reflect the
> "case sensitivity or lack thereof" of the underlying database. I believe
> that MySQL is case in-sensitive by default, while Postgres, sqllite and
> others are case-sensitive by default.
> If using an LDAP backend, then I think this is case-sensitive.
That heavily depends on which LDAP Attributes you use. The usual suspects user
for authentication: "uid", "cn" and keystone's default "sn" are all defined to
be case-insensitive.
> The above seems to be inconsistent. It might become even more so when we
> pass the filters to the backend. Given that other projects already pass
> filters to the backend, we may also have inter-project inconsistencies that
> bleed through to the user experience. Should we make at least a
> recommendation that the backend should case-sensitive (you can configure
> MySQL to be so)? Insist on it? Ignore it and keep things as they are?
For LDAP it will be pretty hard to enforce any particular matching. Especially
if we want to support people using their existing LDAP servers as a backend for
keystone. You could of course enforce this on the LDAP client side (i.e. inside
keystone's LDAP backend), but I am not sure if that is really a good idea. It
might have a negative impact once filtering is implemented in the LDAP backend.
As Dolph already suggested we should not allow usernames that just differ in
capitalization ("JDoe" vs. "jdoe") to co-exist. (Which could be an argument
for handling users case-insensitive in general)
--
Ralf
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Case sensitivity & backend databases
On Thu, Sep 26, 2013 at 4:44 AM, Ralf Haferkamp wrote:
>
> As Dolph already suggested we should not allow usernames that just differ
> in
> capitalization ("JDoe" vs. "jdoe") to co-exist. (Which could be an
> argument
> for handling users case-insensitive in general)
>
This enforcement should be handled by the LDAP server if the organization
thinks it's important to have users with names unique without respect for
capitalization. LDAP servers can also enforce normal security enhancers
like password strength, expiration, and locking out users after invalid
logins that the SQL backend doesn't support.
My recommendation is that Keystone should get away from dealing with
creating/updating users to avoid reinventing the wheel (and making a wheel
that's missing bells and whistles). If comparing user names is a problem,
let's limit it to our custom SQL backend and not let it spread to other
more featureful backends.
- Brant
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Case sensitivity & backend databases
On Thu, Sep 26, 2013 at 11:02 AM, Brant Knudson wrote:
>
> On Thu, Sep 26, 2013 at 4:44 AM, Ralf Haferkamp wrote:
>
>>
>> As Dolph already suggested we should not allow usernames that just differ
>> in
>> capitalization ("JDoe" vs. "jdoe") to co-exist. (Which could be an
>> argument
>> for handling users case-insensitive in general)
>>
>
> This enforcement should be handled by the LDAP server if the organization
> thinks it's important to have users with names unique without respect for
> capitalization. LDAP servers can also enforce normal security enhancers
> like password strength, expiration, and locking out users after invalid
> logins that the SQL backend doesn't support.
>
> My recommendation is that Keystone should get away from dealing with
> creating/updating users to avoid reinventing the wheel (and making a wheel
> that's missing bells and whistles). If comparing user names is a problem,
> let's limit it to our custom SQL backend and not let it spread to other
> more featureful backends.
>
++; this confusion specifically stems from keystone's implementation
against SQL, where keystone manages users directly
>
>
> - Brant
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
--
-Dolph
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Case sensitivity & backend databases
Excerpts from Henry Nash's message of 2013-09-25 01:45:32 -0700: > Hi > > Do we specify somewhere whether text field matching in the API is case > sensitive or in-sensitive? I'm thinking about filters, as well as user and > domain names in authentication. I think our current implementation will > always be case sensitive for filters (since we do that in python and do not, > yet, pass the filters to the backends), while authentication will reflect the > "case sensitivity or lack thereof" of the underlying database. I believe > that MySQL is case in-sensitive by default, while Postgres, sqllite and > others are case-sensitive by default. If using an LDAP backend, then I think > this is case-sensitive. > > The above seems to be inconsistent. It might become even more so when we > pass the filters to the backend. Given that other projects already pass > filters to the backend, we may also have inter-project inconsistencies that > bleed through to the user experience. Should we make at least a > recommendation that the backend should case-sensitive (you can configure > MySQL to be so)? Insist on it? Ignore it and keep things as they are? The collation controls case sensitivity. The default collations in MySQL are all case in-sensitive, and this is a good thing for many reasons. I don't want a user "spamaps" on the same domain where I have "SpamapS". If you want to force it, you can do so at many levels, from the server down to the column. utf8_bin would be the case-sensitive collation to use. But that is not actually what you want. The problem and the part where programmers get surprised by this is that we are calling fields VARCHAR when we mean VARBINARY. Strings that always need to be 100% identical and are not for human consumption, should be _BINARY_. In the keystone user table, id, extra, password, and domain_id are all "varchar". This is a common mistake in SQL column design. Since user is a utf8 table, the index on id varchar(64) is 64*3 bytes wide. This is because we have to reserve enough space for 64 UTF-8 characters (and MySQL only does up to 3-byte UTF8, really obscure UTF-8 can be as long as 6 bytes!). We don't intend to interpret this as a human ever. So, this should be VARBINARY(64) everywhere it is used. The change would have several effects: 1) Indexes that mention the field would shrink by 128 bytes per key. 2) One could have two rows with identical hex values in id that varied only by case. -- This is not an actual problem, but an effect of the change. 3) Sorting by this field will now just use the binary value of each character, not the language collation. When do you sort by a hex id? Anyway, doing this on all ID fields and obviously-not-utf8-containing fields will have a net effect of making the database leaner, so I think it is worth a wide spread effort not just in keystone but in all of OpenStack. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Case sensitivity & backend databases
On Thu, Sep 26, 2013 at 11:02:22AM -0500, Brant Knudson wrote: > On Thu, Sep 26, 2013 at 4:44 AM, Ralf Haferkamp wrote: [..] > My recommendation is that Keystone should get away from dealing with > creating/updating users to avoid reinventing the wheel (and making a wheel > that's missing bells and whistles). I agree. People who want to use Keystone with the LDAP Backend (or any other than the custom SQL backend) will very likely already have the tools and processes in place to manage their users. > If comparing user names is a problem, let's limit it to our custom SQL > backend and not let it spread to other more featureful backends. Yes, that sounds like a good idea. -- Ralf ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
