On 16/07/2013, at 1:10 AM, Adam Young <ayo...@redhat.com> wrote: > On 07/15/2013 04:06 AM, Kieran Spear wrote: >> Hi all, >> >> I want to backport the fix for the "Token List in Memcache can consume >> an entire memcache page" bug[1] to Grizzly, but I had a couple of >> questions: >> >> 1. Why do we need to store the entire token data in the >> usertoken-<userid> key? This data always seems to be hashed before >> indexing into the 'token-<tokenid>' keys anyway. The size of the >> memcache data for a user's token list currently grows by 4k every time >> a new PKI token is created. It doesn't take long to hit 1MB at this >> rate even with the above fix. > Yep. The reason, though, is that we either take a memory/storage hit (store > the whole token) or a performance hit (reproduce the token data) and we've > gone for the storage hit.
In this case it looks like we're taking a hit from both, since the PKI token "id" from the user token index is retrieved, then hashed and then that key is used to retrieve the token from the "tokens-%s" page anyway. > > >> >> 2. Every time it creates a new token, Keystone loads each token from >> the user's token list with a separate memcache call so it can throw it >> away if it's expired. This seems excessive. Is it anything to worry >> about? If it just checked the first two tokens you'd get the same >> effect on a longer time scale. >> >> I guess part of the answer is to decrease our token expiry time, which >> should mitigate both issues. Failing that we'd consider moving to the >> SQL backend. > HOw about doing both? But if you move to the sql backend, rememeber to > periodically clean up the token table, or you will have storage issues there > as well. No silver bullet, I am afraid. I think we're going to stick with memcache for now (the devil we know :)). With (1) and (2) fixed and the token expiration time tweaked I think memcache will do okay. Kieran > >> >> Cheers, >> Kieran >> >> [1] https://bugs.launchpad.net/keystone/+bug/1171985 >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
