[Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
Addressed by the following patch: https://review.openstack.org/#/c/79105/ ** Changed in: openstack-manuals Status: New => Fix Released ** Changed in: openstack-manuals Milestone: None => icehouse -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
Since it's not already mentioned in this bug, the long term solution here is to simply not persist tokens at all: https://blueprints.launchpad.net/keystone/+spec/ephemeral-pki-tokens ** Also affects: openstack-manuals Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
That seems reasonable. I would suggest adding that as a step to the OpenStack setup documentation. No one has infinite database space, eventually all used OpenStack installations will suffer unless this is done. Regardless of what component (or person) has to complete the task, they'll need to be aware of it. I would prefer to see some kind of "automatically expire old tokens" configuration option so that the maintenance of keystone stays in keystone - whether that be implemented as a expire-on-access query or cron-derived mechanism, but re-implementing cron is not great and a purge tagged on to other operations could cause a slowdown on those operations. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
On 04/05/2014 01:43 PM, Andrew Mann wrote: > A CLI command is an interesting stopgap, but on a heavily utilized > OpenStack installation with automated tools operating against OpenStack, > this has a high manual maintenance cost. Surely there is some better > default that lies in the middle ground between keeping tokens for ever > and ever and requiring a manual removal of tokens? > > As a reference point, I wasn't even aware this was an issue, until one > of our test deployments of grizzly using a limited IO system started > acting horribly (30 second response times). After tracing the problem > from nova to keystone to mysql, I found a 442,000 row token table with >> 440,000 expired tokens. I went and checked our havana test on a > somewhat beefier system and found > 1M rows. > > This issue is a timebomb for any production OS install. > CRON the CLI job. There is no reason to try and integrate a scheduler into Keystone. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
A CLI command is an interesting stopgap, but on a heavily utilized OpenStack installation with automated tools operating against OpenStack, this has a high manual maintenance cost. Surely there is some better default that lies in the middle ground between keeping tokens for ever and ever and requiring a manual removal of tokens? As a reference point, I wasn't even aware this was an issue, until one of our test deployments of grizzly using a limited IO system started acting horribly (30 second response times). After tracing the problem from nova to keystone to mysql, I found a 442,000 row token table with >440,000 expired tokens. I went and checked our havana test on a somewhat beefier system and found > 1M rows. This issue is a timebomb for any production OS install. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
** Changed in: keystone (Ubuntu) Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
** Changed in: keystone Milestone: havana-1 => 2013.2 -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
** Changed in: keystone Importance: Undecided => Medium -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
** Changed in: keystone Status: Fix Committed => Fix Released ** Changed in: keystone Milestone: None => havana-1 -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
Reviewed: https://review.openstack.org/28133 Committed: http://github.com/openstack/keystone/commit/ff76a1b5cd3308cfb0ce936800364e27413ed946 Submitter: Jenkins Branch:master commit ff76a1b5cd3308cfb0ce936800364e27413ed946 Author: Jamie Lennox Date: Fri May 3 14:04:09 2013 +1000 Implement Token Flush via keystone-manage. Creates a cli entry 'token_flush' which removes all expired tokens. Fixes: bug 1032633 Implements: blueprint keystone-manage-token-flush Change-Id: I47eab99b577ff9e9ee74fee08e18fd07c4af5aad ** Changed in: keystone Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
Fix proposed to branch: master Review: https://review.openstack.org/28133 ** Changed in: keystone Status: Invalid => In Progress ** Changed in: keystone Assignee: (unassigned) => Jamie Lennox (jamielennox) -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
** Changed in: keystone (Ubuntu) Assignee: (unassigned) => David Höppner (0xffea) -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
Moving this issue to a BP: https://blueprints.launchpad.net/keystone/+spec/keystone-manage-token- flush ** Changed in: keystone Status: Confirmed => Invalid -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
** Changed in: keystone (Ubuntu) Importance: Undecided => Medium -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
** Changed in: keystone Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
** Also affects: keystone Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1032633] Re: Keystone's token table grows unconditionally when using SQL backend.
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: keystone (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to keystone in Ubuntu. https://bugs.launchpad.net/bugs/1032633 Title: Keystone's token table grows unconditionally when using SQL backend. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/keystone/+bug/1032633/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs