Almost everything is working... Only 1 tenant doesn't work anymore, its instances have IP but it can not ping the gateway (Neutron L3 unreachable only for 1 Tenant, very weird), others tenants are working as expected...
I'll keep testing it... Cheers! On 8 March 2014 21:07, Martinx - ジェームズ <thiagocmarti...@gmail.com> wrote: > Guys, > > Please, ignore my last message... I'm thinking that I'm facing a different > problem, out off this topic. > > Now, instances can not ping its gateway (Neutron L3 is unreachable). I'll > make a different post if necessary. > > Sorry about the buzz. > > Tks, > Thiago > > > On 8 March 2014 16:53, Martinx - ジェームズ <thiagocmarti...@gmail.com> wrote: > >> Well, I think I just destroyed the entire Cloud Computing environment... >> Security Groups isn't working, it is impossible to reach the tenant's >> subnet, they are just closed (i.e. not working), all sites hosted here, are >> down. Damn... What a mess... :-/ >> >> Is there any way to try to fix this? I have the dump of keystone db, >> before dropping the INDEXes... >> >> >> On 8 March 2014 16:12, Martinx - ジェームズ <thiagocmarti...@gmail.com> wrote: >> >>> Stackers, >>> >>> I'll mark this as fixed, look: >>> >>> --- >>> root@controller-1:~# apt-get -f install >>> Reading package lists... Done >>> Building dependency tree >>> Reading state information... Done >>> 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. >>> 1 not fully installed or removed. >>> After this operation, 0 B of additional disk space will be used. >>> *Setting up keystone (1:2013.2.2-0ubuntu1~cloud0) ...* >>> --- >>> >>> Keystone seems to be upgraded now... >>> >>> Thanks! >>> Thiago >>> >>> >>> On 8 March 2014 16:06, Martinx - ジェームズ <thiagocmarti...@gmail.com>wrote: >>> >>>> Well, >>>> >>>> I just did a shot in the dark: >>>> >>>> >>>> > ALTER TABLE `keystone`.`token` DROP INDEX `ix_token_expires_valid`; >>>> >>>> And "keystone-manage db_sync" seems to be running again... But, I have >>>> no idea if this DROP I did will break my OpenStack, or not... >>>> >>>> I don't want to waste more time from you guys with this but, maybe it >>>> is not a big deal... What do you guys think?! >>>> >>>> Thanks! >>>> Thiago >>>> >>>> >>>> On 8 March 2014 15:52, Martinx - ジェームズ <thiagocmarti...@gmail.com>wrote: >>>> >>>>> Hi! >>>>> >>>>> I'm seeing some progress here, look: >>>>> >>>>> --- >>>>> root@controller-1:~# pt-duplicate-key-checker --host localhost >>>>> --databases=keystone --ask-pass >>>>> Enter password: >>>>> # >>>>> ######################################################################## >>>>> # keystone.token >>>>> >>>>> # >>>>> ######################################################################## >>>>> >>>>> # ix_token_expires is a left-prefix of ix_token_expires_valid >>>>> # Key definitions: >>>>> # KEY `ix_token_expires` (`expires`), >>>>> # KEY `ix_token_expires_valid` (`expires`,`valid`) >>>>> # Column types: >>>>> # `expires` datetime default null >>>>> # `valid` tinyint(1) not null >>>>> # To remove this duplicate index, execute: >>>>> ALTER TABLE `keystone`.`token` DROP INDEX `ix_token_expires`; >>>>> >>>>> # >>>>> ######################################################################## >>>>> # Summary of indexes >>>>> >>>>> # >>>>> ######################################################################## >>>>> >>>>> # Size Duplicate Indexes 1321470 >>>>> # Total Duplicate Indexes 1 >>>>> # Total Indexes 42 >>>>> ---- >>>>> >>>>> --- >>>>> mysql> ALTER TABLE `keystone`.`token` DROP INDEX `ix_token_expires`; >>>>> Query OK, 0 rows affected (1.03 sec) >>>>> Records: 0 Duplicates: 0 Warnings: 0 >>>>> --- >>>>> >>>>> --- >>>>> No more duplicate key found by "pt-duplicate-key-checker" but, same >>>>> problem with "keystone-manage "db_sync", look: >>>>> >>>>> root@controller-1:~# keystone-manage db_sync >>>>> 2014-03-08 15:46:59.186 8043 CRITICAL keystone [-] (OperationalError) >>>>> (1061, "Duplicate key name 'ix_token_expires_valid'") 'CREATE INDEX >>>>> ix_token_expires_valid ON token (expires, valid)' () >>>>> --- >>>>> >>>>> Guys, what can I do!? I really need to keep this OpenStack >>>>> installation intact, I can not afford losing it... >>>>> >>>>> Tks! >>>>> Thiago >>>>> >>>>> >>>>> On 8 March 2014 04:39, Martinx - ジェームズ <thiagocmarti...@gmail.com>wrote: >>>>> >>>>>> Lance, >>>>>> >>>>>> I think I know what had happened... The Keystone upgrade process died >>>>>> in the middle of the upgrade. >>>>>> >>>>>> Now, the Keystone SQL database is stucked at some point "in the >>>>>> middle" of both versions... >>>>>> >>>>>> My "keystone" MySQL db dump is huge, more than >1G... >>>>>> >>>>>> Any tips about how to fix this?! >>>>>> >>>>>> I really appreciate any help! >>>>>> >>>>>> Tks! >>>>>> Thiago >>>>>> >>>>>> >>>>>> On 8 March 2014 02:55, Martinx - ジェームズ <thiagocmarti...@gmail.com>wrote: >>>>>> >>>>>>> Lance, >>>>>>> >>>>>>> I can provide everything, no problem... I'm doing this to reproduce >>>>>>> it: >>>>>>> >>>>>>> >>>>>>> root@controller-1:~# keystone-manage db_sync >>>>>>> 2014-03-08 02:44:46.723 5407 CRITICAL keystone [-] >>>>>>> (OperationalError) (1061, "Duplicate key name 'ix_token_expires_valid'") >>>>>>> 'CREATE INDEX ix_token_expires_valid ON token (expires, valid)' () >>>>>>> >>>>>>> >>>>>>> Log shows: >>>>>>> >>>>>>> ==> /var/log/keystone/keystone.log <== >>>>>>> 2014-03-08 02:44:46.723 5407 CRITICAL keystone [-] >>>>>>> (OperationalError) (1061, "Duplicate key name 'ix_token_expires_valid'") >>>>>>> 'CREATE INDEX ix_token_expires_valid ON token (expires, valid)' () >>>>>>> >>>>>>> >>>>>>> root@controller-1:~# dpkg -l | grep keystone >>>>>>> iF keystone 1:2013.2.2-0ubuntu1~cloud0 >>>>>>> OpenStack identity service - Daemons >>>>>>> ii python-keystone 1:2013.2.2-0ubuntu1~cloud0 >>>>>>> OpenStack identity service - Python library >>>>>>> ii python-keystoneclient 1:0.3.2-0ubuntu1~cloud0 >>>>>>> Client library for OpenStack Identity API >>>>>>> >>>>>>> ... >>>>>>> >>>>>>> If I run "apt-get -f install", to try to fix it ("iF" dpkg status >>>>>>> above), same problem, look: >>>>>>> >>>>>>> --- >>>>>>> root@controller-1:~# apt-get -f install >>>>>>> Reading package lists... Done >>>>>>> Building dependency tree >>>>>>> Reading state information... Done >>>>>>> 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. >>>>>>> 1 not fully installed or removed. >>>>>>> After this operation, 0 B of additional disk space will be used. >>>>>>> Setting up keystone (1:2013.2.2-0ubuntu1~cloud0) ... >>>>>>> 2014-03-08 02:49:11.050 5469 CRITICAL keystone [-] >>>>>>> (OperationalError) (1061, "Duplicate key name 'ix_token_expires_valid'") >>>>>>> 'CREATE INDEX ix_token_expires_valid ON token (expires, valid)' () >>>>>>> dpkg: error processing keystone (--configure): >>>>>>> subprocess installed post-installation script returned error exit >>>>>>> status 1 >>>>>>> Errors were encountered while processing: >>>>>>> keystone >>>>>>> E: Sub-process /usr/bin/dpkg returned an error code (1) >>>>>>> --- >>>>>>> >>>>>>> I'm lost... :-/ >>>>>>> >>>>>>> Maybe the upgrade procedure got stucked, then the second time I >>>>>>> tried, it triggered this error... I'm not sure... >>>>>>> >>>>>>> Tks! >>>>>>> Thiago >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 8 March 2014 01:55, Lance D Bragstad <ldbra...@us.ibm.com> wrote: >>>>>>> >>>>>>>> Thiago, >>>>>>>> >>>>>>>> Are you able to provide a little more information? Log files maybe? >>>>>>>> What are the steps you're taking to recreate the issue? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Best Regards, >>>>>>>> >>>>>>>> Lance Bragstad >>>>>>>> ldbra...@us.ibm.com >>>>>>>> >>>>>>>> Martinx - ジェームズ <thiagocmarti...@gmail.com> wrote on 03/07/2014 >>>>>>>> 08:34:11 PM: >>>>>>>> >>>>>>>> >>>>>>>> > From: Martinx - ジェームズ <thiagocmarti...@gmail.com> >>>>>>>> >>>>>>>> > To: Lance D Bragstad/Rochester/IBM@IBMUS, >>>>>>>> > Cc: "openstack@lists.openstack.org" < >>>>>>>> openstack@lists.openstack.org> >>>>>>>> > Date: 03/07/2014 08:34 PM >>>>>>>> > Subject: Re: [Openstack] Error on upgrading to latest Keystone - >>>>>>>> > Ubuntu 12.04.4 + UCA >>>>>>>> > >>>>>>>> > Thanks Lance! >>>>>>>> >>>>>>>> > >>>>>>>> > I just did: "keystone-manage db_sync" and the output is the same: >>>>>>>> > >>>>>>>> > --- >>>>>>>> > 2014-03-07 23:23:48.564 2258 CRITICAL keystone [-] >>>>>>>> > (OperationalError) (1061, "Duplicate key name >>>>>>>> > 'ix_token_expires_valid'") 'CREATE INDEX ix_token_expires_valid >>>>>>>> ON >>>>>>>> > token (expires, valid)' () >>>>>>>> > --- >>>>>>>> > >>>>>>>> > I just read [1] and [2] but, not a clue about how to fix it. >>>>>>>> > >>>>>>>> > My Keystone is still running and I have no idea if I can restart >>>>>>>> it >>>>>>>> > or not... What if it doesn't come back?! My entire Cloud will >>>>>>>> become >>>>>>>> > useless... This is a production environment, it was working >>>>>>>> smoothly >>>>>>>> > for months... :'( >>>>>>>> > >>>>>>>> > Do you know if there is something that I can do directly on MySQL >>>>>>>> to >>>>>>>> > try to fix it?! >>>>>>>> > >>>>>>>> > Best, >>>>>>>> > Thiago >>>>>>>> > >>>>>>>> >>>>>>>> > On 7 March 2014 23:18, Lance D Bragstad <ldbra...@us.ibm.com> >>>>>>>> wrote: >>>>>>>> > Thiago, >>>>>>>> > >>>>>>>> > Your database might be out of sync with the latest Keystone code >>>>>>>> if >>>>>>>> > you just performed an update. Try migrating your database [1]. >>>>>>>> The >>>>>>>> > link gives some information on how to do that, but you'll >>>>>>>> > essentially want to do a `keystone-manage db_sync`, also check >>>>>>>> the >>>>>>>> > man pages [2]. This should ensure your Keystone database has the >>>>>>>> > proper schema for the Keystone code you're running. >>>>>>>> > >>>>>>>> > [1] http://docs.openstack.org/developer/keystone/ >>>>>>>> > configuration.html#step-2-sync-your-new-empty-database >>>>>>>> > [2] >>>>>>>> http://docs.openstack.org/developer/keystone/man/keystone-manage.html >>>>>>>> > >>>>>>>> > Best Regards, >>>>>>>> > >>>>>>>> > Lance Bragstad >>>>>>>> > ldbra...@us.ibm.com >>>>>>>> > >>>>>>>> > Martinx - ジェームズ <thiagocmarti...@gmail.com> wrote on 03/07/2014 >>>>>>>> > 06:50:01 PM: >>>>>>>> > >>>>>>>> > > From: Martinx - ジェームズ <thiagocmarti...@gmail.com> >>>>>>>> > > To: "openstack@lists.openstack.org" < >>>>>>>> openstack@lists.openstack.org>, >>>>>>>> > > Date: 03/07/2014 07:03 PM >>>>>>>> > > Subject: [Openstack] Error on upgrading to latest Keystone - >>>>>>>> Ubuntu >>>>>>>> > > 12.04.4 + UCA >>>>>>>> > >>>>>>>> > > >>>>>>>> > > Guys, >>>>>>>> > > >>>>>>>> > > I just did: >>>>>>>> > > >>>>>>>> > > apt-get update >>>>>>>> > > apt-get dist-upgrade >>>>>>>> > > >>>>>>>> > > And I'm seeing the following error: >>>>>>>> > > >>>>>>>> > > --- >>>>>>>> > > Setting up keystone (1:2013.2.2-0ubuntu1~cloud0) ... >>>>>>>> > > 2014-03-07 21:45:39.473 15540 CRITICAL keystone [-] >>>>>>>> > > (OperationalError) (1061, "Duplicate key name >>>>>>>> > > 'ix_token_expires_valid'") 'CREATE INDEX ix_token_expires_valid >>>>>>>> ON >>>>>>>> > > token (expires, valid)' () >>>>>>>> > > dpkg: error processing keystone (--configure): >>>>>>>> > > subprocess installed post-installation script returned error >>>>>>>> exit status 1 >>>>>>>> > > Errors were encountered while processing: >>>>>>>> > > keystone >>>>>>>> > > E: Sub-process /usr/bin/dpkg returned an error code (1) >>>>>>>> > > --- >>>>>>>> > > >>>>>>>> > > What can I do?! >>>>>>>> > > >>>>>>>> > > Tks! >>>>>>>> > > Thiago_______________________________________________ >>>>>>>> > > Mailing list: >>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>>>>>> > > Post to : openstack@lists.openstack.org >>>>>>>> > > Unsubscribe : >>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack