Are all of your instance_types tables identical across the cell/API DBs? And also marked is_public?
-AH On Mon, Nov 23, 2015 at 6:25 PM, Belmiro Moreira < [email protected]> wrote: > Hi Matt, > I'm rolling kilo 2015.1.1 > > After resizing an instance (state: verify_resize) I have the following in > the instance_extra table: > > API cell DB: > {"new": null, "old": null, "cur": {"nova_object.version": "1.1", > "nova_object.changes": ["root_gb", "name", "ephemeral_gb", "memory_mb", > "vcpus", "extra_specs", "swap", "rxtx_factor", "flavorid", "vcpu_weight", > "id"], "nova_object.name": "Flavor", "nova_object.data": {"root_gb": 20, > "name": "m1.small", "ephemeral_gb": 0, "memory_mb": 2048, "vcpus": 1, > "extra_specs": {}, "swap": 0, "rxtx_factor": 1.0, "flavorid": "2", > "vcpu_weight": null, "id": 5}, "nova_object.namespace": "nova"}} > > and in the compute cell DB: > {"new": {"nova_object.version": "1.1", "nova_object.changes": ["deleted", > "ephemeral_gb", "updated_at", "disabled", "extra_specs", "rxtx_factor", > "is_public", "flavorid", "id", "root_gb", "deleted_at", "name", > "created_at", "memory_mb", "vcpus", "swap", "vcpu_weight"], " > nova_object.name": "Flavor", "nova_object.data": {"disabled": false, > "root_gb": 40, "name": "m1.medium", "flavorid": "3", "deleted": false, > "created_at": null, "ephemeral_gb": 0, "updated_at": null, "memory_mb": > 4096, "vcpus": 2, "extra_specs": {}, "swap": 0, "rxtx_factor": 1.0, > "is_public": true, "deleted_at": null, "vcpu_weight": 0, "id": 1}, > "nova_object.namespace": "nova"}, "old": {"nova_object.version": "1.1", > "nova_object.changes": ["root_gb", "name", "ephemeral_gb", "memory_mb", > "vcpus", "extra_specs", "swap", "rxtx_factor", "flavorid", "vcpu_weight", > "id"], "nova_object.name": "Flavor", "nova_object.data": {"root_gb": 20, > "name": "m1.small", "ephemeral_gb": 0, "memory_mb": 2048, "vcpus": 1, > "extra_specs": {}, "swap": 0, "rxtx_factor": 1.0, "flavorid": "2", > "vcpu_weight": null, "id": 5}, "nova_object.namespace": "nova"}, "cur": > {"nova_object.version": "1.1", "nova_object.changes": ["deleted", > "ephemeral_gb", "updated_at", "disabled", "extra_specs", "rxtx_factor", > "is_public", "flavorid", "id", "root_gb", "deleted_at", "name", > "created_at", "memory_mb", "vcpus", "swap", "vcpu_weight"], " > nova_object.name": "Flavor", "nova_object.data": {"disabled": false, > "root_gb": 40, "name": "m1.medium", "flavorid": "3", "deleted": false, > "created_at": null, "ephemeral_gb": 0, "updated_at": null, "memory_mb": > 4096, "vcpus": 2, "extra_specs": {}, "swap": 0, "rxtx_factor": 1.0, > "is_public": true, "deleted_at": null, "vcpu_weight": 0, "id": 1}, > "nova_object.namespace": "nova"}} > > > The quota is calculated in the API cell and the new flavor information is > missing there. > I don't see any relevant/related logs using DEBUG. > > thanks, > Belmiro > > > On Sun, Nov 22, 2015 at 1:02 AM, Matt Riedemann < > [email protected]> wrote: > >> >> >> On 11/21/2015 3:47 PM, Belmiro Moreira wrote: >> >>> Hi, >>> We are about to upgrade nova to kilo using cells and we noticed >>> the resize/migrate functionality is not working properly. >>> >>> The instance is correctly resized/migrated but fails to >>> “confirm resize” with the following trace: >>> >>> 2015-11-21 22:40:49.804 26786 ERROR nova.api.openstack.wsgi >>> [req-67f6a225-602c-426c-a33a-02c3f5e3093d belmiro >>> 40c7729b-c8e5-432f-81d5-a41543c8bf13] Exception handling resource: >>> 'NoneType' object has no attribute '__getitem__' >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi Traceback >>> (most recent call last): >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi File >>> "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 821, >>> in _process_stack >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi >>> action_result = self.dispatch(meth, request, action_args) >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi File >>> "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 911, >>> in dispatch >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi return >>> method(req=request, **action_args) >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi File >>> "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", >>> line >>> 823, in _action_confirm_resize >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi >>> self.compute_api.confirm_resize(context, instance) >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi File >>> "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 233, in >>> wrapped >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi return >>> func(self, context, target, *args, **kwargs) >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi File >>> "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 222, in >>> inner >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi return >>> function(self, context, instance, *args, **kwargs) >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi File >>> "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 250, in >>> _wrapped >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi return >>> fn(self, context, instance, *args, **kwargs) >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi File >>> "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 203, in >>> inner >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi return >>> f(self, context, instance, *args, **kw) >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi File >>> "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2452, in >>> confirm_resize >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi deltas = >>> self._downsize_quota_delta(context, instance) >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi File >>> "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2537, in >>> _downsize_quota_delta >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi return >>> API._resize_quota_delta(context, new_flavor, old_flavor, 1, -1) >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi File >>> "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2504, in >>> _resize_quota_delta >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi if >>> compare * _quota_delta('vcpus') > 0: >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi File >>> "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2501, in >>> _quota_delta >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi return >>> sense * (new_flavor[resource] - old_flavor[resource]) >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi TypeError: >>> 'NoneType' object has no attribute '__getitem__' >>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi >>> >>> >>> >>> I can't reproduce in master so this must have been fixed meanwhile, >>> however I can't identify the patch. >>> >>> Is someone experiencing the same problem? and know the commit that >>> fixed it? >>> >>> thanks, >>> Belmiro >>> >>> >>> _______________________________________________ >>> OpenStack-operators mailing list >>> [email protected] >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators >>> >>> >> I'm not seeing anything very different in that specific utility code path >> in liberty or mitaka, so I'd have to assume there is something going on in >> the online flavor migration when lazy-loading the flavor from the instance >> object such that the old or new flavor is None. >> >> Are you rolling to kilo 2015.1.0 or the latest release 2015.1.2? >> >> Are there any debug log messages before the error/trace where it says >> it's lazy-loading the flavor and doing anything there? If you know the >> instance in question, you could check the instance_system_metadata and >> instance_extra tables to see if the old/new flavor information has been >> migrated successfully. >> >> I don't think you'd have this problem in master because the online flavor >> migration was dropped in liberty (there is a DB migration that stops you >> from continuing if you haven't manually migrated the flavors yet). >> >> -- >> >> Thanks, >> >> Matt Riedemann >> >> >> _______________________________________________ >> OpenStack-operators mailing list >> [email protected] >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators >> > > > _______________________________________________ > OpenStack-operators mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > >
_______________________________________________ OpenStack-operators mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
