Removing nova from this for tracking purposes. I can open a separate bug for the side effect logging issue in nova.
** No longer affects: nova -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1819910 Title: test_server_connectivity_live_migration intermittently fails with NoValidHost due to DestinationHypervisorTooOld Status in tempest: In Progress Bug description: Seen here: http://logs.openstack.org/94/637594/1/gate/tempest- slow/90def65/controller/logs/screen-n-super- cond.txt.gz#_Mar_13_01_08_16_120854 Mar 13 01:08:16.120854 ubuntu-xenial-inap-mtl01-0003740112 nova-conductor[25115]: WARNING nova.scheduler.utils [None req-a9345308-4e20-4976-ac6d-a1c529d14b16 tempest-TestNetworkAdvancedServerOps-840936673 tempest-TestNetworkAdvancedServerOps-840936673] Failed to compute_task_migrate_server: No valid host was found. There are not enough hosts available. Mar 13 01:08:16.121077 ubuntu-xenial-inap-mtl01-0003740112 nova-conductor[25115]: Traceback (most recent call last): Mar 13 01:08:16.121335 ubuntu-xenial-inap-mtl01-0003740112 nova-conductor[25115]: File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 226, in inner Mar 13 01:08:16.121545 ubuntu-xenial-inap-mtl01-0003740112 nova-conductor[25115]: return func(*args, **kwargs) Mar 13 01:08:16.121819 ubuntu-xenial-inap-mtl01-0003740112 nova-conductor[25115]: File "/opt/stack/nova/nova/scheduler/manager.py", line 154, in select_destinations Mar 13 01:08:16.122027 ubuntu-xenial-inap-mtl01-0003740112 nova-conductor[25115]: allocation_request_version, return_alternates) Mar 13 01:08:16.122232 ubuntu-xenial-inap-mtl01-0003740112 nova-conductor[25115]: File "/opt/stack/nova/nova/scheduler/filter_scheduler.py", line 91, in select_destinations Mar 13 01:08:16.122518 ubuntu-xenial-inap-mtl01-0003740112 nova-conductor[25115]: allocation_request_version, return_alternates) Mar 13 01:08:16.122719 ubuntu-xenial-inap-mtl01-0003740112 nova-conductor[25115]: File "/opt/stack/nova/nova/scheduler/filter_scheduler.py", line 244, in _schedule Mar 13 01:08:16.122924 ubuntu-xenial-inap-mtl01-0003740112 nova-conductor[25115]: claimed_instance_uuids) Mar 13 01:08:16.123128 ubuntu-xenial-inap-mtl01-0003740112 nova-conductor[25115]: File "/opt/stack/nova/nova/scheduler/filter_scheduler.py", line 281, in _ensure_sufficient_hosts Mar 13 01:08:16.123322 ubuntu-xenial-inap-mtl01-0003740112 nova-conductor[25115]: raise exception.NoValidHost(reason=reason) Mar 13 01:08:16.123623 ubuntu-xenial-inap-mtl01-0003740112 nova-conductor[25115]: NoValidHost: No valid host was found. There are not enough hosts available. Mar 13 01:08:16.123829 ubuntu-xenial-inap-mtl01-0003740112 nova-conductor[25115]: : NoValidHost_Remote: No valid host was found. There are not enough hosts available. Mar 13 01:08:16.124035 ubuntu-xenial-inap-mtl01-0003740112 nova-conductor[25115]: WARNING nova.scheduler.utils [None req-a9345308-4e20-4976-ac6d-a1c529d14b16 tempest-TestNetworkAdvancedServerOps-840936673 tempest-TestNetworkAdvancedServerOps-840936673] [instance: c2fd36f9-0b41-439c-9700-17aa0abe13c2] Setting instance to ACTIVE state.: NoValidHost_Remote: No valid host was found. There are not enough hosts available. And it looks like that is actually due to this: http://logs.openstack.org/94/637594/1/gate/tempest- slow/90def65/controller/logs/screen-n-super- cond.txt.gz#_Mar_13_01_08_15_842358 Mar 13 01:08:15.842358 ubuntu-xenial-inap-mtl01-0003740112 nova- conductor[25115]: WARNING nova.scheduler.client.report [None req-a9345308-4e20-4976-ac6d-a1c529d14b16 tempest- TestNetworkAdvancedServerOps-840936673 tempest- TestNetworkAdvancedServerOps-840936673] Failed to save allocation for c2fd36f9-0b41-439c-9700-17aa0abe13c2. Got HTTP 400: {"errors": [{"status": 400, "request_id": "req-2ba69e9f-63ac- 4f71-9dd0-2d5b97fcfbe8", "detail": "The server could not comply with the request since it is either malformed or otherwise incorrect.\n\n JSON does not validate: {} does not have enough properties Failed validating 'minProperties' in schema['properties']['allocations']['items']['properties']['resources']: {'additionalProperties': False, 'minProperties': 1, 'patternProperties': {'^[0-9A-Z_]+$': {'minimum': 1, 'type': 'integer'}}, 'type': 'object'} On instance['allocations'][0]['resources']: {} ", "title": "Bad Request"}]}: DestinationHypervisorTooOld: The instance requires a newer hypervisor version than has been provided. The DestinationHypervisorTooOld error there is misleading, it looks like the real failure was an allocation claim in placement: http://logs.openstack.org/94/637594/1/gate/tempest- slow/90def65/controller/logs/screen-placement- api.txt.gz#_Mar_13_01_08_15_839694 Mar 13 01:08:15.839694 ubuntu-xenial-inap-mtl01-0003740112 devstack@placement-api.service[18498]: DEBUG nova.api.openstack.placement.wsgi_wrapper [req-a9345308-4e20-4976-ac6d-a1c529d14b16 req-2ba69e9f-63ac-4f71-9dd0-2d5b97fcfbe8 service placement] Placement API returning an error response: JSON does not validate: {} does not have enough properties Mar 13 01:08:15.839924 ubuntu-xenial-inap-mtl01-0003740112 devstack@placement-api.service[18498]: Failed validating 'minProperties' in schema['properties']['allocations']['items']['properties']['resources']: Mar 13 01:08:15.840362 ubuntu-xenial-inap-mtl01-0003740112 devstack@placement-api.service[18498]: {'additionalProperties': False, Mar 13 01:08:15.840584 ubuntu-xenial-inap-mtl01-0003740112 devstack@placement-api.service[18498]: 'minProperties': 1, Mar 13 01:08:15.840799 ubuntu-xenial-inap-mtl01-0003740112 devstack@placement-api.service[18498]: 'patternProperties': {'^[0-9A-Z_]+$': {'minimum': 1, Mar 13 01:08:15.841007 ubuntu-xenial-inap-mtl01-0003740112 devstack@placement-api.service[18498]: 'type': 'integer'}}, Mar 13 01:08:15.841218 ubuntu-xenial-inap-mtl01-0003740112 devstack@placement-api.service[18498]: 'type': 'object'} Mar 13 01:08:15.841424 ubuntu-xenial-inap-mtl01-0003740112 devstack@placement-api.service[18498]: On instance['allocations'][0]['resources']: Mar 13 01:08:15.841624 ubuntu-xenial-inap-mtl01-0003740112 devstack@placement-api.service[18498]: {} {{(pid=18500) call_func /opt/stack/nova/nova/api/openstack/placement/wsgi_wrapper.py:31}} To manage notifications about this bug go to: https://bugs.launchpad.net/tempest/+bug/1819910/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
