Public bug reported:
It seems that sometimes on slow nodes, tests which requires advanced image,
like e.g.
neutron_tempest_plugin.scenario.test_multicast.MulticastTestIPv4.test_multicast_between_vms_on_same_network
or
neutron_tempest_plugin.scenario.test_trunk.TrunkTest.test_subport_connectivity
may fail due to global timeout reach.
So we should give some more time for such tests, just to make sure that cleanup
after test can perform fine and test will not fail in that last phase.
Examples of failure:
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:39 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-type': 'application/json',
'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1',
'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version',
'x-openstack-request-id': 'req-f6803ed4-cd44-4aaf-852b-9642166f66fd',
'x-compute-request-id': 'req-f6803ed4-cd44-4aaf-852b-9642166f66fd',
'connection': 'close', 'status': '204', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/6d8abb96-e8de-4833-86f0-2307493e42f9'}
Body: b''
2020-03-24 19:34:41,164 19073 INFO [tempest.lib.common.rest_client] Request
(MulticastTestIPv4:_run_cleanups): 200 GET
https://158.69.70.109/compute/v2.1/servers/6d8abb96-e8de-4833-86f0-2307493e42f9
0.664s
2020-03-24 19:34:41,164 19073 DEBUG [tempest.lib.common.rest_client] Request
- Headers: {'Content-Type': 'application/json', 'Accept': 'application/json',
'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:40 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-length': '1636', 'content-type':
'application/json', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id':
'req-9e05c04f-d73f-410e-a593-9732cd0d368b', 'x-compute-request-id':
'req-9e05c04f-d73f-410e-a593-9732cd0d368b', 'connection': 'close', 'status':
'200', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/6d8abb96-e8de-4833-86f0-2307493e42f9'}
Body: b'{"server": {"id": "6d8abb96-e8de-4833-86f0-2307493e42f9",
"name": "tempest-multicast-server-968193459", "status": "ACTIVE", "tenant_id":
"1247c419c0bd41bb948eacb87e8bf8e3", "user_id":
"3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId":
"9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id":
"3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]},
"flavor": {"id": "d1", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/flavors/d1"}]}, "created":
"2020-03-24T19:24:33Z", "updated": "2020-03-24T19:34:40Z", "addresses":
{"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.12",
"OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:e2:f4:69"},
{"version": 4, "addr": "172.24.5.147", "OS-EXT-IPS:type": "floating",
"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:e2:f4:69"}]}, "accessIPv4": "",
"accessIPv6": "", "links": [{"rel": "self", "href":
"https://158.69.70.109/compute/v2.1/servers/6d8abb96-e8de-4833-86f0-2307493e42f9"},
{"rel": "bookmark", "href":
"https://158.69.70.109/compute/servers/6d8abb96-e8de-4833-86f0-2307493e42f9"}],
"OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone":
"nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661",
"OS-SRV-USG:launched_at": "2020-03-24T19:24:40.000000",
"OS-SRV-USG:terminated_at": null, "security_groups": [{"name":
"secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state":
"active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached":
[]}}'
2020-03-24 19:34:41,762 19073 INFO [tempest.lib.common.rest_client] Request
(MulticastTestIPv4:_run_cleanups): 204 DELETE
https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f
0.322s
2020-03-24 19:34:41,762 19073 DEBUG [tempest.lib.common.rest_client] Request
- Headers: {'Content-Type': 'application/json', 'Accept': 'application/json',
'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:41 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-type': 'application/json',
'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1',
'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version',
'x-openstack-request-id': 'req-06f636f8-ebd2-4974-979b-cf7a81e1f29f',
'x-compute-request-id': 'req-06f636f8-ebd2-4974-979b-cf7a81e1f29f',
'connection': 'close', 'status': '204', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b''
2020-03-24 19:34:42,583 19073 INFO [tempest.lib.common.rest_client] Request
(MulticastTestIPv4:_run_cleanups): 200 GET
https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f
0.820s
2020-03-24 19:34:42,583 19073 DEBUG [tempest.lib.common.rest_client] Request
- Headers: {'Content-Type': 'application/json', 'Accept': 'application/json',
'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:41 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-length': '1636', 'content-type':
'application/json', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id':
'req-6b4b40e9-7181-497d-b8dd-f5e32189b897', 'x-compute-request-id':
'req-6b4b40e9-7181-497d-b8dd-f5e32189b897', 'connection': 'close', 'status':
'200', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b'{"server": {"id": "fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f",
"name": "tempest-multicast-server-1504820408", "status": "ACTIVE", "tenant_id":
"1247c419c0bd41bb948eacb87e8bf8e3", "user_id":
"3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId":
"9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id":
"3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]},
"flavor": {"id": "d1", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/flavors/d1"}]}, "created":
"2020-03-24T19:19:26Z", "updated": "2020-03-24T19:34:42Z", "addresses":
{"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.8",
"OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"},
{"version": 4, "addr": "172.24.5.125", "OS-EXT-IPS:type": "floating",
"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"}]}, "accessIPv4": "",
"accessIPv6": "", "links": [{"rel": "self", "href":
"https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"},
{"rel": "bookmark", "href":
"https://158.69.70.109/compute/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"}],
"OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone":
"nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661",
"OS-SRV-USG:launched_at": "2020-03-24T19:19:36.000000",
"OS-SRV-USG:terminated_at": null, "security_groups": [{"name":
"secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state":
"active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached":
[]}}'
2020-03-24 19:34:44,202 19073 INFO [tempest.lib.common.rest_client] Request
(MulticastTestIPv4:_run_cleanups): 200 GET
https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f
0.612s
2020-03-24 19:34:44,203 19073 DEBUG [tempest.lib.common.rest_client] Request
- Headers: {'Content-Type': 'application/json', 'Accept': 'application/json',
'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:43 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-length': '1636', 'content-type':
'application/json', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id':
'req-54679f2b-ea1a-43c2-920c-757819b2845f', 'x-compute-request-id':
'req-54679f2b-ea1a-43c2-920c-757819b2845f', 'connection': 'close', 'status':
'200', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b'{"server": {"id": "fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f",
"name": "tempest-multicast-server-1504820408", "status": "ACTIVE", "tenant_id":
"1247c419c0bd41bb948eacb87e8bf8e3", "user_id":
"3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId":
"9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id":
"3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]},
"flavor": {"id": "d1", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/flavors/d1"}]}, "created":
"2020-03-24T19:19:26Z", "updated": "2020-03-24T19:34:43Z", "addresses":
{"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.8",
"OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"},
{"version": 4, "addr": "172.24.5.125", "OS-EXT-IPS:type": "floating",
"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"}]}, "accessIPv4": "",
"accessIPv6": "", "links": [{"rel": "self", "href":
"https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"},
{"rel": "bookmark", "href":
"https://158.69.70.109/compute/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"}],
"OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone":
"nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661",
"OS-SRV-USG:launched_at": "2020-03-24T19:19:36.000000",
"OS-SRV-USG:terminated_at": null, "security_groups": [{"name":
"secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state":
"active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached":
[]}}'
2020-03-24 19:34:45,520 19073 INFO [tempest.lib.common.rest_client] Request
(MulticastTestIPv4:_run_cleanups): 200 GET
https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f
0.307s
2020-03-24 19:34:45,521 19073 DEBUG [tempest.lib.common.rest_client] Request
- Headers: {'Content-Type': 'application/json', 'Accept': 'application/json',
'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:45 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-length': '1636', 'content-type':
'application/json', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id':
'req-7affb2b1-108d-4de5-b6f4-a42602c7c6e7', 'x-compute-request-id':
'req-7affb2b1-108d-4de5-b6f4-a42602c7c6e7', 'connection': 'close', 'status':
'200', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b'{"server": {"id": "fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f",
"name": "tempest-multicast-server-1504820408", "status": "ACTIVE", "tenant_id":
"1247c419c0bd41bb948eacb87e8bf8e3", "user_id":
"3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId":
"9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id":
"3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]},
"flavor": {"id": "d1", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/flavors/d1"}]}, "created":
"2020-03-24T19:19:26Z", "updated": "2020-03-24T19:34:43Z", "addresses":
{"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.8",
"OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"},
{"version": 4, "addr": "172.24.5.125", "OS-EXT-IPS:type": "floating",
"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"}]}, "accessIPv4": "",
"accessIPv6": "", "links": [{"rel": "self", "href":
"https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"},
{"rel": "bookmark", "href":
"https://158.69.70.109/compute/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"}],
"OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone":
"nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661",
"OS-SRV-USG:launched_at": "2020-03-24T19:19:36.000000",
"OS-SRV-USG:terminated_at": null, "security_groups": [{"name":
"secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state":
"active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached":
[]}}'
2020-03-24 19:34:46,735 19073 INFO [tempest.lib.common.rest_client] Request
(MulticastTestIPv4:_run_cleanups): 200 GET
https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f
0.206s
2020-03-24 19:34:46,735 19073 DEBUG [tempest.lib.common.rest_client] Request
- Headers: {'Content-Type': 'application/json', 'Accept': 'application/json',
'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:46 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-length': '1322', 'content-type':
'application/json', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id':
'req-d0381bc4-6667-48c3-8c04-0faa3bb95f00', 'x-compute-request-id':
'req-d0381bc4-6667-48c3-8c04-0faa3bb95f00', 'connection': 'close', 'status':
'200', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b'{"server": {"id": "fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f",
"name": "tempest-multicast-server-1504820408", "status": "ACTIVE", "tenant_id":
"1247c419c0bd41bb948eacb87e8bf8e3", "user_id":
"3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId":
"9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id":
"3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]},
"flavor": {"id": "d1", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/flavors/d1"}]}, "created":
"2020-03-24T19:19:26Z", "updated": "2020-03-24T19:34:43Z", "addresses": {},
"accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href":
"https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"},
{"rel": "bookmark", "href":
"https://158.69.70.109/compute/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"}],
"OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone":
"nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661",
"OS-SRV-USG:launched_at": "2020-03-24T19:19:36.000000",
"OS-SRV-USG:terminated_at": null, "OS-EXT-STS:task_state": "deleting",
"OS-EXT-STS:vm_state": "active", "OS-EXT-STS:power_state": 1,
"os-extended-volumes:volumes_attached": []}}'
2020-03-24 19:34:47,847 19073 INFO [tempest.lib.common.rest_client] Request
(MulticastTestIPv4:_run_cleanups): 404 GET
https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f
0.100s
2020-03-24 19:34:47,848 19073 DEBUG [tempest.lib.common.rest_client] Request
- Headers: {'Content-Type': 'application/json', 'Accept': 'application/json',
'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:47 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'content-type':
'application/json; charset=UTF-8', 'content-length': '111',
'x-openstack-request-id': 'req-3af792f4-16b1-4320-8e66-d6a192ad7a5f',
'x-compute-request-id': 'req-3af792f4-16b1-4320-8e66-d6a192ad7a5f',
'connection': 'close', 'status': '404', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b'{"itemNotFound": {"code": 404, "message": "Instance
fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f could not be found."}}'
2020-03-24 19:34:48,060 19073 INFO [tempest.lib.common.rest_client] Request
(MulticastTestIPv4:_run_cleanups): 204 DELETE
https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6
0.211s
2020-03-24 19:34:48,060 19073 DEBUG [tempest.lib.common.rest_client] Request
- Headers: {'Content-Type': 'application/json', 'Accept': 'application/json',
'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:47 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-type': 'application/json',
'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1',
'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version',
'x-openstack-request-id': 'req-60a7c83f-4b76-4c94-9cf6-e57fbed2b982',
'x-compute-request-id': 'req-60a7c83f-4b76-4c94-9cf6-e57fbed2b982',
'connection': 'close', 'status': '204', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6'}
Body: b''
2020-03-24 19:34:48,438 19073 INFO [tempest.lib.common.rest_client] Request
(MulticastTestIPv4:_run_cleanups): 200 GET
https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6
0.377s
2020-03-24 19:34:48,438 19073 DEBUG [tempest.lib.common.rest_client] Request
- Headers: {'Content-Type': 'application/json', 'Accept': 'application/json',
'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:48 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-length': '1637', 'content-type':
'application/json', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id':
'req-980b088a-799e-432c-ad02-bf6204a115f6', 'x-compute-request-id':
'req-980b088a-799e-432c-ad02-bf6204a115f6', 'connection': 'close', 'status':
'200', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6'}
Body: b'{"server": {"id": "dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6",
"name": "tempest-multicast-server-1305662029", "status": "ACTIVE", "tenant_id":
"1247c419c0bd41bb948eacb87e8bf8e3", "user_id":
"3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId":
"9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id":
"3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]},
"flavor": {"id": "d1", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/flavors/d1"}]}, "created":
"2020-03-24T19:14:43Z", "updated": "2020-03-24T19:34:48Z", "addresses":
{"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.11",
"OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:aa:46:f7"},
{"version": 4, "addr": "172.24.5.249", "OS-EXT-IPS:type": "floating",
"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:aa:46:f7"}]}, "accessIPv4": "",
"accessIPv6": "", "links": [{"rel": "self", "href":
"https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6"},
{"rel": "bookmark", "href":
"https://158.69.70.109/compute/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6"}],
"OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone":
"nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661",
"OS-SRV-USG:launched_at": "2020-03-24T19:15:13.000000",
"OS-SRV-USG:terminated_at": null, "security_groups": [{"name":
"secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state":
"active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached":
[]}}'
2020-03-24 19:34:50,655 19073 INFO [tempest.lib.common.rest_client] Request
(MulticastTestIPv4:_run_cleanups): 200 GET
https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6
1.210s
2020-03-24 19:34:50,656 19073 DEBUG [tempest.lib.common.rest_client] Request
- Headers: {'Content-Type': 'application/json', 'Accept': 'application/json',
'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:49 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-length': '1637', 'content-type':
'application/json', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id':
'req-846415cc-4906-4239-b483-2725e4cb4800', 'x-compute-request-id':
'req-846415cc-4906-4239-b483-2725e4cb4800', 'connection': 'close', 'status':
'200', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6'}
Body: b'{"server": {"id": "dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6",
"name": "tempest-multicast-server-1305662029", "status": "ACTIVE", "tenant_id":
"1247c419c0bd41bb948eacb87e8bf8e3", "user_id":
"3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId":
"9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id":
"3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]},
"flavor": {"id": "d1", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/flavors/d1"}]}, "created":
"2020-03-24T19:14:43Z", "updated": "2020-03-24T19:34:49Z", "addresses":
{"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.11",
"OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:aa:46:f7"},
{"version": 4, "addr": "172.24.5.249", "OS-EXT-IPS:type": "floating",
"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:aa:46:f7"}]}, "accessIPv4": "",
"accessIPv6": "", "links": [{"rel": "self", "href":
"https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6"},
{"rel": "bookmark", "href":
"https://158.69.70.109/compute/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6"}],
"OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone":
"nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661",
"OS-SRV-USG:launched_at": "2020-03-24T19:15:13.000000",
"OS-SRV-USG:terminated_at": null, "security_groups": [{"name":
"secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state":
"active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached":
[]}}'
2020-03-24 19:34:51,764 19073 INFO [tempest.lib.common.rest_client] Request
(MulticastTestIPv4:_run_cleanups): 404 GET
https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6
0.098s
2020-03-24 19:34:51,765 19073 DEBUG [tempest.lib.common.rest_client] Request
- Headers: {'Content-Type': 'application/json', 'Accept': 'application/json',
'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:51 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'content-type':
'application/json; charset=UTF-8', 'content-length': '111',
'x-openstack-request-id': 'req-12f4f0a5-7fbc-4f3c-8000-1d3e5c8e3fa1',
'x-compute-request-id': 'req-12f4f0a5-7fbc-4f3c-8000-1d3e5c8e3fa1',
'connection': 'close', 'status': '404', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6'}
Body: b'{"itemNotFound": {"code": 404, "message": "Instance
dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6 could not be found."}}'
}}}
Traceback (most recent call last):
File "/opt/stack/tempest/tempest/lib/common/utils/test_utils.py", line 84, in
call_and_ignore_notfound_exc
return func(*args, **kwargs)
File "/opt/stack/tempest/tempest/common/waiters.py", line 111, in
wait_for_server_termination
time.sleep(client.build_interval)
File "/usr/local/lib/python3.6/dist-packages/fixtures/_fixtures/timeout.py",
line 52, in signal_handler
raise TimeoutException()
fixtures._fixtures.timeout.TimeoutException
links to examples of failures:
https://103900b4a03cdd60217d-625a0eb0440aa527fbdb216e8991f5a6.ssl.cf5.rackcdn.com/714726/1/check/neutron-ovn-tempest-ovs-release/2b52c20/testr_results.html
https://5481f30c7f7864f4d07b-85bb929ac58f1a419407783b8f581cf5.ssl.cf5.rackcdn.com/710755/11/check/neutron-tempest-plugin-scenario-openvswitch/2d4ae79/testr_results.html
logstash query:
http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22line%20111%2C%20in%20wait_for_server_termination%5C%22
** Affects: neutron
Importance: Critical
Assignee: Slawek Kaplonski (slaweq)
Status: Confirmed
** Tags: gate-failure tempest
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1872693
Title:
Timeout exception in scenario tests with advanced image
Status in neutron:
Confirmed
Bug description:
It seems that sometimes on slow nodes, tests which requires advanced image,
like e.g.
neutron_tempest_plugin.scenario.test_multicast.MulticastTestIPv4.test_multicast_between_vms_on_same_network
or
neutron_tempest_plugin.scenario.test_trunk.TrunkTest.test_subport_connectivity
may fail due to global timeout reach.
So we should give some more time for such tests, just to make sure that
cleanup after test can perform fine and test will not fail in that last phase.
Examples of failure:
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:39 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-type': 'application/json',
'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1',
'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version',
'x-openstack-request-id': 'req-f6803ed4-cd44-4aaf-852b-9642166f66fd',
'x-compute-request-id': 'req-f6803ed4-cd44-4aaf-852b-9642166f66fd',
'connection': 'close', 'status': '204', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/6d8abb96-e8de-4833-86f0-2307493e42f9'}
Body: b''
2020-03-24 19:34:41,164 19073 INFO [tempest.lib.common.rest_client]
Request (MulticastTestIPv4:_run_cleanups): 200 GET
https://158.69.70.109/compute/v2.1/servers/6d8abb96-e8de-4833-86f0-2307493e42f9
0.664s
2020-03-24 19:34:41,164 19073 DEBUG [tempest.lib.common.rest_client]
Request - Headers: {'Content-Type': 'application/json', 'Accept':
'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:40 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-length': '1636', 'content-type':
'application/json', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id':
'req-9e05c04f-d73f-410e-a593-9732cd0d368b', 'x-compute-request-id':
'req-9e05c04f-d73f-410e-a593-9732cd0d368b', 'connection': 'close', 'status':
'200', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/6d8abb96-e8de-4833-86f0-2307493e42f9'}
Body: b'{"server": {"id": "6d8abb96-e8de-4833-86f0-2307493e42f9",
"name": "tempest-multicast-server-968193459", "status": "ACTIVE", "tenant_id":
"1247c419c0bd41bb948eacb87e8bf8e3", "user_id":
"3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId":
"9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id":
"3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]},
"flavor": {"id": "d1", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/flavors/d1"}]}, "created":
"2020-03-24T19:24:33Z", "updated": "2020-03-24T19:34:40Z", "addresses":
{"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.12",
"OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:e2:f4:69"},
{"version": 4, "addr": "172.24.5.147", "OS-EXT-IPS:type": "floating",
"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:e2:f4:69"}]}, "accessIPv4": "",
"accessIPv6": "", "links": [{"rel": "self", "href":
"https://158.69.70.109/compute/v2.1/servers/6d8abb96-e8de-4833-86f0-2307493e42f9"},
{"rel": "bookmark", "href":
"https://158.69.70.109/compute/servers/6d8abb96-e8de-4833-86f0-2307493e42f9"}],
"OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone":
"nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661",
"OS-SRV-USG:launched_at": "2020-03-24T19:24:40.000000",
"OS-SRV-USG:terminated_at": null, "security_groups": [{"name":
"secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state":
"active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached":
[]}}'
2020-03-24 19:34:41,762 19073 INFO [tempest.lib.common.rest_client]
Request (MulticastTestIPv4:_run_cleanups): 204 DELETE
https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f
0.322s
2020-03-24 19:34:41,762 19073 DEBUG [tempest.lib.common.rest_client]
Request - Headers: {'Content-Type': 'application/json', 'Accept':
'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:41 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-type': 'application/json',
'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1',
'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version',
'x-openstack-request-id': 'req-06f636f8-ebd2-4974-979b-cf7a81e1f29f',
'x-compute-request-id': 'req-06f636f8-ebd2-4974-979b-cf7a81e1f29f',
'connection': 'close', 'status': '204', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b''
2020-03-24 19:34:42,583 19073 INFO [tempest.lib.common.rest_client]
Request (MulticastTestIPv4:_run_cleanups): 200 GET
https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f
0.820s
2020-03-24 19:34:42,583 19073 DEBUG [tempest.lib.common.rest_client]
Request - Headers: {'Content-Type': 'application/json', 'Accept':
'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:41 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-length': '1636', 'content-type':
'application/json', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id':
'req-6b4b40e9-7181-497d-b8dd-f5e32189b897', 'x-compute-request-id':
'req-6b4b40e9-7181-497d-b8dd-f5e32189b897', 'connection': 'close', 'status':
'200', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b'{"server": {"id": "fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f",
"name": "tempest-multicast-server-1504820408", "status": "ACTIVE", "tenant_id":
"1247c419c0bd41bb948eacb87e8bf8e3", "user_id":
"3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId":
"9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id":
"3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]},
"flavor": {"id": "d1", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/flavors/d1"}]}, "created":
"2020-03-24T19:19:26Z", "updated": "2020-03-24T19:34:42Z", "addresses":
{"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.8",
"OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"},
{"version": 4, "addr": "172.24.5.125", "OS-EXT-IPS:type": "floating",
"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"}]}, "accessIPv4": "",
"accessIPv6": "", "links": [{"rel": "self", "href":
"https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"},
{"rel": "bookmark", "href":
"https://158.69.70.109/compute/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"}],
"OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone":
"nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661",
"OS-SRV-USG:launched_at": "2020-03-24T19:19:36.000000",
"OS-SRV-USG:terminated_at": null, "security_groups": [{"name":
"secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state":
"active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached":
[]}}'
2020-03-24 19:34:44,202 19073 INFO [tempest.lib.common.rest_client]
Request (MulticastTestIPv4:_run_cleanups): 200 GET
https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f
0.612s
2020-03-24 19:34:44,203 19073 DEBUG [tempest.lib.common.rest_client]
Request - Headers: {'Content-Type': 'application/json', 'Accept':
'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:43 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-length': '1636', 'content-type':
'application/json', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id':
'req-54679f2b-ea1a-43c2-920c-757819b2845f', 'x-compute-request-id':
'req-54679f2b-ea1a-43c2-920c-757819b2845f', 'connection': 'close', 'status':
'200', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b'{"server": {"id": "fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f",
"name": "tempest-multicast-server-1504820408", "status": "ACTIVE", "tenant_id":
"1247c419c0bd41bb948eacb87e8bf8e3", "user_id":
"3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId":
"9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id":
"3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]},
"flavor": {"id": "d1", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/flavors/d1"}]}, "created":
"2020-03-24T19:19:26Z", "updated": "2020-03-24T19:34:43Z", "addresses":
{"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.8",
"OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"},
{"version": 4, "addr": "172.24.5.125", "OS-EXT-IPS:type": "floating",
"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"}]}, "accessIPv4": "",
"accessIPv6": "", "links": [{"rel": "self", "href":
"https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"},
{"rel": "bookmark", "href":
"https://158.69.70.109/compute/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"}],
"OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone":
"nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661",
"OS-SRV-USG:launched_at": "2020-03-24T19:19:36.000000",
"OS-SRV-USG:terminated_at": null, "security_groups": [{"name":
"secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state":
"active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached":
[]}}'
2020-03-24 19:34:45,520 19073 INFO [tempest.lib.common.rest_client]
Request (MulticastTestIPv4:_run_cleanups): 200 GET
https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f
0.307s
2020-03-24 19:34:45,521 19073 DEBUG [tempest.lib.common.rest_client]
Request - Headers: {'Content-Type': 'application/json', 'Accept':
'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:45 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-length': '1636', 'content-type':
'application/json', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id':
'req-7affb2b1-108d-4de5-b6f4-a42602c7c6e7', 'x-compute-request-id':
'req-7affb2b1-108d-4de5-b6f4-a42602c7c6e7', 'connection': 'close', 'status':
'200', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b'{"server": {"id": "fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f",
"name": "tempest-multicast-server-1504820408", "status": "ACTIVE", "tenant_id":
"1247c419c0bd41bb948eacb87e8bf8e3", "user_id":
"3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId":
"9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id":
"3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]},
"flavor": {"id": "d1", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/flavors/d1"}]}, "created":
"2020-03-24T19:19:26Z", "updated": "2020-03-24T19:34:43Z", "addresses":
{"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.8",
"OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"},
{"version": 4, "addr": "172.24.5.125", "OS-EXT-IPS:type": "floating",
"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"}]}, "accessIPv4": "",
"accessIPv6": "", "links": [{"rel": "self", "href":
"https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"},
{"rel": "bookmark", "href":
"https://158.69.70.109/compute/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"}],
"OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone":
"nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661",
"OS-SRV-USG:launched_at": "2020-03-24T19:19:36.000000",
"OS-SRV-USG:terminated_at": null, "security_groups": [{"name":
"secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state":
"active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached":
[]}}'
2020-03-24 19:34:46,735 19073 INFO [tempest.lib.common.rest_client]
Request (MulticastTestIPv4:_run_cleanups): 200 GET
https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f
0.206s
2020-03-24 19:34:46,735 19073 DEBUG [tempest.lib.common.rest_client]
Request - Headers: {'Content-Type': 'application/json', 'Accept':
'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:46 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-length': '1322', 'content-type':
'application/json', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id':
'req-d0381bc4-6667-48c3-8c04-0faa3bb95f00', 'x-compute-request-id':
'req-d0381bc4-6667-48c3-8c04-0faa3bb95f00', 'connection': 'close', 'status':
'200', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b'{"server": {"id": "fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f",
"name": "tempest-multicast-server-1504820408", "status": "ACTIVE", "tenant_id":
"1247c419c0bd41bb948eacb87e8bf8e3", "user_id":
"3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId":
"9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id":
"3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]},
"flavor": {"id": "d1", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/flavors/d1"}]}, "created":
"2020-03-24T19:19:26Z", "updated": "2020-03-24T19:34:43Z", "addresses": {},
"accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href":
"https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"},
{"rel": "bookmark", "href":
"https://158.69.70.109/compute/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"}],
"OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone":
"nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661",
"OS-SRV-USG:launched_at": "2020-03-24T19:19:36.000000",
"OS-SRV-USG:terminated_at": null, "OS-EXT-STS:task_state": "deleting",
"OS-EXT-STS:vm_state": "active", "OS-EXT-STS:power_state": 1,
"os-extended-volumes:volumes_attached": []}}'
2020-03-24 19:34:47,847 19073 INFO [tempest.lib.common.rest_client]
Request (MulticastTestIPv4:_run_cleanups): 404 GET
https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f
0.100s
2020-03-24 19:34:47,848 19073 DEBUG [tempest.lib.common.rest_client]
Request - Headers: {'Content-Type': 'application/json', 'Accept':
'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:47 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'content-type':
'application/json; charset=UTF-8', 'content-length': '111',
'x-openstack-request-id': 'req-3af792f4-16b1-4320-8e66-d6a192ad7a5f',
'x-compute-request-id': 'req-3af792f4-16b1-4320-8e66-d6a192ad7a5f',
'connection': 'close', 'status': '404', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b'{"itemNotFound": {"code": 404, "message": "Instance
fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f could not be found."}}'
2020-03-24 19:34:48,060 19073 INFO [tempest.lib.common.rest_client]
Request (MulticastTestIPv4:_run_cleanups): 204 DELETE
https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6
0.211s
2020-03-24 19:34:48,060 19073 DEBUG [tempest.lib.common.rest_client]
Request - Headers: {'Content-Type': 'application/json', 'Accept':
'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:47 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-type': 'application/json',
'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1',
'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version',
'x-openstack-request-id': 'req-60a7c83f-4b76-4c94-9cf6-e57fbed2b982',
'x-compute-request-id': 'req-60a7c83f-4b76-4c94-9cf6-e57fbed2b982',
'connection': 'close', 'status': '204', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6'}
Body: b''
2020-03-24 19:34:48,438 19073 INFO [tempest.lib.common.rest_client]
Request (MulticastTestIPv4:_run_cleanups): 200 GET
https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6
0.377s
2020-03-24 19:34:48,438 19073 DEBUG [tempest.lib.common.rest_client]
Request - Headers: {'Content-Type': 'application/json', 'Accept':
'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:48 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-length': '1637', 'content-type':
'application/json', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id':
'req-980b088a-799e-432c-ad02-bf6204a115f6', 'x-compute-request-id':
'req-980b088a-799e-432c-ad02-bf6204a115f6', 'connection': 'close', 'status':
'200', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6'}
Body: b'{"server": {"id": "dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6",
"name": "tempest-multicast-server-1305662029", "status": "ACTIVE", "tenant_id":
"1247c419c0bd41bb948eacb87e8bf8e3", "user_id":
"3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId":
"9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id":
"3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]},
"flavor": {"id": "d1", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/flavors/d1"}]}, "created":
"2020-03-24T19:14:43Z", "updated": "2020-03-24T19:34:48Z", "addresses":
{"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.11",
"OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:aa:46:f7"},
{"version": 4, "addr": "172.24.5.249", "OS-EXT-IPS:type": "floating",
"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:aa:46:f7"}]}, "accessIPv4": "",
"accessIPv6": "", "links": [{"rel": "self", "href":
"https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6"},
{"rel": "bookmark", "href":
"https://158.69.70.109/compute/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6"}],
"OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone":
"nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661",
"OS-SRV-USG:launched_at": "2020-03-24T19:15:13.000000",
"OS-SRV-USG:terminated_at": null, "security_groups": [{"name":
"secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state":
"active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached":
[]}}'
2020-03-24 19:34:50,655 19073 INFO [tempest.lib.common.rest_client]
Request (MulticastTestIPv4:_run_cleanups): 200 GET
https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6
1.210s
2020-03-24 19:34:50,656 19073 DEBUG [tempest.lib.common.rest_client]
Request - Headers: {'Content-Type': 'application/json', 'Accept':
'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:49 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'content-length': '1637', 'content-type':
'application/json', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id':
'req-846415cc-4906-4239-b483-2725e4cb4800', 'x-compute-request-id':
'req-846415cc-4906-4239-b483-2725e4cb4800', 'connection': 'close', 'status':
'200', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6'}
Body: b'{"server": {"id": "dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6",
"name": "tempest-multicast-server-1305662029", "status": "ACTIVE", "tenant_id":
"1247c419c0bd41bb948eacb87e8bf8e3", "user_id":
"3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId":
"9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id":
"3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]},
"flavor": {"id": "d1", "links": [{"rel": "bookmark", "href":
"https://158.69.70.109/compute/flavors/d1"}]}, "created":
"2020-03-24T19:14:43Z", "updated": "2020-03-24T19:34:49Z", "addresses":
{"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.11",
"OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:aa:46:f7"},
{"version": 4, "addr": "172.24.5.249", "OS-EXT-IPS:type": "floating",
"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:aa:46:f7"}]}, "accessIPv4": "",
"accessIPv6": "", "links": [{"rel": "self", "href":
"https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6"},
{"rel": "bookmark", "href":
"https://158.69.70.109/compute/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6"}],
"OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone":
"nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661",
"OS-SRV-USG:launched_at": "2020-03-24T19:15:13.000000",
"OS-SRV-USG:terminated_at": null, "security_groups": [{"name":
"secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state":
"active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached":
[]}}'
2020-03-24 19:34:51,764 19073 INFO [tempest.lib.common.rest_client]
Request (MulticastTestIPv4:_run_cleanups): 404 GET
https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6
0.098s
2020-03-24 19:34:51,765 19073 DEBUG [tempest.lib.common.rest_client]
Request - Headers: {'Content-Type': 'application/json', 'Accept':
'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:51 GMT', 'server':
'Apache/2.4.29 (Ubuntu)', 'openstack-api-version': 'compute 2.1',
'x-openstack-nova-api-version': '2.1', 'vary':
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'content-type':
'application/json; charset=UTF-8', 'content-length': '111',
'x-openstack-request-id': 'req-12f4f0a5-7fbc-4f3c-8000-1d3e5c8e3fa1',
'x-compute-request-id': 'req-12f4f0a5-7fbc-4f3c-8000-1d3e5c8e3fa1',
'connection': 'close', 'status': '404', 'content-location':
'https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6'}
Body: b'{"itemNotFound": {"code": 404, "message": "Instance
dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6 could not be found."}}'
}}}
Traceback (most recent call last):
File "/opt/stack/tempest/tempest/lib/common/utils/test_utils.py", line 84,
in call_and_ignore_notfound_exc
return func(*args, **kwargs)
File "/opt/stack/tempest/tempest/common/waiters.py", line 111, in
wait_for_server_termination
time.sleep(client.build_interval)
File
"/usr/local/lib/python3.6/dist-packages/fixtures/_fixtures/timeout.py", line
52, in signal_handler
raise TimeoutException()
fixtures._fixtures.timeout.TimeoutException
links to examples of failures:
https://103900b4a03cdd60217d-625a0eb0440aa527fbdb216e8991f5a6.ssl.cf5.rackcdn.com/714726/1/check/neutron-ovn-tempest-ovs-release/2b52c20/testr_results.html
https://5481f30c7f7864f4d07b-85bb929ac58f1a419407783b8f581cf5.ssl.cf5.rackcdn.com/710755/11/check/neutron-tempest-plugin-scenario-openvswitch/2d4ae79/testr_results.html
logstash query:
http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22line%20111%2C%20in%20wait_for_server_termination%5C%22
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1872693/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
