Public bug reported:

There are two snapshot test failures in this job run under class
ImagesOneServerTestJSON:

http://logs.openstack.org/47/623247/2/check/nova-cells-v1/18338f0/job-
output.txt.gz

2018-12-06 23:40:45.318619 | primary | {1} 
tempest.api.compute.images.test_images_oneserver.ImagesOneServerTestJSON.test_create_image_specify_multibyte_character_image_name
 [0.238073s] ... FAILED
2018-12-06 23:40:45.318645 | primary |
2018-12-06 23:40:45.318688 | primary | Captured pythonlogging:
2018-12-06 23:40:45.318730 | primary | ~~~~~~~~~~~~~~~~~~~~~~~
2018-12-06 23:40:45.318952 | primary |     2018-12-06 23:40:45,023 5498 INFO    
 [tempest.lib.common.rest_client] Request 
(ImagesOneServerTestJSON:test_create_image_specify_multibyte_character_image_name):
 409 POST 
http://158.69.74.95/compute/v2.1/servers/b76c5c2c-290f-4734-b751-57585b6ee86c/action
 0.122s
2018-12-06 23:40:45.319576 | primary |     2018-12-06 23:40:45,023 5498 DEBUG   
 [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 
'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
2018-12-06 23:40:45.319635 | primary |             Body: {"createImage": 
{"name": "tempest-\u20a1-25178005"}}
2018-12-06 23:40:45.319981 | primary |         Response - Headers: 
{u'x-compute-request-id': 'req-07883b0e-53dd-48c9-ae73-28f232ad2547', 
u'x-openstack-request-id': 'req-07883b0e-53dd-48c9-ae73-28f232ad2547', 
u'openstack-api-version': 'compute 2.1', 'status': '409', 
u'x-openstack-nova-api-version': '2.1', u'vary': 
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'content-location': 
'http://158.69.74.95/compute/v2.1/servers/b76c5c2c-290f-4734-b751-57585b6ee86c/action',
 u'content-length': '151', u'server': 'Apache/2.4.18 (Ubuntu)', u'date': 'Thu, 
06 Dec 2018 23:40:44 GMT', u'content-type': 'application/json; charset=UTF-8', 
u'connection': 'close'}
2018-12-06 23:40:45.320102 | primary |             Body: {"conflictingRequest": 
{"message": "Cannot 'createImage' instance b76c5c2c-290f-4734-b751-57585b6ee86c 
while it is in vm_state building", "code": 409}}
2018-12-06 23:40:45.321130 | primary |
2018-12-06 23:40:45.321154 | primary |
2018-12-06 23:40:45.321174 | primary | Captured traceback:
2018-12-06 23:40:45.321194 | primary | ~~~~~~~~~~~~~~~~~~~
2018-12-06 23:40:45.321223 | primary |     Traceback (most recent call last):
2018-12-06 23:40:45.321302 | primary |       File 
"tempest/api/compute/images/test_images_oneserver.py", line 105, in 
test_create_image_specify_multibyte_character_image_name
2018-12-06 23:40:45.321333 | primary |         self.server_id, name=utf8_name)
2018-12-06 23:40:45.321385 | primary |       File 
"tempest/lib/services/compute/images_client.py", line 43, in create_image
2018-12-06 23:40:45.321404 | primary |         post_body)
2018-12-06 23:40:45.321447 | primary |       File 
"tempest/lib/common/rest_client.py", line 280, in post
2018-12-06 23:40:45.321514 | primary |         return self.request('POST', url, 
extra_headers, headers, body, chunked)
2018-12-06 23:40:45.321662 | primary |       File 
"tempest/lib/services/compute/base_compute_client.py", line 48, in request
2018-12-06 23:40:45.321744 | primary |         method, url, extra_headers, 
headers, body, chunked)
2018-12-06 23:40:45.321818 | primary |       File 
"tempest/lib/common/rest_client.py", line 676, in request
2018-12-06 23:40:45.321876 | primary |         self._error_checker(resp, 
resp_body)
2018-12-06 23:40:45.321931 | primary |       File 
"tempest/lib/common/rest_client.py", line 797, in _error_checker
2018-12-06 23:40:45.321969 | primary |         raise 
exceptions.Conflict(resp_body, resp=resp)
2018-12-06 23:40:45.322016 | primary |     tempest.lib.exceptions.Conflict: 
Conflict with state of target resource
2018-12-06 23:40:45.322099 | primary |     Details: {u'message': u"Cannot 
'createImage' instance b76c5c2c-290f-4734-b751-57585b6ee86c while it is in 
vm_state building", u'code': 409}

and

2018-12-06 23:40:45.006764 | primary | {1}
tempest.api.compute.images.test_images_oneserver.ImagesOneServerTestJSON.test_create_delete_image
[210.094519s] ... FAILED

2018-12-06 23:40:45.315570 | primary | Captured traceback:
2018-12-06 23:40:45.315590 | primary | ~~~~~~~~~~~~~~~~~~~
2018-12-06 23:40:45.316307 | primary |     Traceback (most recent call last):
2018-12-06 23:40:45.316397 | primary |       File 
"tempest/api/compute/images/test_images_oneserver.py", line 70, in 
test_create_delete_image
2018-12-06 23:40:45.316423 | primary |         wait_until='ACTIVE')
2018-12-06 23:40:45.316473 | primary |       File 
"tempest/api/compute/base.py", line 373, in create_image_from_server
2018-12-06 23:40:45.317497 | primary |         server_id, 'ACTIVE')
2018-12-06 23:40:45.317580 | primary |       File "tempest/common/waiters.py", 
line 96, in wait_for_server_status
2018-12-06 23:40:45.317619 | primary |         raise 
lib_exc.TimeoutException(message)
2018-12-06 23:40:45.318138 | primary |     
tempest.lib.exceptions.TimeoutException: Request timed out
2018-12-06 23:40:45.318320 | primary |     Details: 
(ImagesOneServerTestJSON:test_create_delete_image) Server 
b76c5c2c-290f-4734-b751-57585b6ee86c failed to reach ACTIVE status and task 
state "None" within the required time (196 s). Current status: BUILD. Current 
task state: None.

We've seen similar race bugs for the cells v1 job and snapshot tests in
the past, see bug 1620761.

Given the deprecated nature of cells v1 I suggest we simply disable
snapshot testing in the cells v1 job.

** Affects: nova
     Importance: Medium
         Status: Confirmed


** Tags: cells gate-failure snapshot v1

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1807407

Title:
  snapshot related tests intermittently fail in cells v1 job

Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  There are two snapshot test failures in this job run under class
  ImagesOneServerTestJSON:

  http://logs.openstack.org/47/623247/2/check/nova-cells-v1/18338f0/job-
  output.txt.gz

  2018-12-06 23:40:45.318619 | primary | {1} 
tempest.api.compute.images.test_images_oneserver.ImagesOneServerTestJSON.test_create_image_specify_multibyte_character_image_name
 [0.238073s] ... FAILED
  2018-12-06 23:40:45.318645 | primary |
  2018-12-06 23:40:45.318688 | primary | Captured pythonlogging:
  2018-12-06 23:40:45.318730 | primary | ~~~~~~~~~~~~~~~~~~~~~~~
  2018-12-06 23:40:45.318952 | primary |     2018-12-06 23:40:45,023 5498 INFO  
   [tempest.lib.common.rest_client] Request 
(ImagesOneServerTestJSON:test_create_image_specify_multibyte_character_image_name):
 409 POST 
http://158.69.74.95/compute/v2.1/servers/b76c5c2c-290f-4734-b751-57585b6ee86c/action
 0.122s
  2018-12-06 23:40:45.319576 | primary |     2018-12-06 23:40:45,023 5498 DEBUG 
   [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 
'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
  2018-12-06 23:40:45.319635 | primary |             Body: {"createImage": 
{"name": "tempest-\u20a1-25178005"}}
  2018-12-06 23:40:45.319981 | primary |         Response - Headers: 
{u'x-compute-request-id': 'req-07883b0e-53dd-48c9-ae73-28f232ad2547', 
u'x-openstack-request-id': 'req-07883b0e-53dd-48c9-ae73-28f232ad2547', 
u'openstack-api-version': 'compute 2.1', 'status': '409', 
u'x-openstack-nova-api-version': '2.1', u'vary': 
'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'content-location': 
'http://158.69.74.95/compute/v2.1/servers/b76c5c2c-290f-4734-b751-57585b6ee86c/action',
 u'content-length': '151', u'server': 'Apache/2.4.18 (Ubuntu)', u'date': 'Thu, 
06 Dec 2018 23:40:44 GMT', u'content-type': 'application/json; charset=UTF-8', 
u'connection': 'close'}
  2018-12-06 23:40:45.320102 | primary |             Body: 
{"conflictingRequest": {"message": "Cannot 'createImage' instance 
b76c5c2c-290f-4734-b751-57585b6ee86c while it is in vm_state building", "code": 
409}}
  2018-12-06 23:40:45.321130 | primary |
  2018-12-06 23:40:45.321154 | primary |
  2018-12-06 23:40:45.321174 | primary | Captured traceback:
  2018-12-06 23:40:45.321194 | primary | ~~~~~~~~~~~~~~~~~~~
  2018-12-06 23:40:45.321223 | primary |     Traceback (most recent call last):
  2018-12-06 23:40:45.321302 | primary |       File 
"tempest/api/compute/images/test_images_oneserver.py", line 105, in 
test_create_image_specify_multibyte_character_image_name
  2018-12-06 23:40:45.321333 | primary |         self.server_id, name=utf8_name)
  2018-12-06 23:40:45.321385 | primary |       File 
"tempest/lib/services/compute/images_client.py", line 43, in create_image
  2018-12-06 23:40:45.321404 | primary |         post_body)
  2018-12-06 23:40:45.321447 | primary |       File 
"tempest/lib/common/rest_client.py", line 280, in post
  2018-12-06 23:40:45.321514 | primary |         return self.request('POST', 
url, extra_headers, headers, body, chunked)
  2018-12-06 23:40:45.321662 | primary |       File 
"tempest/lib/services/compute/base_compute_client.py", line 48, in request
  2018-12-06 23:40:45.321744 | primary |         method, url, extra_headers, 
headers, body, chunked)
  2018-12-06 23:40:45.321818 | primary |       File 
"tempest/lib/common/rest_client.py", line 676, in request
  2018-12-06 23:40:45.321876 | primary |         self._error_checker(resp, 
resp_body)
  2018-12-06 23:40:45.321931 | primary |       File 
"tempest/lib/common/rest_client.py", line 797, in _error_checker
  2018-12-06 23:40:45.321969 | primary |         raise 
exceptions.Conflict(resp_body, resp=resp)
  2018-12-06 23:40:45.322016 | primary |     tempest.lib.exceptions.Conflict: 
Conflict with state of target resource
  2018-12-06 23:40:45.322099 | primary |     Details: {u'message': u"Cannot 
'createImage' instance b76c5c2c-290f-4734-b751-57585b6ee86c while it is in 
vm_state building", u'code': 409}

  and

  2018-12-06 23:40:45.006764 | primary | {1}
  
tempest.api.compute.images.test_images_oneserver.ImagesOneServerTestJSON.test_create_delete_image
  [210.094519s] ... FAILED

  2018-12-06 23:40:45.315570 | primary | Captured traceback:
  2018-12-06 23:40:45.315590 | primary | ~~~~~~~~~~~~~~~~~~~
  2018-12-06 23:40:45.316307 | primary |     Traceback (most recent call last):
  2018-12-06 23:40:45.316397 | primary |       File 
"tempest/api/compute/images/test_images_oneserver.py", line 70, in 
test_create_delete_image
  2018-12-06 23:40:45.316423 | primary |         wait_until='ACTIVE')
  2018-12-06 23:40:45.316473 | primary |       File 
"tempest/api/compute/base.py", line 373, in create_image_from_server
  2018-12-06 23:40:45.317497 | primary |         server_id, 'ACTIVE')
  2018-12-06 23:40:45.317580 | primary |       File 
"tempest/common/waiters.py", line 96, in wait_for_server_status
  2018-12-06 23:40:45.317619 | primary |         raise 
lib_exc.TimeoutException(message)
  2018-12-06 23:40:45.318138 | primary |     
tempest.lib.exceptions.TimeoutException: Request timed out
  2018-12-06 23:40:45.318320 | primary |     Details: 
(ImagesOneServerTestJSON:test_create_delete_image) Server 
b76c5c2c-290f-4734-b751-57585b6ee86c failed to reach ACTIVE status and task 
state "None" within the required time (196 s). Current status: BUILD. Current 
task state: None.

  We've seen similar race bugs for the cells v1 job and snapshot tests
  in the past, see bug 1620761.

  Given the deprecated nature of cells v1 I suggest we simply disable
  snapshot testing in the cells v1 job.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1807407/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to     : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

Reply via email to