[Yahoo-eng-team] [Bug 1700583] Re: No volume Block Device Mapping in assisted snapshot with Quobyte

2018-01-09 Thread Silvan Kaiser
Whatever the exact reason was it no longer occurs. I'm currently back to
run our CIs with the current DevStack master and no longer hit this,
therefore marking as invalid.

** Changed in: devstack
   Status: New => Invalid

** Changed in: nova
   Status: Opinion => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1700583

Title:
  No volume Block Device Mapping in assisted snapshot with Quobyte

Status in devstack:
  Invalid
Status in OpenStack Compute (nova):
  Invalid

Bug description:
  *Update*: Please see comment #5 for details on how DevStack is involved in 
this issue.
  Also more current CI runs with this issue can be found at:
  - http://78.46.57.153:8081/refs-changes-95-490195-1/
  - http://78.46.57.153:8081/refs-changes-21-490021-3/
  */Update*

  
  Since roughly last friday Quobyte CIs with a Cinder / Nova setup fail with an 
error during assisted snapshots in four tests:
  
test_create_ebs_image_and_check_boot|test_snapshot_create_delete_with_volume_in_use|test_snapshot_create_offline_delete_online|test_volume_boot_pattern

  The error declares that the volume to be snapshotted has no block
  device mapping.

  I currently cannot identify if this is a Nova or Cinder issue, so
  please keep an open mind when looking into this.

  Complete example CI test runs with all logs can be found at e.g.:

  http://78.46.57.153:8081/refs-changes-26-475226-3/
  http://78.46.57.153:8081/refs-changes-02-476402-2/

  n-api log excerpt:
  [...]
  2017-06-23 14:24:18.137 10730 DEBUG nova.api.openstack.wsgi 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Action: 'create', 
calling method: >, body: {"snapshot": {"create_info": {"snapshot_id": 
"30bd2ad6-37a0-4027-a24e-00657adb0cca", "type": "qcow2", "new_file": 
"volume-43245219-bd54-4b03-a371-cca5836e58e5.30bd2ad6-37a0-4027-a24e-00657adb0cca"},
 "volume_id": "43245219-bd54-4b03-a371-cca5836e58e5"}} _process_stack 
/opt/stack/nova/nova/api/openstack/wsgi.py:609
  2017-06-23 14:24:18.438 10730 INFO nova.api.openstack.wsgi 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] HTTP exception thrown: 
No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5.: 
HTTPBadRequest: No volume Block Device Mapping with id 
43245219-bd54-4b03-a371-cca5836e58e5.
  2017-06-23 14:24:18.440 10730 DEBUG nova.api.openstack.wsgi 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Returning 400 to user: 
No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5. 
__call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1029
  2017-06-23 14:24:18.441 10730 INFO nova.osapi_compute.wsgi.server 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] 127.0.0.1 "POST 
/v2.1/os-assisted-volume-snapshots HTTP/1.1" status: 400 len: 543 time: 
0.6530399
  2017-06-23 14:24:18.770 10730 INFO nova.osapi_compute.wsgi.server 
[req-a0c161b6-12ba-4dbf-aa10-5fa34565441c 
tempest-TestVolumeBootPattern-1485741991 
tempest-TestVolumeBootPattern-1485741991] 127.0.0.1 "GET 
/v2.1/servers/0fcae274-2d09-45e7-ba84-df2b53526895 HTTP/1.1" status: 200 len: 
1762 time: 0.8687029
  [...]

  Reading the test logs I see for instance that this error is thrown right 
after the volume attachment
  to the VM is successful. I'd expect the snapshot API call to be able to find 
the mapping in that case. Example snippet:

   [... attaching volume to VM]

  2017-06-26 13:13:43,182 28633 INFO [tempest.lib.common.rest_client] 
Request (VolumesSnapshotTestJSON:test_snapshot_create_offline_delete_online): 
200 GET 
http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097
 0.223s
  2017-06-26 13:13:43,182 28633 DEBUG[tempest.lib.common.rest_client] 
Request - Headers: {'Content-Type': 'application/json', 'Accept': 
'application/json', 'X-Auth-Token': ''}
  Body: None
  Response - Headers: {u'content-type': 'application/json', 
'content-location': 
'http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097',
 u'x-compute-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', 
u'connection': 'close', 'status': '200', u'content-length': '904', 
u'x-openstack-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', u'date': 
'Mon, 26 Jun 2017
   13:13:43 GMT'}
  Body: {"volume": {"status": "attaching", "user_id": 
"8fa70618b61e4290908a964b6c8d11d9", "attachments": [], "links": [{"href": 
"http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";,
 "rel": "self"}, {"href": 
"http://127.0.0.1:8776/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";,
 "rel": "bookmark"}], "availability_zone": "nova", "bootable": "false", 
"encrypted": false, "created_at": "2017-06-26T13:13:29.00", "description": 
null, "os-vol-tenant-attr:tenant_id": "1

[Yahoo-eng-team] [Bug 1700583] Re: No volume Block Device Mapping in assisted snapshot with Quobyte

2017-08-03 Thread Sylvain Bauza
Putting in Opinion as I'm not sure it's a Nova problem.
Silvan, if you need some help for debugging, just ask in our IRC channel if you 
want.


** Changed in: nova
   Status: New => Opinion

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1700583

Title:
  No volume Block Device Mapping in assisted snapshot with Quobyte

Status in devstack:
  New
Status in OpenStack Compute (nova):
  Opinion

Bug description:
  *Update*: Please see comment #5 for details on how DevStack is involved in 
this issue.
  Also more current CI runs with this issue can be found at:
  - http://78.46.57.153:8081/refs-changes-95-490195-1/
  - http://78.46.57.153:8081/refs-changes-21-490021-3/
  */Update*

  
  Since roughly last friday Quobyte CIs with a Cinder / Nova setup fail with an 
error during assisted snapshots in four tests:
  
test_create_ebs_image_and_check_boot|test_snapshot_create_delete_with_volume_in_use|test_snapshot_create_offline_delete_online|test_volume_boot_pattern

  The error declares that the volume to be snapshotted has no block
  device mapping.

  I currently cannot identify if this is a Nova or Cinder issue, so
  please keep an open mind when looking into this.

  Complete example CI test runs with all logs can be found at e.g.:

  http://78.46.57.153:8081/refs-changes-26-475226-3/
  http://78.46.57.153:8081/refs-changes-02-476402-2/

  n-api log excerpt:
  [...]
  2017-06-23 14:24:18.137 10730 DEBUG nova.api.openstack.wsgi 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Action: 'create', 
calling method: >, body: {"snapshot": {"create_info": {"snapshot_id": 
"30bd2ad6-37a0-4027-a24e-00657adb0cca", "type": "qcow2", "new_file": 
"volume-43245219-bd54-4b03-a371-cca5836e58e5.30bd2ad6-37a0-4027-a24e-00657adb0cca"},
 "volume_id": "43245219-bd54-4b03-a371-cca5836e58e5"}} _process_stack 
/opt/stack/nova/nova/api/openstack/wsgi.py:609
  2017-06-23 14:24:18.438 10730 INFO nova.api.openstack.wsgi 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] HTTP exception thrown: 
No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5.: 
HTTPBadRequest: No volume Block Device Mapping with id 
43245219-bd54-4b03-a371-cca5836e58e5.
  2017-06-23 14:24:18.440 10730 DEBUG nova.api.openstack.wsgi 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Returning 400 to user: 
No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5. 
__call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1029
  2017-06-23 14:24:18.441 10730 INFO nova.osapi_compute.wsgi.server 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] 127.0.0.1 "POST 
/v2.1/os-assisted-volume-snapshots HTTP/1.1" status: 400 len: 543 time: 
0.6530399
  2017-06-23 14:24:18.770 10730 INFO nova.osapi_compute.wsgi.server 
[req-a0c161b6-12ba-4dbf-aa10-5fa34565441c 
tempest-TestVolumeBootPattern-1485741991 
tempest-TestVolumeBootPattern-1485741991] 127.0.0.1 "GET 
/v2.1/servers/0fcae274-2d09-45e7-ba84-df2b53526895 HTTP/1.1" status: 200 len: 
1762 time: 0.8687029
  [...]

  Reading the test logs I see for instance that this error is thrown right 
after the volume attachment
  to the VM is successful. I'd expect the snapshot API call to be able to find 
the mapping in that case. Example snippet:

   [... attaching volume to VM]

  2017-06-26 13:13:43,182 28633 INFO [tempest.lib.common.rest_client] 
Request (VolumesSnapshotTestJSON:test_snapshot_create_offline_delete_online): 
200 GET 
http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097
 0.223s
  2017-06-26 13:13:43,182 28633 DEBUG[tempest.lib.common.rest_client] 
Request - Headers: {'Content-Type': 'application/json', 'Accept': 
'application/json', 'X-Auth-Token': ''}
  Body: None
  Response - Headers: {u'content-type': 'application/json', 
'content-location': 
'http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097',
 u'x-compute-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', 
u'connection': 'close', 'status': '200', u'content-length': '904', 
u'x-openstack-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', u'date': 
'Mon, 26 Jun 2017
   13:13:43 GMT'}
  Body: {"volume": {"status": "attaching", "user_id": 
"8fa70618b61e4290908a964b6c8d11d9", "attachments": [], "links": [{"href": 
"http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";,
 "rel": "self"}, {"href": 
"http://127.0.0.1:8776/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";,
 "rel": "bookmark"}], "availability_zone": "nova", "bootable": "false", 
"encrypted": false, "created_at": "2017-06-26T13:13:29.00", "description": 
null, "os-vol-tenant-attr:tenant_id": "182a9178e3334660ad978457a2aecc9d", 
"updated_at": "2017-06-26T13:13:41.00", "volume_type

[Yahoo-eng-team] [Bug 1700583] Re: No volume Block Device Mapping in assisted snapshot with Quobyte

2017-08-03 Thread Silvan Kaiser
Ok, some more insights:
- i can workaround this issue by pinning the CIs DevStack version to commit 
0d9c896 . Using the following commit f3d5331 shows the issues described above. 
So this seems to be a DevStack related issue
- So far i could not determine why this is caused but the affecting change 
relates to Matts Email note on openstack-dev: 
http://lists.openstack.org/pipermail/openstack-dev/2017-July/120120.html

So, maybe this is only a DevStack config issue with my setup or this
might be a code issue in Nova. Help for debugging this is appreciated.

** Also affects: devstack
   Importance: Undecided
   Status: New

** Description changed:

+ *Update*: Please see comment #5 for details on how DevStack is involved in 
this issue.
+ Also more current CI runs with this issue can be found at:
+ - http://78.46.57.153:8081/refs-changes-95-490195-1/
+ - http://78.46.57.153:8081/refs-changes-21-490021-3/
+ */Update*
+ 
+ 
  Since roughly last friday Quobyte CIs with a Cinder / Nova setup fail with an 
error during assisted snapshots in four tests:
  
test_create_ebs_image_and_check_boot|test_snapshot_create_delete_with_volume_in_use|test_snapshot_create_offline_delete_online|test_volume_boot_pattern
  
  The error declares that the volume to be snapshotted has no block device
  mapping.
  
  I currently cannot identify if this is a Nova or Cinder issue, so please
  keep an open mind when looking into this.
  
  Complete example CI test runs with all logs can be found at e.g.:
  
- 
  http://78.46.57.153:8081/refs-changes-26-475226-3/
  http://78.46.57.153:8081/refs-changes-02-476402-2/
- 
  
  n-api log excerpt:
  [...]
  2017-06-23 14:24:18.137 10730 DEBUG nova.api.openstack.wsgi 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Action: 'create', 
calling method: >, body: {"snapshot": {"create_info": {"snapshot_id": 
"30bd2ad6-37a0-4027-a24e-00657adb0cca", "type": "qcow2", "new_file": 
"volume-43245219-bd54-4b03-a371-cca5836e58e5.30bd2ad6-37a0-4027-a24e-00657adb0cca"},
 "volume_id": "43245219-bd54-4b03-a371-cca5836e58e5"}} _process_stack 
/opt/stack/nova/nova/api/openstack/wsgi.py:609
  2017-06-23 14:24:18.438 10730 INFO nova.api.openstack.wsgi 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] HTTP exception thrown: 
No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5.: 
HTTPBadRequest: No volume Block Device Mapping with id 
43245219-bd54-4b03-a371-cca5836e58e5.
  2017-06-23 14:24:18.440 10730 DEBUG nova.api.openstack.wsgi 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Returning 400 to user: 
No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5. 
__call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1029
  2017-06-23 14:24:18.441 10730 INFO nova.osapi_compute.wsgi.server 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] 127.0.0.1 "POST 
/v2.1/os-assisted-volume-snapshots HTTP/1.1" status: 400 len: 543 time: 
0.6530399
  2017-06-23 14:24:18.770 10730 INFO nova.osapi_compute.wsgi.server 
[req-a0c161b6-12ba-4dbf-aa10-5fa34565441c 
tempest-TestVolumeBootPattern-1485741991 
tempest-TestVolumeBootPattern-1485741991] 127.0.0.1 "GET 
/v2.1/servers/0fcae274-2d09-45e7-ba84-df2b53526895 HTTP/1.1" status: 200 len: 
1762 time: 0.8687029
  [...]
  
- 
- Reading the test logs I see for instance that this error is thrown right 
after the volume attachment 
+ Reading the test logs I see for instance that this error is thrown right 
after the volume attachment
  to the VM is successful. I'd expect the snapshot API call to be able to find 
the mapping in that case. Example snippet:
  
+  [... attaching volume to VM]
  
-  [... attaching volume to VM]
- 
- 2017-06-26 13:13:43,182 28633 INFO [tempest.lib.common.rest_client] 
Request (VolumesSnapshotTestJSON:test_snapshot_create_offline_delete_online): 
200 GET 
http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097
 0.223s
- 2017-06-26 13:13:43,182 28633 DEBUG[tempest.lib.common.rest_client] 
Request - Headers: {'Content-Type': 'application/json', 'Accept': 
'application/json', 'X-Auth-Token': ''}
- Body: None
- Response - Headers: {u'content-type': 'application/json', 
'content-location': 
'http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097',
 u'x-compute-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', 
u'connection': 'close', 'status': '200', u'content-length': '904', 
u'x-openstack-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', u'date': 
'Mon, 26 Jun 2017
-  13:13:43 GMT'}
- Body: {"volume": {"status": "attaching", "user_id": 
"8fa70618b61e4290908a964b6c8d11d9", "attachments": [], "links": [{"href": 
"http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";,
 "rel": "self"}, {"href": 
"http://127.0.0.1:8776/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";,

[Yahoo-eng-team] [Bug 1700583] Re: No volume Block Device Mapping in assisted snapshot with Quobyte

2017-08-03 Thread Silvan Kaiser
This issue is back hitting our Cinder and Nova CIs. I'm digging for more
information and will post updates here.

** Changed in: nova
   Status: Invalid => New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1700583

Title:
  No volume Block Device Mapping in assisted snapshot with Quobyte

Status in OpenStack Compute (nova):
  New

Bug description:
  Since roughly last friday Quobyte CIs with a Cinder / Nova setup fail with an 
error during assisted snapshots in four tests:
  
test_create_ebs_image_and_check_boot|test_snapshot_create_delete_with_volume_in_use|test_snapshot_create_offline_delete_online|test_volume_boot_pattern

  The error declares that the volume to be snapshotted has no block
  device mapping.

  I currently cannot identify if this is a Nova or Cinder issue, so
  please keep an open mind when looking into this.

  Complete example CI test runs with all logs can be found at e.g.:

  
  http://78.46.57.153:8081/refs-changes-26-475226-3/
  http://78.46.57.153:8081/refs-changes-02-476402-2/

  
  n-api log excerpt:
  [...]
  2017-06-23 14:24:18.137 10730 DEBUG nova.api.openstack.wsgi 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Action: 'create', 
calling method: >, body: {"snapshot": {"create_info": {"snapshot_id": 
"30bd2ad6-37a0-4027-a24e-00657adb0cca", "type": "qcow2", "new_file": 
"volume-43245219-bd54-4b03-a371-cca5836e58e5.30bd2ad6-37a0-4027-a24e-00657adb0cca"},
 "volume_id": "43245219-bd54-4b03-a371-cca5836e58e5"}} _process_stack 
/opt/stack/nova/nova/api/openstack/wsgi.py:609
  2017-06-23 14:24:18.438 10730 INFO nova.api.openstack.wsgi 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] HTTP exception thrown: 
No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5.: 
HTTPBadRequest: No volume Block Device Mapping with id 
43245219-bd54-4b03-a371-cca5836e58e5.
  2017-06-23 14:24:18.440 10730 DEBUG nova.api.openstack.wsgi 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Returning 400 to user: 
No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5. 
__call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1029
  2017-06-23 14:24:18.441 10730 INFO nova.osapi_compute.wsgi.server 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] 127.0.0.1 "POST 
/v2.1/os-assisted-volume-snapshots HTTP/1.1" status: 400 len: 543 time: 
0.6530399
  2017-06-23 14:24:18.770 10730 INFO nova.osapi_compute.wsgi.server 
[req-a0c161b6-12ba-4dbf-aa10-5fa34565441c 
tempest-TestVolumeBootPattern-1485741991 
tempest-TestVolumeBootPattern-1485741991] 127.0.0.1 "GET 
/v2.1/servers/0fcae274-2d09-45e7-ba84-df2b53526895 HTTP/1.1" status: 200 len: 
1762 time: 0.8687029
  [...]

  
  Reading the test logs I see for instance that this error is thrown right 
after the volume attachment 
  to the VM is successful. I'd expect the snapshot API call to be able to find 
the mapping in that case. Example snippet:

  
   [... attaching volume to VM]

  2017-06-26 13:13:43,182 28633 INFO [tempest.lib.common.rest_client] 
Request (VolumesSnapshotTestJSON:test_snapshot_create_offline_delete_online): 
200 GET 
http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097
 0.223s
  2017-06-26 13:13:43,182 28633 DEBUG[tempest.lib.common.rest_client] 
Request - Headers: {'Content-Type': 'application/json', 'Accept': 
'application/json', 'X-Auth-Token': ''}
  Body: None
  Response - Headers: {u'content-type': 'application/json', 
'content-location': 
'http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097',
 u'x-compute-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', 
u'connection': 'close', 'status': '200', u'content-length': '904', 
u'x-openstack-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', u'date': 
'Mon, 26 Jun 2017
   13:13:43 GMT'}
  Body: {"volume": {"status": "attaching", "user_id": 
"8fa70618b61e4290908a964b6c8d11d9", "attachments": [], "links": [{"href": 
"http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";,
 "rel": "self"}, {"href": 
"http://127.0.0.1:8776/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";,
 "rel": "bookmark"}], "availability_zone": "nova", "bootable": "false", 
"encrypted": false, "created_at": "2017-06-26T13:13:29.00", "description": 
null, "os-vol-tenant-attr:tenant_id": "182a9178e3334660ad978457a2aecc9d", 
"updated_at": "2017-06-26T13:13:41.00", "volume_type": "Quobyte", "name": 
"tempest-VolumesSnapshotTestJSON-Volume-1014042071", "replication_status": 
null, "consistencygroup_id": null, "source_volid": null, "snapshot_id": null, 
"multiattach": false, "metadata": {}, "id": 
"df77bf3f-8602-4f28-a999-f4c73a3e9097", "size": 1}}
  2017-06-26 13:13:44,333 28633 INFO [temp

[Yahoo-eng-team] [Bug 1700583] Re: No volume Block Device Mapping in assisted snapshot with Quobyte

2017-06-27 Thread Silvan Kaiser
Verified builds are working again, even on old and previously failing
changes. This means the issue was somewhere in the DevStack/tempest/...
env and was solved yesterday by somebody. :)

** Changed in: nova
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1700583

Title:
  No volume Block Device Mapping in assisted snapshot with Quobyte

Status in OpenStack Compute (nova):
  Invalid

Bug description:
  Since roughly last friday Quobyte CIs with a Cinder / Nova setup fail with an 
error during assisted snapshots in four tests:
  
test_create_ebs_image_and_check_boot|test_snapshot_create_delete_with_volume_in_use|test_snapshot_create_offline_delete_online|test_volume_boot_pattern

  The error declares that the volume to be snapshotted has no block
  device mapping.

  I currently cannot identify if this is a Nova or Cinder issue, so
  please keep an open mind when looking into this.

  Complete example CI test runs with all logs can be found at e.g.:

  
  http://78.46.57.153:8081/refs-changes-26-475226-3/
  http://78.46.57.153:8081/refs-changes-02-476402-2/

  
  n-api log excerpt:
  [...]
  2017-06-23 14:24:18.137 10730 DEBUG nova.api.openstack.wsgi 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Action: 'create', 
calling method: >, body: {"snapshot": {"create_info": {"snapshot_id": 
"30bd2ad6-37a0-4027-a24e-00657adb0cca", "type": "qcow2", "new_file": 
"volume-43245219-bd54-4b03-a371-cca5836e58e5.30bd2ad6-37a0-4027-a24e-00657adb0cca"},
 "volume_id": "43245219-bd54-4b03-a371-cca5836e58e5"}} _process_stack 
/opt/stack/nova/nova/api/openstack/wsgi.py:609
  2017-06-23 14:24:18.438 10730 INFO nova.api.openstack.wsgi 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] HTTP exception thrown: 
No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5.: 
HTTPBadRequest: No volume Block Device Mapping with id 
43245219-bd54-4b03-a371-cca5836e58e5.
  2017-06-23 14:24:18.440 10730 DEBUG nova.api.openstack.wsgi 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Returning 400 to user: 
No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5. 
__call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1029
  2017-06-23 14:24:18.441 10730 INFO nova.osapi_compute.wsgi.server 
[req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] 127.0.0.1 "POST 
/v2.1/os-assisted-volume-snapshots HTTP/1.1" status: 400 len: 543 time: 
0.6530399
  2017-06-23 14:24:18.770 10730 INFO nova.osapi_compute.wsgi.server 
[req-a0c161b6-12ba-4dbf-aa10-5fa34565441c 
tempest-TestVolumeBootPattern-1485741991 
tempest-TestVolumeBootPattern-1485741991] 127.0.0.1 "GET 
/v2.1/servers/0fcae274-2d09-45e7-ba84-df2b53526895 HTTP/1.1" status: 200 len: 
1762 time: 0.8687029
  [...]

  
  Reading the test logs I see for instance that this error is thrown right 
after the volume attachment 
  to the VM is successful. I'd expect the snapshot API call to be able to find 
the mapping in that case. Example snippet:

  
   [... attaching volume to VM]

  2017-06-26 13:13:43,182 28633 INFO [tempest.lib.common.rest_client] 
Request (VolumesSnapshotTestJSON:test_snapshot_create_offline_delete_online): 
200 GET 
http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097
 0.223s
  2017-06-26 13:13:43,182 28633 DEBUG[tempest.lib.common.rest_client] 
Request - Headers: {'Content-Type': 'application/json', 'Accept': 
'application/json', 'X-Auth-Token': ''}
  Body: None
  Response - Headers: {u'content-type': 'application/json', 
'content-location': 
'http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097',
 u'x-compute-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', 
u'connection': 'close', 'status': '200', u'content-length': '904', 
u'x-openstack-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', u'date': 
'Mon, 26 Jun 2017
   13:13:43 GMT'}
  Body: {"volume": {"status": "attaching", "user_id": 
"8fa70618b61e4290908a964b6c8d11d9", "attachments": [], "links": [{"href": 
"http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";,
 "rel": "self"}, {"href": 
"http://127.0.0.1:8776/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";,
 "rel": "bookmark"}], "availability_zone": "nova", "bootable": "false", 
"encrypted": false, "created_at": "2017-06-26T13:13:29.00", "description": 
null, "os-vol-tenant-attr:tenant_id": "182a9178e3334660ad978457a2aecc9d", 
"updated_at": "2017-06-26T13:13:41.00", "volume_type": "Quobyte", "name": 
"tempest-VolumesSnapshotTestJSON-Volume-1014042071", "replication_status": 
null, "consistencygroup_id": null, "source_volid": null, "snapshot_id": null, 
"multiattach": false, "metadata": {}, "id": 
"df77bf3f-8602-4f28-a999