[Yahoo-eng-team] [Bug 1700583] Re: No volume Block Device Mapping in assisted snapshot with Quobyte
Whatever the exact reason was it no longer occurs. I'm currently back to run our CIs with the current DevStack master and no longer hit this, therefore marking as invalid. ** Changed in: devstack Status: New => Invalid ** Changed in: nova Status: Opinion => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1700583 Title: No volume Block Device Mapping in assisted snapshot with Quobyte Status in devstack: Invalid Status in OpenStack Compute (nova): Invalid Bug description: *Update*: Please see comment #5 for details on how DevStack is involved in this issue. Also more current CI runs with this issue can be found at: - http://78.46.57.153:8081/refs-changes-95-490195-1/ - http://78.46.57.153:8081/refs-changes-21-490021-3/ */Update* Since roughly last friday Quobyte CIs with a Cinder / Nova setup fail with an error during assisted snapshots in four tests: test_create_ebs_image_and_check_boot|test_snapshot_create_delete_with_volume_in_use|test_snapshot_create_offline_delete_online|test_volume_boot_pattern The error declares that the volume to be snapshotted has no block device mapping. I currently cannot identify if this is a Nova or Cinder issue, so please keep an open mind when looking into this. Complete example CI test runs with all logs can be found at e.g.: http://78.46.57.153:8081/refs-changes-26-475226-3/ http://78.46.57.153:8081/refs-changes-02-476402-2/ n-api log excerpt: [...] 2017-06-23 14:24:18.137 10730 DEBUG nova.api.openstack.wsgi [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Action: 'create', calling method: >, body: {"snapshot": {"create_info": {"snapshot_id": "30bd2ad6-37a0-4027-a24e-00657adb0cca", "type": "qcow2", "new_file": "volume-43245219-bd54-4b03-a371-cca5836e58e5.30bd2ad6-37a0-4027-a24e-00657adb0cca"}, "volume_id": "43245219-bd54-4b03-a371-cca5836e58e5"}} _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:609 2017-06-23 14:24:18.438 10730 INFO nova.api.openstack.wsgi [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] HTTP exception thrown: No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5.: HTTPBadRequest: No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5. 2017-06-23 14:24:18.440 10730 DEBUG nova.api.openstack.wsgi [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Returning 400 to user: No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5. __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1029 2017-06-23 14:24:18.441 10730 INFO nova.osapi_compute.wsgi.server [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] 127.0.0.1 "POST /v2.1/os-assisted-volume-snapshots HTTP/1.1" status: 400 len: 543 time: 0.6530399 2017-06-23 14:24:18.770 10730 INFO nova.osapi_compute.wsgi.server [req-a0c161b6-12ba-4dbf-aa10-5fa34565441c tempest-TestVolumeBootPattern-1485741991 tempest-TestVolumeBootPattern-1485741991] 127.0.0.1 "GET /v2.1/servers/0fcae274-2d09-45e7-ba84-df2b53526895 HTTP/1.1" status: 200 len: 1762 time: 0.8687029 [...] Reading the test logs I see for instance that this error is thrown right after the volume attachment to the VM is successful. I'd expect the snapshot API call to be able to find the mapping in that case. Example snippet: [... attaching volume to VM] 2017-06-26 13:13:43,182 28633 INFO [tempest.lib.common.rest_client] Request (VolumesSnapshotTestJSON:test_snapshot_create_offline_delete_online): 200 GET http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097 0.223s 2017-06-26 13:13:43,182 28633 DEBUG[tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': ''} Body: None Response - Headers: {u'content-type': 'application/json', 'content-location': 'http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097', u'x-compute-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', u'connection': 'close', 'status': '200', u'content-length': '904', u'x-openstack-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', u'date': 'Mon, 26 Jun 2017 13:13:43 GMT'} Body: {"volume": {"status": "attaching", "user_id": "8fa70618b61e4290908a964b6c8d11d9", "attachments": [], "links": [{"href": "http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";, "rel": "self"}, {"href": "http://127.0.0.1:8776/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";, "rel": "bookmark"}], "availability_zone": "nova", "bootable": "false", "encrypted": false, "created_at": "2017-06-26T13:13:29.00", "description": null, "os-vol-tenant-attr:tenant_id": "1
[Yahoo-eng-team] [Bug 1700583] Re: No volume Block Device Mapping in assisted snapshot with Quobyte
Putting in Opinion as I'm not sure it's a Nova problem. Silvan, if you need some help for debugging, just ask in our IRC channel if you want. ** Changed in: nova Status: New => Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1700583 Title: No volume Block Device Mapping in assisted snapshot with Quobyte Status in devstack: New Status in OpenStack Compute (nova): Opinion Bug description: *Update*: Please see comment #5 for details on how DevStack is involved in this issue. Also more current CI runs with this issue can be found at: - http://78.46.57.153:8081/refs-changes-95-490195-1/ - http://78.46.57.153:8081/refs-changes-21-490021-3/ */Update* Since roughly last friday Quobyte CIs with a Cinder / Nova setup fail with an error during assisted snapshots in four tests: test_create_ebs_image_and_check_boot|test_snapshot_create_delete_with_volume_in_use|test_snapshot_create_offline_delete_online|test_volume_boot_pattern The error declares that the volume to be snapshotted has no block device mapping. I currently cannot identify if this is a Nova or Cinder issue, so please keep an open mind when looking into this. Complete example CI test runs with all logs can be found at e.g.: http://78.46.57.153:8081/refs-changes-26-475226-3/ http://78.46.57.153:8081/refs-changes-02-476402-2/ n-api log excerpt: [...] 2017-06-23 14:24:18.137 10730 DEBUG nova.api.openstack.wsgi [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Action: 'create', calling method: >, body: {"snapshot": {"create_info": {"snapshot_id": "30bd2ad6-37a0-4027-a24e-00657adb0cca", "type": "qcow2", "new_file": "volume-43245219-bd54-4b03-a371-cca5836e58e5.30bd2ad6-37a0-4027-a24e-00657adb0cca"}, "volume_id": "43245219-bd54-4b03-a371-cca5836e58e5"}} _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:609 2017-06-23 14:24:18.438 10730 INFO nova.api.openstack.wsgi [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] HTTP exception thrown: No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5.: HTTPBadRequest: No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5. 2017-06-23 14:24:18.440 10730 DEBUG nova.api.openstack.wsgi [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Returning 400 to user: No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5. __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1029 2017-06-23 14:24:18.441 10730 INFO nova.osapi_compute.wsgi.server [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] 127.0.0.1 "POST /v2.1/os-assisted-volume-snapshots HTTP/1.1" status: 400 len: 543 time: 0.6530399 2017-06-23 14:24:18.770 10730 INFO nova.osapi_compute.wsgi.server [req-a0c161b6-12ba-4dbf-aa10-5fa34565441c tempest-TestVolumeBootPattern-1485741991 tempest-TestVolumeBootPattern-1485741991] 127.0.0.1 "GET /v2.1/servers/0fcae274-2d09-45e7-ba84-df2b53526895 HTTP/1.1" status: 200 len: 1762 time: 0.8687029 [...] Reading the test logs I see for instance that this error is thrown right after the volume attachment to the VM is successful. I'd expect the snapshot API call to be able to find the mapping in that case. Example snippet: [... attaching volume to VM] 2017-06-26 13:13:43,182 28633 INFO [tempest.lib.common.rest_client] Request (VolumesSnapshotTestJSON:test_snapshot_create_offline_delete_online): 200 GET http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097 0.223s 2017-06-26 13:13:43,182 28633 DEBUG[tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': ''} Body: None Response - Headers: {u'content-type': 'application/json', 'content-location': 'http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097', u'x-compute-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', u'connection': 'close', 'status': '200', u'content-length': '904', u'x-openstack-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', u'date': 'Mon, 26 Jun 2017 13:13:43 GMT'} Body: {"volume": {"status": "attaching", "user_id": "8fa70618b61e4290908a964b6c8d11d9", "attachments": [], "links": [{"href": "http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";, "rel": "self"}, {"href": "http://127.0.0.1:8776/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";, "rel": "bookmark"}], "availability_zone": "nova", "bootable": "false", "encrypted": false, "created_at": "2017-06-26T13:13:29.00", "description": null, "os-vol-tenant-attr:tenant_id": "182a9178e3334660ad978457a2aecc9d", "updated_at": "2017-06-26T13:13:41.00", "volume_type
[Yahoo-eng-team] [Bug 1700583] Re: No volume Block Device Mapping in assisted snapshot with Quobyte
Ok, some more insights: - i can workaround this issue by pinning the CIs DevStack version to commit 0d9c896 . Using the following commit f3d5331 shows the issues described above. So this seems to be a DevStack related issue - So far i could not determine why this is caused but the affecting change relates to Matts Email note on openstack-dev: http://lists.openstack.org/pipermail/openstack-dev/2017-July/120120.html So, maybe this is only a DevStack config issue with my setup or this might be a code issue in Nova. Help for debugging this is appreciated. ** Also affects: devstack Importance: Undecided Status: New ** Description changed: + *Update*: Please see comment #5 for details on how DevStack is involved in this issue. + Also more current CI runs with this issue can be found at: + - http://78.46.57.153:8081/refs-changes-95-490195-1/ + - http://78.46.57.153:8081/refs-changes-21-490021-3/ + */Update* + + Since roughly last friday Quobyte CIs with a Cinder / Nova setup fail with an error during assisted snapshots in four tests: test_create_ebs_image_and_check_boot|test_snapshot_create_delete_with_volume_in_use|test_snapshot_create_offline_delete_online|test_volume_boot_pattern The error declares that the volume to be snapshotted has no block device mapping. I currently cannot identify if this is a Nova or Cinder issue, so please keep an open mind when looking into this. Complete example CI test runs with all logs can be found at e.g.: - http://78.46.57.153:8081/refs-changes-26-475226-3/ http://78.46.57.153:8081/refs-changes-02-476402-2/ - n-api log excerpt: [...] 2017-06-23 14:24:18.137 10730 DEBUG nova.api.openstack.wsgi [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Action: 'create', calling method: >, body: {"snapshot": {"create_info": {"snapshot_id": "30bd2ad6-37a0-4027-a24e-00657adb0cca", "type": "qcow2", "new_file": "volume-43245219-bd54-4b03-a371-cca5836e58e5.30bd2ad6-37a0-4027-a24e-00657adb0cca"}, "volume_id": "43245219-bd54-4b03-a371-cca5836e58e5"}} _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:609 2017-06-23 14:24:18.438 10730 INFO nova.api.openstack.wsgi [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] HTTP exception thrown: No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5.: HTTPBadRequest: No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5. 2017-06-23 14:24:18.440 10730 DEBUG nova.api.openstack.wsgi [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Returning 400 to user: No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5. __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1029 2017-06-23 14:24:18.441 10730 INFO nova.osapi_compute.wsgi.server [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] 127.0.0.1 "POST /v2.1/os-assisted-volume-snapshots HTTP/1.1" status: 400 len: 543 time: 0.6530399 2017-06-23 14:24:18.770 10730 INFO nova.osapi_compute.wsgi.server [req-a0c161b6-12ba-4dbf-aa10-5fa34565441c tempest-TestVolumeBootPattern-1485741991 tempest-TestVolumeBootPattern-1485741991] 127.0.0.1 "GET /v2.1/servers/0fcae274-2d09-45e7-ba84-df2b53526895 HTTP/1.1" status: 200 len: 1762 time: 0.8687029 [...] - - Reading the test logs I see for instance that this error is thrown right after the volume attachment + Reading the test logs I see for instance that this error is thrown right after the volume attachment to the VM is successful. I'd expect the snapshot API call to be able to find the mapping in that case. Example snippet: + [... attaching volume to VM] - [... attaching volume to VM] - - 2017-06-26 13:13:43,182 28633 INFO [tempest.lib.common.rest_client] Request (VolumesSnapshotTestJSON:test_snapshot_create_offline_delete_online): 200 GET http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097 0.223s - 2017-06-26 13:13:43,182 28633 DEBUG[tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': ''} - Body: None - Response - Headers: {u'content-type': 'application/json', 'content-location': 'http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097', u'x-compute-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', u'connection': 'close', 'status': '200', u'content-length': '904', u'x-openstack-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', u'date': 'Mon, 26 Jun 2017 - 13:13:43 GMT'} - Body: {"volume": {"status": "attaching", "user_id": "8fa70618b61e4290908a964b6c8d11d9", "attachments": [], "links": [{"href": "http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";, "rel": "self"}, {"href": "http://127.0.0.1:8776/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";,
[Yahoo-eng-team] [Bug 1700583] Re: No volume Block Device Mapping in assisted snapshot with Quobyte
This issue is back hitting our Cinder and Nova CIs. I'm digging for more information and will post updates here. ** Changed in: nova Status: Invalid => New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1700583 Title: No volume Block Device Mapping in assisted snapshot with Quobyte Status in OpenStack Compute (nova): New Bug description: Since roughly last friday Quobyte CIs with a Cinder / Nova setup fail with an error during assisted snapshots in four tests: test_create_ebs_image_and_check_boot|test_snapshot_create_delete_with_volume_in_use|test_snapshot_create_offline_delete_online|test_volume_boot_pattern The error declares that the volume to be snapshotted has no block device mapping. I currently cannot identify if this is a Nova or Cinder issue, so please keep an open mind when looking into this. Complete example CI test runs with all logs can be found at e.g.: http://78.46.57.153:8081/refs-changes-26-475226-3/ http://78.46.57.153:8081/refs-changes-02-476402-2/ n-api log excerpt: [...] 2017-06-23 14:24:18.137 10730 DEBUG nova.api.openstack.wsgi [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Action: 'create', calling method: >, body: {"snapshot": {"create_info": {"snapshot_id": "30bd2ad6-37a0-4027-a24e-00657adb0cca", "type": "qcow2", "new_file": "volume-43245219-bd54-4b03-a371-cca5836e58e5.30bd2ad6-37a0-4027-a24e-00657adb0cca"}, "volume_id": "43245219-bd54-4b03-a371-cca5836e58e5"}} _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:609 2017-06-23 14:24:18.438 10730 INFO nova.api.openstack.wsgi [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] HTTP exception thrown: No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5.: HTTPBadRequest: No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5. 2017-06-23 14:24:18.440 10730 DEBUG nova.api.openstack.wsgi [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Returning 400 to user: No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5. __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1029 2017-06-23 14:24:18.441 10730 INFO nova.osapi_compute.wsgi.server [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] 127.0.0.1 "POST /v2.1/os-assisted-volume-snapshots HTTP/1.1" status: 400 len: 543 time: 0.6530399 2017-06-23 14:24:18.770 10730 INFO nova.osapi_compute.wsgi.server [req-a0c161b6-12ba-4dbf-aa10-5fa34565441c tempest-TestVolumeBootPattern-1485741991 tempest-TestVolumeBootPattern-1485741991] 127.0.0.1 "GET /v2.1/servers/0fcae274-2d09-45e7-ba84-df2b53526895 HTTP/1.1" status: 200 len: 1762 time: 0.8687029 [...] Reading the test logs I see for instance that this error is thrown right after the volume attachment to the VM is successful. I'd expect the snapshot API call to be able to find the mapping in that case. Example snippet: [... attaching volume to VM] 2017-06-26 13:13:43,182 28633 INFO [tempest.lib.common.rest_client] Request (VolumesSnapshotTestJSON:test_snapshot_create_offline_delete_online): 200 GET http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097 0.223s 2017-06-26 13:13:43,182 28633 DEBUG[tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': ''} Body: None Response - Headers: {u'content-type': 'application/json', 'content-location': 'http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097', u'x-compute-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', u'connection': 'close', 'status': '200', u'content-length': '904', u'x-openstack-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', u'date': 'Mon, 26 Jun 2017 13:13:43 GMT'} Body: {"volume": {"status": "attaching", "user_id": "8fa70618b61e4290908a964b6c8d11d9", "attachments": [], "links": [{"href": "http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";, "rel": "self"}, {"href": "http://127.0.0.1:8776/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";, "rel": "bookmark"}], "availability_zone": "nova", "bootable": "false", "encrypted": false, "created_at": "2017-06-26T13:13:29.00", "description": null, "os-vol-tenant-attr:tenant_id": "182a9178e3334660ad978457a2aecc9d", "updated_at": "2017-06-26T13:13:41.00", "volume_type": "Quobyte", "name": "tempest-VolumesSnapshotTestJSON-Volume-1014042071", "replication_status": null, "consistencygroup_id": null, "source_volid": null, "snapshot_id": null, "multiattach": false, "metadata": {}, "id": "df77bf3f-8602-4f28-a999-f4c73a3e9097", "size": 1}} 2017-06-26 13:13:44,333 28633 INFO [temp
[Yahoo-eng-team] [Bug 1700583] Re: No volume Block Device Mapping in assisted snapshot with Quobyte
Verified builds are working again, even on old and previously failing changes. This means the issue was somewhere in the DevStack/tempest/... env and was solved yesterday by somebody. :) ** Changed in: nova Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1700583 Title: No volume Block Device Mapping in assisted snapshot with Quobyte Status in OpenStack Compute (nova): Invalid Bug description: Since roughly last friday Quobyte CIs with a Cinder / Nova setup fail with an error during assisted snapshots in four tests: test_create_ebs_image_and_check_boot|test_snapshot_create_delete_with_volume_in_use|test_snapshot_create_offline_delete_online|test_volume_boot_pattern The error declares that the volume to be snapshotted has no block device mapping. I currently cannot identify if this is a Nova or Cinder issue, so please keep an open mind when looking into this. Complete example CI test runs with all logs can be found at e.g.: http://78.46.57.153:8081/refs-changes-26-475226-3/ http://78.46.57.153:8081/refs-changes-02-476402-2/ n-api log excerpt: [...] 2017-06-23 14:24:18.137 10730 DEBUG nova.api.openstack.wsgi [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Action: 'create', calling method: >, body: {"snapshot": {"create_info": {"snapshot_id": "30bd2ad6-37a0-4027-a24e-00657adb0cca", "type": "qcow2", "new_file": "volume-43245219-bd54-4b03-a371-cca5836e58e5.30bd2ad6-37a0-4027-a24e-00657adb0cca"}, "volume_id": "43245219-bd54-4b03-a371-cca5836e58e5"}} _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:609 2017-06-23 14:24:18.438 10730 INFO nova.api.openstack.wsgi [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] HTTP exception thrown: No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5.: HTTPBadRequest: No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5. 2017-06-23 14:24:18.440 10730 DEBUG nova.api.openstack.wsgi [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] Returning 400 to user: No volume Block Device Mapping with id 43245219-bd54-4b03-a371-cca5836e58e5. __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1029 2017-06-23 14:24:18.441 10730 INFO nova.osapi_compute.wsgi.server [req-484e4bd2-e091-4411-9252-38549f7c9dbd admin admin] 127.0.0.1 "POST /v2.1/os-assisted-volume-snapshots HTTP/1.1" status: 400 len: 543 time: 0.6530399 2017-06-23 14:24:18.770 10730 INFO nova.osapi_compute.wsgi.server [req-a0c161b6-12ba-4dbf-aa10-5fa34565441c tempest-TestVolumeBootPattern-1485741991 tempest-TestVolumeBootPattern-1485741991] 127.0.0.1 "GET /v2.1/servers/0fcae274-2d09-45e7-ba84-df2b53526895 HTTP/1.1" status: 200 len: 1762 time: 0.8687029 [...] Reading the test logs I see for instance that this error is thrown right after the volume attachment to the VM is successful. I'd expect the snapshot API call to be able to find the mapping in that case. Example snippet: [... attaching volume to VM] 2017-06-26 13:13:43,182 28633 INFO [tempest.lib.common.rest_client] Request (VolumesSnapshotTestJSON:test_snapshot_create_offline_delete_online): 200 GET http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097 0.223s 2017-06-26 13:13:43,182 28633 DEBUG[tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': ''} Body: None Response - Headers: {u'content-type': 'application/json', 'content-location': 'http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097', u'x-compute-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', u'connection': 'close', 'status': '200', u'content-length': '904', u'x-openstack-request-id': 'req-fc80f9a0-dd4c-4e9d-b044-39007bbcdbae', u'date': 'Mon, 26 Jun 2017 13:13:43 GMT'} Body: {"volume": {"status": "attaching", "user_id": "8fa70618b61e4290908a964b6c8d11d9", "attachments": [], "links": [{"href": "http://127.0.0.1:8776/v2/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";, "rel": "self"}, {"href": "http://127.0.0.1:8776/182a9178e3334660ad978457a2aecc9d/volumes/df77bf3f-8602-4f28-a999-f4c73a3e9097";, "rel": "bookmark"}], "availability_zone": "nova", "bootable": "false", "encrypted": false, "created_at": "2017-06-26T13:13:29.00", "description": null, "os-vol-tenant-attr:tenant_id": "182a9178e3334660ad978457a2aecc9d", "updated_at": "2017-06-26T13:13:41.00", "volume_type": "Quobyte", "name": "tempest-VolumesSnapshotTestJSON-Volume-1014042071", "replication_status": null, "consistencygroup_id": null, "source_volid": null, "snapshot_id": null, "multiattach": false, "metadata": {}, "id": "df77bf3f-8602-4f28-a999