Reviewed: https://review.openstack.org/538961
Committed:
https://git.openstack.org/cgit/openstack/nova/commit/?id=4a2c9a4887a219a6d4dfe83c430b040713fc4109
Submitter: Zuul
Branch: master
commit 4a2c9a4887a219a6d4dfe83c430b040713fc4109
Author: Matt Riedemann <mriedem...@gmail.com>
Date: Mon Jan 29 10:50:36 2018 -0500
Rollback instance.image_ref on failed rebuild
When rebuilding and changing the image, we run the new image
through the scheduler to see if it's valid for the instance
on its current compute host. The API saves off the new image
ref on the instance before casting to conductor to run through
the scheduler. If the scheduler fails, the instance.image_ref was
not being rolled back, which meant a user could attempt the rebuild
with the same invalid image a second time and the API, seeing the
instance.image_ref hasn't changed (even though it's not the actual
backing image for the server), will bypass the scheduler and rebuild
the instance with that invalid image.
This fixes the issue by using the original image ref, passed from
API to conductor during rebuild, to reset the instance.image_ref
in the case of a failure.
Note that there are other things changed on the instance in the API
which this patch does not attempt to recover as that's a bigger
work item which likely involves substantial refactoring of the code.
Closes-Bug: #1746032
Change-Id: I3399a66fe9b1297cd6b0dca440145393ceaef41f
** Changed in: nova
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1746032
Title:
By rebuilding twice with the same "forbidden" image one can circumvent
scheduler rebuild restrictions
Status in OpenStack Compute (nova):
Fix Released
Status in OpenStack Compute (nova) newton series:
Won't Fix
Status in OpenStack Compute (nova) ocata series:
In Progress
Status in OpenStack Compute (nova) pike series:
In Progress
Bug description:
Description
===========
Since CVE-2017-16239, we call to the scheduler when doing a rebuild
with a new image. If the scheduler refuses a rebuild because a filter
forbids the new image on the instance's host (for example,
IsolatedHostsFilter), at first there was no indication of this in the
API (bug 1744325). Currently, with the fix for bug 1744325 merged [1],
the instance goes to ERROR to indicate the refused rebuild. However,
by rebuilding again with the same "forbidden" image it is possible to
circumvent scheduler restrictions.
Steps to reproduce
==================
1. Configure IsolatedHostsFilter:
[filter_scheduler]
enabled_filters = [...],IsolatedHostsFilter
isolated_images = 41d3e5ca-14cf-436c-9413-4826b5c8bdb1
isolated_hosts = ubuntu
restrict_isolated_hosts_to_isolated_images = true
2. Have two images, one isolated and one not:
$ openstack image list
8d0581a5-ed9d-4b98-a766-a41efbc99929 | centos | active
41d3e5ca-14cf-436c-9413-4826b5c8bdb1 | cirros-0.3.5-x86_64-disk | active
cirros is the isolated one
3. Have only one hypervisor (the isolated one):
$ openstack hypervisor list
ubuntu | QEMU | 192.168.100.194 | up
5. Boot a cirros (isolated) image:
$ openstack server create \
--image 41d3e5ca-14cf-436c-9413-4826b5c8bdb1 \
--flavor m1.nano \
cirros-test-expect-success
$ openstack server list
cirros-test-expect-success | ACTIVE | [...] |
cirros-0.3.5-x86_64-disk | m1.nano
6. Rebuild the cirros instance with centos (this should be refused by
the scheduler):
$ nova --debug rebuild cirros-test-expect-success centos
DEBUG (session:722) POST call to compute for
http://192.168.100.194/compute/v2.1/servers/d9d98bf7-623e-4587-b82c-06f36abf59cb/action
used request id req-c234346a-6e05-47cf-a0cd-45f89d11e15d
8. Observe the instance going to ERROR,
but still showing the new centos image :
$ nova show cirros-test-expect-success
[...]
status | ERROR
image | centos (8d0581a5-ed9d-4b98-a766-a41efbc99929)
[...]
9. Rebuild again with the same centos image:
$ nova rebuild cirros-test-expect-success centos
10. The rebuild goes through.
Expected result
===============
At step 10, the rebuild should still be refused.
Actual result
=============
The rebuild is allowed.
Environment
===========
1. Exact version of OpenStack you are running. See the following
Was reported in Red Hat OpenStack 12, affects newton through
master.
2. Which hypervisor did you use?
libvirt+kvm
[1] https://review.openstack.org/#/c/536268/
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1746032/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
