On Fri, Jan 8, 2016 at 11:54 AM, Sam Matzek <[email protected]> wrote: > On Fri, Jan 8, 2016 at 8:31 AM, Flavio Percoco <[email protected]> wrote: >> On 29/12/15 07:41 -0600, Sam Matzek wrote: >>> >>> On Thu, Dec 24, 2015 at 7:49 AM, Mikhail Fedosin <[email protected]> >>> wrote: >>>> >>>> Hello, it's another topic about glance v2 adoption in Nova, but it's >>>> different from the others. I want to declare that there is a set of >>>> commits, >>>> that make Nova version agnostic and allow to work with both glance apis. >>>> The >>>> idea of the solution is to determine the current api version in the >>>> beginning and make appropriate requests after. >>>> (https://review.openstack.org/#/c/228578/, >>>> https://review.openstack.org/#/c/238309/, >>>> https://review.openstack.org/#/c/259097/) >>>> >>>> Indeed, it requires some additional (and painful) work, but now all >>>> tempest >>>> tests pass in Jenkins. >>>> >>>> Note: this thread is not about xenplugin, there is another topic, called >>>> 'Xenplugin + Glance_v2 = Hate' >>>> >>>> Here the main issues we faced and how we've solved them: >>>> >>>> 1. "changes-since" filter for image-list is not supported in v2 api. >>>> Steve >>>> Lewis made a great job and implemented a set of filters with comparison >>>> operators https://review.openstack.org/#/c/197388/. Filtering by >>>> 'changes-since' is completely similar to 'gte:updated_at'. >>>> >>>> 2. Filtering by custom properties must have prefix 'property-'. In v2 >>>> it's >>>> not required. >>>> >>>> 3. V2 states that all custom properties must be image attributes, but >>>> Nova >>>> assumes that they are included in 'properties' dictionary. It's fixed >>>> with >>>> glanceclient's method 'is_base_property(prop_name)', that returns False >>>> in >>>> case of custom property. >>>> >>>> 4. is_public=True/False is visibility="public"/"private" respectively. >>>> >>>> 5. Deleting custom image properties in Nova is performed with >>>> 'purge_props' >>>> flag. If it's set to True, then all prop names, that are not included in >>>> updated data, will be removed. In case of v2 we have to explicitly >>>> specify >>>> prop names in the list param 'remove_props'. To implement this behaviour, >>>> if >>>> 'purge_props' is set, we make additional 'get' request to determine the >>>> existing properties and put in 'remove_prop' list only those, that are >>>> not >>>> in updated_data. >>>> >>>> 6. My favourite: >>>> There is an ability to activate an empty image by just providing 'size = >>>> 0': >>>> https://review.openstack.org/#/c/9715/, in this case image will be a >>>> collection of metadata. Glance v2 doesn't support this "feature" and >>>> that's >>>> why we have to implement a very dirty workaround: >>>> * v2 requires, that disk_format and container-format must be set >>>> before >>>> the activation. if these params are not provided to 'create' method then >>>> we >>>> hardcode it to 'qcow2' and 'bare'. >>>> * we call 'upload' method with empty data (data = '') to activate >>>> image. >>>> Me (and the rest glance team) think that this image activation with >>>> zero-size is inconsistent and we won't implement it in v2. So, I'm going >>>> to >>>> ask if Nova really needs this feature and maybe it's possible to make it >>>> work without it. >>> >>> >>> Nova uses this functionality when it creates snapshots of volume >>> backed instances, that is, instances that only have Cinder volumes >>> attached and do not have an ephemeral disk. >>> In this case Nova API creates Cinder snapshots for the Cinder volumes >>> and builds the block_device_mapping property with block devices that >>> reference the Cinder snapshots. Nova activates this image with size=0 >>> because this image does not have a disk and simply refers to the >>> collection of Cinder snapshots that collectively comprise the binary >>> image. Callers of Glance outside of Nova may also use the APIs to >>> create "block device mapping" images as well that contain references >>> to Cinder volumes to attach, blank volumes to create, snapshots to >>> create volumes from, etc during the server creation. Not having the >>> ability to create these images with V2 is a loss of function. >> >> >> I disagree. Being able to activate empty images breaks the consistency >> of Glances API and I don't think it should be considered a feature but >> a bug. An active image is an image that can be used to *boot* a VM. If >> the image is empty, you simply can't do that. >> >> Note that pointing to a block device makes the image "non-empty" >> already. There's a spec merged[0] and code written[1] to allow for >> using cinder as backend for Glance. >> >> If there's a missing functionality for the mapping users require, I >> believe activating empty images is not the solution for it. >> >> [0] https://review.openstack.org/183363 >> [1] https://review.openstack.org/166414 >> >>> The callers could supply 1 byte of junk data, like a space character, >>> but that would result in a junk image being stored in Glance's default >>> backing store for the image. It would also give the impression that a >>> real disk image exists for the image in a backing store which could be >>> fetched, which is incorrect. >>> >>> If I remember correctly Glance V2 lets you transition an image from >>> queued to active state with size = 0 if the image is using an external >>> backing store such as http. The store is then called to fetch the >>> size. Would it be possible to allow Glance to consider images with >>> size = 0 and the block_device_mapping property to be "externally >>> sourced" and allow the transition? >> >> >> In v2 you can create an image and then add a location for it (this >> should be enabled, of course). You can even use cinder locations since >> Kilo (or Havana?). > > I don't believe Cinder backing store / Cinder locations can work for > the Nova use case here. In the case of this use case, Nova snapshot > of Cinder backed VM, Nova creates Cinder snapshots of every attached > Cinder volume and puts the snapshots in a BDM list in the Glance > metadata, and moves the image to Active with size = 0 and no volume. > > This inherently creates an image that is pointing at a collection of > multiple image binaries (Cinder snapshots) that collectively make up > the "image". For example the Cinder snapshots could be snapshot0 = > boot disk, snapshot1 = application binaries, snapshot1 = DB volume 1 > snapshot2 = DB volume 2. > > I may be wrong, but my understanding of multiple location support in > Glance is that the multiple locations are intended to all contain > copies / mirrors of the same image binary (singular), and the image > consumer, say, a Nova virt driver, could look at the locations and > make the best choice of location based on its settings for ephemeral > disk backing, etc. If this is the correct understanding then the > multiple locations support using Cinder the Cinder backing store to > reference multiple Cinder snapshots of different Cinder volumes is not > a good fit. > > Now possibly a compromise could exist where in the "snapshot volume > backed VM" flow, Nova changes to set location of the Glance image to > be the Cinder snapshot of the boot disk / root BDM, and then puts the > non-root BDM volumes in the block_device_mapping image metadata. > > This would make the image non-empty and would use the Cinder backing > store for the boot disk while maintaining the existing functionality > of having the image refer to the other Cinder snapshots in the block > device mapping metadata.
I sent the previous reply a bit prematurely. In order for my idea above to work, to have the Glance image use a Cinder location / backing store for the boot disk in this case, the Cinder backing store [1] needs to be enhanced to support Cinder snapshots as well as Cinder volumes. [1] https://github.com/openstack/glance_store/blob/master/glance_store/_drivers/cinder.py > > >> >> Cheers, >> Flavio >> >> -- >> @flaper87 >> Flavio Percoco >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: [email protected]?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
