On 09/06/15 07:38 +0000, Bhandaru, Malini K wrote:
A DB migrate script that puts some token default string only for glance properties that are NULL. Should not change anything else. Hope it works Flavio. Regards Malini
Proposed fix, please read the commit message and comments: https://review.openstack.org/#/c/189661/
-----Original Message----- From: Flavio Percoco [mailto:fla...@redhat.com] Sent: Tuesday, June 09, 2015 12:33 AM To: Bhandaru, Malini K Cc: OpenStack Development Mailing List (not for usage questions); openstack-operat...@lists.openstack.org Subject: Re: [openstack-dev] [glance] [nova] Glance bug with Kilo upgrade & Nova On 09/06/15 09:22 +0200, Flavio Percoco wrote:On 09/06/15 07:09 +0000, Bhandaru, Malini K wrote:Flavio, would a DB script that writes an empty string or NOP or something instead of NULL In the column do the trick? Then the problem degenerates to a new DB upgrade script.I now remembered about a change[0] - that I wrote myself - that required a bump on the API version - which we did - that allows None values to be returned in the API. This is probably what is causing this behavior. A DB migration would certainly work, I'd love to avoid it but I guess that's the best solution in this case.Actually, we can't just migrate the database as there might be custom properties that were explicitly set to None. I'll keep you postedCheers, Flavio [0] https://review.openstack.org/#/c/138184/Regards Malini -----Original Message----- From: Flavio Percoco [mailto:fla...@redhat.com] Sent: Monday, June 08, 2015 11:47 PM To: OpenStack Development Mailing List (not for usage questions) Cc: openstack-operat...@lists.openstack.org Subject: Re: [openstack-dev] [glance] [nova] Glance bug with Kilo upgrade & Nova On 08/06/15 11:46 -0400, Clayton O'Neill wrote:We tested testing Kilo upgrades in our hardware dev environments last week and the second time through ran into this bug which right now is probably a show-stopper for us. https://bugs.launchpad.net/glance/+bug/1419823 The issue here is that the v1 Glance API allows you to create images with properties that are 'NULL' in the Glance database. For example: glance image-create --name cirros_test --disk-format qcow2 --container-format bare --file cirros-0.3.4-x86_64-disk.img --is-public True --is-protected True --progress --property description= It's apparently also fairly easy to end up with a NULL description when editing images properties via Horizon. The issue is that the v2 Glance API returns these NULL properties to the client, which then validates them against the schema returned by the v2 API. This schema returns specifies that the description property *must* be a string. In the Kilo release, Nova has been changed to use the v2 API, so suddenly this matters. The net effect is that end users can pretty easily create properties with NULL values, and then won't be able to boot instances using those images. What makes this worse is that it's completely opaque to end users, since this just reports that no node was available to schedule the instance. However, Nova *only* uses the v2 api to list images if the glance.allowed_direct_url_schemes config key is set in the config file. However, this config item defaults to an empty array, meaning that by default it's *always* set. There doesn't appear to be a way to unset a value with oslo-config that has a default value, blocking off that route to work around the issue. Disabling the v2 Glance API we don't think will work, since Nova appears to assume the v2 API is available.AFAIK, Nova supports V2 image-lists since before Juno when the allowed_direct_url_schemes config option is set. Are you referring to another change? Has the default been changed in Nova? I'm asking because I was working on this migration to V2 and we decided to postpone it to L.Another work around we've looked at is to change the DB schema for image properties (yuck) to not allow NULL values. This results in Glance returning a 500 error since glance-api is attempting to insert an invalid value. This is better than instances failing in an opaque fashion, but still pretty horrible. Has anyone else run into this issue yet? Are there other work arounds that we've not thought of other than "Don't create images with NULL properties?" User education is definitely an option, but given the failure mode, it's not a great solution for us.I believe this needs to be fixed in the client rather than the API and/or the schemas. I'll take a look at this right away. Another workaround could be updateting `schema-image.json` to add the schemas that are missing and let the client download the final schema from the V2 API. Keep an eye on the bug, patches coming your way (assuming what I have in mind will work). Flavio -- @flaper87 Flavio Percoco-- @flaper87 Flavio Percoco__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev-- @flaper87 Flavio Percoco
-- @flaper87 Flavio Percoco
pgpwKgygYhSmM.pgp
Description: PGP signature
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
