Re: [openstack-dev] Notification Schemas ...
Thanks for the comments Jay. Haven't acted on the comments yet, but I've just pushed an update with some simple validation capability. Fairly quick and dirty stuff, just wrapping the schema validation capability of python-jsonschema. You need jsonschema in your python env to run it. Eddie From: Sandy Walsh [sandy.wa...@rackspace.com] Sent: Monday, January 19, 2015 7:53 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] Notification Schemas ... Thanks Jay, good feedback. Comments inline ... From: Jay Pipes [jaypi...@gmail.com] Sent: Sunday, January 18, 2015 10:47 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] Notification Schemas ... On 01/18/2015 04:39 PM, Sandy Walsh wrote: Eddie Sheffield has pulled together a strawman set of notification schemas for Nova and Glance. https://github.com/StackTach/notification-schemas Some important things that I see are missing, so far... please let me know what your thoughts are regarding these. 1) There needs to be some method of listing the notification codes. By code, I mean compute.instance_create.start, or possibly the CADF event codes, which I believe I recommended way back a-when the original ML thread started. The title contains the event name https://github.com/StackTach/notification-schemas/blob/master/nova/compute-instance-create-end.json#L4 but perhaps a wider CADF-like category would be good. And we should have a close eye on ensuring we're capturing the broadest CADF attributes now. That said, CADF and schemas are complimentary. Once we have the existing data mapped we should be able to determine which parts fit in the CADF envelope directly and which parts need to go in the attachment part (which will still need a separate schema definition). These attachments could lead the way to changes to the core CADF spec. The CADF team has stated they're receptive to that. The notification driver can output the desired wire protocol. 2) Each notification message payload must contain a version in it. We need some ability to evolve the notification schemas over time, and a version in the payload is a pre-requisite for that. Agreed. I suspect we'll be adding a common object to https://github.com/StackTach/notification-schemas/blob/master/nova/objects.json which will contain all that version stuff. Next round for sure. This was just to capture what we have now. All the best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Glance] Granularity of policies
I encountered an interesting situation with Glance policies. Basically we have a situation where users in certain roles are not allowed to make certain calls at all. In this specific case, we don't want users in those roles listing or viewing members. When listing members, these users receive a 403 (Forbidden) but when showing an individual member the users receive 404 (Not Found). So the problem is that there are a couple of situations here and we don't (can't?) distinguish the exact intent: 1) A user IS allowed to make the call but isn't allowed to see a particular member - in that case 404 makes sense because a 403 could imply the user actually is there, you just can't look see them directly. 2) A user IS NOT allowed to make the call at all. In this case a 403 makes more sense because the user is forbidden at the call level. At this point I'm mainly trying to spark some conversation on this. This feels a bit inconsistent if users get 403 for a whole set of calls they are barred from but 404 for others which are sub calls of the others (e.g. listing members vs. showing a specific one.) But I don't have a specific proposals at this time - first I'm trying to find out if others feel this is a problem which should be addressed. If so I'm willing to work on a blueprint and implementation. - Eddie ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Glance]supporting of v1 and v2 glance APIs in Nova
A few days ago, I met some problems when using 'createimage' feature in Nova, we found that using V1 of glanceclient has some problem with processing of metadata, the version number and even the glance URIs are both hardcoded in Nova. then, we found the bluepring[1] proposed, and the maillist[2] which talked about the topic before, mainly focused on version autodiscovery by keystone catalog and config option for nova. But we still need changes in Nova because the incompatible behavior between v1 and v2, especially when creating and uploading an image file. The review request[3] of the bp is abandoned for now. So, what I want to confirm is, how could this situation be handled? I mailed Eddie Sheffield, but got no answer, so bring it up here. [1]: https://blueprints.launchpad.net/nova/+spec/use-glance-v2-api [2]: http://markmail.org/message/uqrpufsmh4qp5pgy [4]: https://review.openstack.org/#/c/38414/ Hi Lingxian, I'm afraid I somehow didn't see the email you sent to me directly. We recently held the Glance Mini-summit and this work was discussed. Andrew Laski from the Nova team was also in attendance and provided some input from their perspective. Essentially we decided that while autodiscovery is desirable, we want to roll that functionality into a much-improved python-glanceclient which will present a version-agnostic programming api to users of the library. So the immediate plan is to go back to the approach outlined in the bp and merge prop you reference above. Then in the near future produce the new glanceclient followed by updating Nova to use the new library which will address the concerns of autodiscovery among other things. Timeline-wise, I've been a bit covered up with some other work but will be getting back to this within a week. There were some concerns about the size of the patch so rather than unabandoning the existing one I will be trying to put up multiple, smaller patches. Please let me know if you have any specific concerns or requirements so they can be addressed. Eddie Sheffield Rackspace Hosting, Inc. eddie.sheffi...@rackspace.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Glance] Support of v1 and v2 glance APIs in Nova
Mark Washenberger mark.washenber...@markwash.net said: ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev I am not a fan of all the specific talk to glance code we have in nova, moving more of that into glanceclient can only be a good thing. For the XenServer itegration, for efficiency reasons, we need glance to talk from dom0, so it has dom0 making the final HTTP call. So we would need a way of extracting that info from the glance client. But that seems better than having that code in nova. I know in Glance we've largely taken the view that the client should be as thin and lightweight as possible so users of the client can make use of it however they best see fit. There was an earlier patch that would have moved the whole image service layer into glanceclient that was rejected. So I think there is a division in philosophies here as well. Indeed, I think I was being a bit of a stinker on this issue. Mea culpa. I've had some time to think and I realized that there is a bit of complexity here that needs to be untangled. Historically, the glance client (and I think *most* openstack clients) have had versioned directories that attempt to be as faithful a representation of the given version of an API as possible. That was a history I was trying to maintain for continuity's sake in the past. However, with some more thought, this historical objective seems literally insane to me. In fact, it makes it basically impossible to publish a useful client library because such a library has no control to smooth over backwards incompatibility from one major version to the next. At this point I'm a lot more interested in Ghe's patch ( https://review.openstack.org/#/c/33327/) I'm a bit concerned that we might need to make the image client interface even more stripped down in order to focus support on the intersection of v1 and v2 of the image api. In particular, I'm not sure how well the old nova image service api will deal with invalid property values (v2 has property schemas). And there's no support AFAICT for image sharing, and image sharing should not be used in v1 for security reasons. On the other hand, maybe we don't really want to move forward based on how nova viewed the image repository in the past. There might be a better image client api waiting to be discovered by some intrepid openstacker. This could make sense as well if there is some traction for eventually deprecating the v1 api. But in any case, it does sound like we need an image client with its own proper api that can be ported from version to version. /ramble Hmmm, pretty big turnaround but one I mostly agree with. I would like to see more discussion on what this unified interface would look like rather than just pulling in what's in Nova (tho we might converge on that anyway.) I do worry about what to do about unique functionality in the various API versions. It might be that the most common functionality is exposed in the service interface, and if you need some of the more API specific functionality you can use the lower-level client interfaces. Alternatively the interface might contain everything possible; and where it can, smooth over the differences and where it can't, raise NotImplemented exceptions. Mark, can we get some discussion of this in our Glance meeting tomorrow (10/31)? Eddie ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Glance] Support of v1 and v2 glance APIs in Nova
John Garbutt j...@johngarbutt.com said: Going back to Joe's comment: Can both of these cases be covered by configuring the keystone catalog? +1 If both v1 and v2 are present, pick v2, otherwise just pick what is in the catalogue. That seems cool. Not quite sure how the multiple glance endpoints works in the keystone catalog, but should work I assume. We hard code nova right now, and so we probably want to keep that route too? Nova doesn't use the catalog from Keystone when talking to Glance. There is a config value glance_api_servers which defines a list of Glance servers that gets randomized and cycled through. I assume that's what you're referring to with we hard code nova. But currently there's nowhere in this path (internal nova to glance) where the keystone catalog is available. I think some of the confusion may be that Glanceclient at the programmatic client level doesn't talk to keystone. That happens happens higher in the CLI level which doesn't come into play here. From: Russell Bryant rbry...@redhat.com On 10/17/2013 03:12 PM, Eddie Sheffield wrote: Might I propose a compromise? 1) For the VERY short term, keep the config value and get the change otherwise reviewed and hopefully accepted. 2) Immediately file two blueprints: - python-glanceclient - expose a way to discover available versions - nova - depends on the glanceclient bp and allowing autodiscovery of glance version and making the config value optional (tho not deprecated / removed) Supporting both seems reasonable. At least then *most* people don't need to worry about it and it just works, but the override is there if necessary, since multiple people seem to be expressing a desire to have it available. +1 Can we just do this all at once? Adding this to glanceclient doesn't seem like a huge task. I worry about us never getting the full solution, but it seems to have got complicated. The glanceclient side is done, as far as allowing access to the list of available API versions on a given server. It's getting Nova to use this info that's a bit sticky. On 28 October 2013 15:13, Eddie Sheffield eddie.sheffi...@rackspace.com wrote: So...I've been working on this some more and hit a bit of a snag. The Glanceclient change was easy, but I see now that doing this in nova will require a pretty huge change in the way things work. Currently, the API version is grabbed from the config value, the appropriate driver is instantiated, and calls go through that. The problem comes in that the actually glance server isn't communicated with until very late in the process. Nothing sees the servers at the level where the driver is determined. Also there isn't a single glance server but a list of them, and in the even of certain communication failures the list is cycled through until success or a number of retries has passed. So to change this to auto configuring will require turning this upside down, cycling through the servers at a higher level, choosing the appropriate driver for that server, and handling retries at that same level. Doable, but a much larger task than I first was thinking. Also, I don't really want the added overhead of getting the api versions before every call, so I'm thinking that going through the list of servers at startup and discovering the versions then and caching that somehow would be helpful as well. Thoughts? I do worry about that overhead. But with Joe's comment, does it not just boil down to caching the keystone catalog in the context? I am not a fan of all the specific talk to glance code we have in nova, moving more of that into glanceclient can only be a good thing. For the XenServer itegration, for efficiency reasons, we need glance to talk from dom0, so it has dom0 making the final HTTP call. So we would need a way of extracting that info from the glance client. But that seems better than having that code in nova. I know in Glance we've largely taken the view that the client should be as thin and lightweight as possible so users of the client can make use of it however they best see fit. There was an earlier patch that would have moved the whole image service layer into glanceclient that was rejected. So I think there is a division in philosophies here as well. Eddie ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Glance] Support of v1 and v2 glance APIs in Nova
I don't oppose having the version autodiscovered. But I do feel the option should be available to override if desired. I've had many problems with over the years with autoconfiguring systems not doing what I need to be willing to depend on them 100% without a manual override available if at all possible. I'm thinking particularly for testing or upgrade evaluation scenarios. For example, you want to turn on v2 in glance and evaluate it for a bit before committing to using it for all of your nova nodes. The current patch also had a couple of comments from Dan Prince and Chris Behrens early on when this was brought up there supporting the use of a config value. From the implementation side of things, to do this properly would require some changes to the glanceclient to allow querying of available versions. Lacking that ability currently is one reason this is currently a config value. Once the client supports this, the nova side change would likely be quite small. Might I propose a compromise? 1) For the VERY short term, keep the config value and get the change otherwise reviewed and hopefully accepted. 2) Immediately file two blueprints: - python-glanceclient - expose a way to discover available versions - nova - depends on the glanceclient bp and allowing autodiscovery of glance version and making the config value optional (tho not deprecated / removed) Thanks, Eddie Sheffield -Original Message- From: Russell Bryant rbry...@redhat.com Sent: Thursday, October 17, 2013 2:00pm To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Subject: [openstack-dev] [Nova][Glance] Support of v1 and v2 glance APIs in Nova The following blueprint has been proposed for Nova: https://blueprints.launchpad.net/nova/+spec/use-glance-v2-api It proposes a new config option for Nova to allow choosing whether Nova should use v1 or v2 of the Glance API. I'd like to know if we can do this without a config option. Can Nova just discover which versions and pick one (v2) ? And if you don't like the way Nova picks one, a deployer can just only expose one of the APIs on the API endpoint that Nova uses. Thoughts? -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [python-glanceclient] Review request
(Hope this isn't a dup - sent it yesterday but never saw it come through tho another I sent around the same time did. My apologies if it is.) Hi, Could I get some reviews on this patch? It's adding CLI support for using Glance V2 API from the client. https://review.openstack.org/#/c/42741/ Thanks, Eddie ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [glance] Review request
(Hope this isn't a dup - sent it earlier but never saw it come through. My apologies if it is.) Hi, Could I get some reviews on this patch? It's adding CLI support for using Glance V2 API from the client. https://review.openstack.org/#/c/42741/ Thanks, Eddie ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [python-glanceclient] Review request
Hi, Could I get some reviews on this patch? It's adding CLI support for using Glance V2 API from the client. https://review.openstack.org/#/c/42741/ Thanks, Eddie ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Review request
Hi all! I have a patch up for allowing Nova to talk to Glance via either the V1 or V2 API. This is currently blocked by the feature freeze, but if some folks could look at it so I can have it ready to roll when the freeze ends I'd really appreciate it! https://review.openstack.org/#/c/46507/ Thanks, Eddie ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
