Re: [openstack-dev] [all] Capability Discovery API
Here's a possibly relevant use case for this discussion: 1) Running Icehouse OpenStack 2) Keystone reports v3.0 auth capabilities 3) If you actually use the v3.0 auth, then any nova call that gets passed through to cinder fails due to the code in Icehouse being unable to parse the 3.0 service catalog format Due to the limited ability to interrogate OpenStack and determine what is running, we have to auth with v3, and then make a volume related nova call and see if it fails. Afterward we can go down code paths to work around the OS bugs in the presumed version. If a more robust API for determining the running components and their capabilities were available, this would be an easier situation to deal with. The main point of this is that a capabilities API requires an absolute flawless implementation to be sufficient. It fails if a capability is reported as available, but the implementation in that particular release has a bug. The version of implementation code also needs to be exposed through the API for consumers to be able to know when issues are present and work around them. -Andrew On Wed, Mar 18, 2015 at 1:38 PM, Ian Wells wrote: > On 18 March 2015 at 03:33, Duncan Thomas wrote: > >> On 17 March 2015 at 22:02, Davis, Amos (PaaS-Core) < >> amos.steven.da...@hp.com> wrote: >> >>> Ceph/Cinder: >>> LVM or other? >>> SCSI-backed? >>> Any others? >>> >> >> I'm wondering why any of the above matter to an application. >> > > The Neutron requirements list is the same. Everything you've listed > details implementation details with the exception of shared networks (which > are a core feature, and so it's actually rather unclear what you had in > mind there). > > Implementation details should be hidden from cloud users - they don't care > if I'm using ovs/vlan, and they don't care that I change my cloud one day > to run ovs/vxlan, they only care that I deliver a cloud that will run their > application - and since I care that I don't break applications when I make > under the cover changes I will be thinking carefully about that too. I > think you could develop a feature list, mind, just that you've not managed > it here. > > For instance: why is an LVM disk different from one on a Netapp when > you're a cloud application and you always attach a volume via a VM? Well, > it basically isn't, unless there are features (like for instance a minimum > TPS guarantee) that are different between the drivers. Cinder's even > stranger here, since you can have multiple backend drivers simultaneously > and a feature may not be present in all of them. > > Also, in Neutron, the current MTU and VLAN work is intended to expose some > of those features to the app more than they were previously (e.g. 'can I > use a large MTU on this network?'), but there are complexities in exposing > this in advance of running the application. The MTU size is not easy to > discover in advance (it varies depending on what sort of network you're > making), and what MTU you get for a specific network is very dependent on > the network controller (network controllers can choose to not expose it at > all, expose it with upper bounds in place, or expose it and try so hard to > implement what the user requests that it's not immediately obvious whether > a request will succeed or fail, for instance). You could say 'you can ask > for large MTU networks' - that is a straightforward feature - but some apps > will fail to run if they ask and get declined. > > This is not to say there isn't useful work that could be done here, just > that there may be some limitations on what is possible. > -- > Ian. > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Andrew Mann DivvyCloud Inc. www.divvycloud.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Capability Discovery API
On 18 March 2015 at 03:33, Duncan Thomas wrote: > On 17 March 2015 at 22:02, Davis, Amos (PaaS-Core) < > amos.steven.da...@hp.com> wrote: > >> Ceph/Cinder: >> LVM or other? >> SCSI-backed? >> Any others? >> > > I'm wondering why any of the above matter to an application. > The Neutron requirements list is the same. Everything you've listed details implementation details with the exception of shared networks (which are a core feature, and so it's actually rather unclear what you had in mind there). Implementation details should be hidden from cloud users - they don't care if I'm using ovs/vlan, and they don't care that I change my cloud one day to run ovs/vxlan, they only care that I deliver a cloud that will run their application - and since I care that I don't break applications when I make under the cover changes I will be thinking carefully about that too. I think you could develop a feature list, mind, just that you've not managed it here. For instance: why is an LVM disk different from one on a Netapp when you're a cloud application and you always attach a volume via a VM? Well, it basically isn't, unless there are features (like for instance a minimum TPS guarantee) that are different between the drivers. Cinder's even stranger here, since you can have multiple backend drivers simultaneously and a feature may not be present in all of them. Also, in Neutron, the current MTU and VLAN work is intended to expose some of those features to the app more than they were previously (e.g. 'can I use a large MTU on this network?'), but there are complexities in exposing this in advance of running the application. The MTU size is not easy to discover in advance (it varies depending on what sort of network you're making), and what MTU you get for a specific network is very dependent on the network controller (network controllers can choose to not expose it at all, expose it with upper bounds in place, or expose it and try so hard to implement what the user requests that it's not immediately obvious whether a request will succeed or fail, for instance). You could say 'you can ask for large MTU networks' - that is a straightforward feature - but some apps will fail to run if they ask and get declined. This is not to say there isn't useful work that could be done here, just that there may be some limitations on what is possible. -- Ian. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Capability Discovery API
What you're proposing quickly becomes an authorization question. "What capabilities can this service provide?" is a far less useful question than "what capabilities is the user authorized to consume?" More generally, why would you advertise any capability that the user is going to receive a 4xx/5xx for using? It's a longstanding problem that the community has discussed many times in the past. On Tue, Mar 17, 2015 at 3:02 PM, Davis, Amos (PaaS-Core) < amos.steven.da...@hp.com> wrote: > All, > The Application EcoSystem Working Group realized during the mid-cycle > meetup in Philadelphia that there is no way to get the capabilities of an > Openstack cloud so that applications can measure their compatibility > against that cloud. In other words, if we create an Openstack App > Marketplace and have developers make apps to be in that marketplace, then > we'll have no way for apps to verify that they can run on that cloud. We'd > like to ask that there be a standard set of API calls created that allow a > cloud to list its capabilities. The cloud "features" or capabilities list > should return True/False API responses and could include but is not limited > to the below examples. Also, https://review.openstack.org/#/c/162655/ > may be a good starting point for this request. > > > Glance: > URL/upload > types (raw, qcow, etc) > > Nova: > Suspend/Resume VM > Resize > Flavor sizes supported > Images Available > Quota Limits > VNC support > > Neutron: > Types of Networking (neutron, neutron + ml2, nova-network aka linux > bridge, other) > Types of SDN in use? > Shared tenant networks > Anything else? > > > Ceph/Cinder: > LVM or other? > SCSI-backed? > Any others? > > Swift: > ? > > Best Regards, > Amos Davis > amos.da...@hp.com > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Capability Discovery API
On 17 March 2015 at 22:02, Davis, Amos (PaaS-Core) wrote: > Ceph/Cinder: > LVM or other? > SCSI-backed? > Any others? > I'm wondering why any of the above matter to an application. The entire point of cinder is to abstract those details from the application. I'd be very strongly resistant to add any API to cinder that exposed any of the above. Is there some significant difference the above makes, from an application POV? If so, please do let me know, since it is probably a bug There *are* details about cinder that are sensible to expose however: Are consistency groups supported? Is replication supported? Is the backup service enabled? Do snapshots of an attached volume work? Are there restrictions to backing up snapshots, or snapshotted volumes, or volume from snapshots? ...and probably others. I don't think there's a good answer to how to answer these questions yet, and I agree we need to get that on the road map. Some of the above questions should ideally only have one question, but there are limitations on various drivers that we've not yet fixed. -- Duncan Thomas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Capability Discovery API
What does ³Flavor sizes² include? Memory, CPU count? Is there a wide-enough care for other measures of performance or compatibility like: - virtualization type: none (hardware/metal), xen, kvm, hyperv - cpu speed, cache or some form of performance index - volume types: SATA, SSD, iSCSI, and a performance index On 3/17/15, 9:22 PM, "John Dickinson" wrote: > >> On Mar 17, 2015, at 1:02 PM, Davis, Amos (PaaS-Core) >> wrote: >> >> All, >> The Application EcoSystem Working Group realized during the mid-cycle >>meetup in Philadelphia that there is no way to get the capabilities of >>an Openstack cloud so that applications can measure their compatibility >>against that cloud. In other words, if we create an Openstack App >>Marketplace and have developers make apps to be in that marketplace, >>then we'll have no way for apps to verify that they can run on that >>cloud. We'd like to ask that there be a standard set of API calls >>created that allow a cloud to list its capabilities. The cloud >>"features" or capabilities list should return True/False API responses >>and could include but is not limited to the below examples. Also, >>https://review.openstack.org/#/c/162655/ may be a good starting point >>for this request. >> >> >> Glance: >> URL/upload >> types (raw, qcow, etc) >> >> Nova: >> Suspend/Resume VM >> Resize >> Flavor sizes supported >> Images Available >> Quota Limits >> VNC support >> >> Neutron: >> Types of Networking (neutron, neutron + ml2, nova-network aka linux >>bridge, other) >> Types of SDN in use? >> Shared tenant networks >> Anything else? >> >> >> Ceph/Cinder: >> LVM or other? >> SCSI-backed? >> Any others? >> >> Swift: >> ? > >Swift's capabilities are discoverable via an "/info" endpoint. The docs >are at: > >http://docs.openstack.org/developer/swift/api/discoverability.html > >Example output from my dev environment and from Rackspace Cloud Files and >from a SwiftStack lab cluster: > >https://gist.github.com/notmyname/438392d57c2f3d3ee327 > > >Clients use these to ensure a unified experience across clusters and that >features are supported before trying to use them. > >> >> Best Regards, >> Amos Davis >> amos.da...@hp.com >> >> >>_ >>_ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Capability Discovery API
> On Mar 17, 2015, at 1:02 PM, Davis, Amos (PaaS-Core) > wrote: > > All, > The Application EcoSystem Working Group realized during the mid-cycle meetup > in Philadelphia that there is no way to get the capabilities of an Openstack > cloud so that applications can measure their compatibility against that > cloud. In other words, if we create an Openstack App Marketplace and have > developers make apps to be in that marketplace, then we'll have no way for > apps to verify that they can run on that cloud. We'd like to ask that there > be a standard set of API calls created that allow a cloud to list its > capabilities. The cloud "features" or capabilities list should return > True/False API responses and could include but is not limited to the below > examples. Also, https://review.openstack.org/#/c/162655/ may be a good > starting point for this request. > > > Glance: > URL/upload > types (raw, qcow, etc) > > Nova: > Suspend/Resume VM > Resize > Flavor sizes supported > Images Available > Quota Limits > VNC support > > Neutron: > Types of Networking (neutron, neutron + ml2, nova-network aka linux bridge, > other) > Types of SDN in use? > Shared tenant networks > Anything else? > > > Ceph/Cinder: > LVM or other? > SCSI-backed? > Any others? > > Swift: > ? Swift's capabilities are discoverable via an "/info" endpoint. The docs are at: http://docs.openstack.org/developer/swift/api/discoverability.html Example output from my dev environment and from Rackspace Cloud Files and from a SwiftStack lab cluster: https://gist.github.com/notmyname/438392d57c2f3d3ee327 Clients use these to ensure a unified experience across clusters and that features are supported before trying to use them. > > Best Regards, > Amos Davis > amos.da...@hp.com > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev signature.asc Description: Message signed with OpenPGP using GPGMail __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all] Capability Discovery API
All, The Application EcoSystem Working Group realized during the mid-cycle meetup in Philadelphia that there is no way to get the capabilities of an Openstack cloud so that applications can measure their compatibility against that cloud. In other words, if we create an Openstack App Marketplace and have developers make apps to be in that marketplace, then we'll have no way for apps to verify that they can run on that cloud. We'd like to ask that there be a standard set of API calls created that allow a cloud to list its capabilities. The cloud "features" or capabilities list should return True/False API responses and could include but is not limited to the below examples. Also, https://review.openstack.org/#/c/162655/ may be a good starting point for this request. Glance: URL/upload types (raw, qcow, etc) Nova: Suspend/Resume VM Resize Flavor sizes supported Images Available Quota Limits VNC support Neutron: Types of Networking (neutron, neutron + ml2, nova-network aka linux bridge, other) Types of SDN in use? Shared tenant networks Anything else? Ceph/Cinder: LVM or other? SCSI-backed? Any others? Swift: ? Best Regards, Amos Davis amos.da...@hp.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
