[openstack-dev] [sahara] [api] [sdks] [keystone] Sahara APIv2: service discovery
Hi all, In the Sahara world, we are getting ready to expose our experimental v2 API to real users and not just curious devs. Therefore we need to start thinking about service/version discovery of this new API. Earlier this year, the service types authority was created, and one of the things it asserted was that different service types for each API version (like Cinder and Mistral did) is bad. So, it would entail that we should not adopt the `data-processingv2` service type. Unfortunately it's not so easy because Sahara API v1 relies on project ID in the URL, and therefore is expected to be registered with the project ID template in the Keystone service catalog. But API v2 does not accept project ID in the URL. We don't want to break existing clients' ability to discover and use Sahara among all clouds. So if we changed the expectation of the endpoint for the current `dataprocessing` service type to never contain project ID, some clients might get spooked. (Correct me if I'm wrong) So, we either need to break the rules and create the `data-processingv2` type anyway, or we can create a new type just called, for example, `bigdata` which going forward can be used to discover either v1 or v2 without any interop concerns. This is not an aspect of OpenStack I know a lot about, so any guidance is appreciated. Once we figure out a way forward I will make sure patches get proposed to the service types authority repo. Best, Jeremy __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] [api] [sdks] [keystone] Sahara APIv2: service discovery
On 11/30/2017 03:07 PM, Jeremy Freudberg wrote: Hi all, In the Sahara world, we are getting ready to expose our experimental v2 API to real users and not just curious devs. Therefore we need to start thinking about service/version discovery of this new API. \o/ Earlier this year, the service types authority was created, and one of the things it asserted was that different service types for each API version (like Cinder and Mistral did) is bad. So, it would entail that we should not adopt the `data-processingv2` service type. Yes. Please don't... the service-types data has made its way into many places now. Unfortunately it's not so easy because Sahara API v1 relies on project ID in the URL, and therefore is expected to be registered with the project ID template in the Keystone service catalog. But API v2 does not accept project ID in the URL. We don't want to break existing clients' ability to discover and use Sahara among all clouds. So if we changed the expectation of the endpoint for the current `dataprocessing` service type to never contain project ID, some clients might get spooked. (Correct me if I'm wrong) WELL - there's totally a way to do this that works, although it's gonna be somewhat annoying. First and most importantly you need to update python-saharaclient to make sure it can handle it an unversioned endpoint in the catalog (by doing discovery) - and that if it finds an unversioned endpoint in the catalog it knows to prepend project-id to the urls is sends. The easiest/best way to do this it to make sure it's delegating version discovery to keystoneauth ... I will be more than happy to help you get that updated. Then, for now, recommend that *new* deployments put the unversioned endpoint into their catalog, but that existing deployments keep the v1 endpoint in the catalog even if they upgrade sahara to a version that has v2 as well. (The full description of version discovery describes how to get to a newer version even if an older version is in the catalog, so people can opt-in to v2 if it's there with no trouble) That gets us to a state where: - existing deployments with users using v1 are not broken - existing deployments that upgrade can have user's opt-in to v2 easily - new deployments will have both v1 and v2 - but users who want to use v1 will have to do so with a client that understands actually doing discovery Then let it sit that way for a while, and we can work to make sure that other clients with sahara support are also up to date with version discovery. There will eventually come a point where a deployer will decide they want to change their catalog from /v1/{project_id} to / ... but by then we should have all the clients able to understand discovery fully. So, we either need to break the rules and create the `data-processingv2` type anyway, or we can create a new type just called, for example, `bigdata` which going forward can be used to discover either v1 or v2 without any interop concerns. I think renaming to bigdata is less terrible than data-processing2 ... but let's see if we can't get things to work the other day first - there's a lot of churn otherwise. This is not an aspect of OpenStack I know a lot about, so any guidance is appreciated. Once we figure out a way forward I will make sure patches get proposed to the service types authority repo. Almost nobody does. :) But we can totally figure this one out. Monty __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] [api] [sdks] [keystone] Sahara APIv2: service discovery
On Friday, 1 December 2017 01:34:36 CET Monty Taylor wrote: > First and most importantly you need to update python-saharaclient to > make sure it can handle it an unversioned endpoint in the catalog (by > doing discovery) - and that if it finds an unversioned endpoint in the > catalog it knows to prepend project-id to the urls is sends. The > easiest/best way to do this it to make sure it's delegating version > discovery to keystoneauth ... I will be more than happy to help you get > that updated. > > Then, for now, recommend that *new* deployments put the unversioned > endpoint into their catalog, but that existing deployments keep the v1 > endpoint in the catalog even if they upgrade sahara to a version that > has v2 as well. (The full description of version discovery describes how > to get to a newer version even if an older version is in the catalog, so > people can opt-in to v2 if it's there with no trouble) > > That gets us to a state where: > > - existing deployments with users using v1 are not broken > - existing deployments that upgrade can have user's opt-in to v2 easily > - new deployments will have both v1 and v2 - but users who want to use > v1 will have to do so with a client that understands actually doing > discovery Does it work even if we would like to keep v1 as default for a while? v2, at least in the first release, will be marked as experimental; hopefully it should stabilize soon, but still. Ciao -- Luigi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] [api] [sdks] [keystone] Sahara APIv2: service discovery
On 12/01/2017 05:04 AM, Luigi Toscano wrote: On Friday, 1 December 2017 01:34:36 CET Monty Taylor wrote: First and most importantly you need to update python-saharaclient to make sure it can handle it an unversioned endpoint in the catalog (by doing discovery) - and that if it finds an unversioned endpoint in the catalog it knows to prepend project-id to the urls is sends. The easiest/best way to do this it to make sure it's delegating version discovery to keystoneauth ... I will be more than happy to help you get that updated. Then, for now, recommend that *new* deployments put the unversioned endpoint into their catalog, but that existing deployments keep the v1 endpoint in the catalog even if they upgrade sahara to a version that has v2 as well. (The full description of version discovery describes how to get to a newer version even if an older version is in the catalog, so people can opt-in to v2 if it's there with no trouble) That gets us to a state where: - existing deployments with users using v1 are not broken - existing deployments that upgrade can have user's opt-in to v2 easily - new deployments will have both v1 and v2 - but users who want to use v1 will have to do so with a client that understands actually doing discovery Does it work even if we would like to keep v1 as default for a while? v2, at least in the first release, will be marked as experimental; hopefully it should stabilize soon, but still. Totally. In the version discovery document returned by sahara, keep v1 listed as "CURRENT" and list v2 as "EXPERIMENTAL". Then, when you're ready to declare v2 as the recommended API, change v1 to "SUPPORTED" and v1 to "CURRENT". __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] [api] [sdks] [keystone] Sahara APIv2: service discovery
I've been delayed in actually starting the grunt work on this. If anyone else besides Monty is also able to chime in, feel free. I'm a bit lost in trying to find examples of clients doing version discovery + endpoint manipulation the "right way". If I could find a good example life would be easier. Stuff that constitutes the "right way": - use keystoneauth for version discovery (should we be using add_catalog_discover_hack [0] in Sahara's case?) - actually using keystoneauth discovery features when creating the client object (in my case, modifying [1]) - somehow putting the project id back in the URL (depending on at what point in the process this happens, can that just be "%(project_id)s" or must it be the actual project id, not sure) All help appreciated. Thanks! [0] https://github.com/openstack/keystoneauth/blob/master/keystoneauth1/discover.py#L1224 [1] https://github.com/openstack/python-saharaclient/blob/master/saharaclient/api/client.py On Thu, Nov 30, 2017 at 7:34 PM, Monty Taylor wrote: > > On 11/30/2017 03:07 PM, Jeremy Freudberg wrote: >> >> Hi all, >> >> In the Sahara world, we are getting ready to expose our experimental >> v2 API to real users and not just curious devs. Therefore we need to >> start thinking about service/version discovery of this new API. > > > \o/ > >> Earlier this year, the service types authority was created, and one of >> the things it asserted was that different service types for each API >> version (like Cinder and Mistral did) is bad. >> >> So, it would entail that we should not adopt the `data-processingv2` >> service type. > > > Yes. Please don't... the service-types data has made its way into many places > now. > >> Unfortunately it's not so easy because Sahara API v1 relies on project >> ID in the URL, and therefore is expected to be registered with the >> project ID template in the Keystone service catalog. But API v2 does >> not accept project ID in the URL. >> >> We don't want to break existing clients' ability to discover and use >> Sahara among all clouds. So if we changed the expectation of the >> endpoint for the current `dataprocessing` service type to never >> contain project ID, some clients might get spooked. (Correct me if I'm >> wrong) > > > WELL - there's totally a way to do this that works, although it's gonna be > somewhat annoying. > > First and most importantly you need to update python-saharaclient to make > sure it can handle it an unversioned endpoint in the catalog (by doing > discovery) - and that if it finds an unversioned endpoint in the catalog it > knows to prepend project-id to the urls is sends. The easiest/best way to do > this it to make sure it's delegating version discovery to keystoneauth ... I > will be more than happy to help you get that updated. > > Then, for now, recommend that *new* deployments put the unversioned endpoint > into their catalog, but that existing deployments keep the v1 endpoint in the > catalog even if they upgrade sahara to a version that has v2 as well. (The > full description of version discovery describes how to get to a newer version > even if an older version is in the catalog, so people can opt-in to v2 if > it's there with no trouble) > > That gets us to a state where: > > - existing deployments with users using v1 are not broken > - existing deployments that upgrade can have user's opt-in to v2 easily > - new deployments will have both v1 and v2 - but users who want to use v1 > will have to do so with a client that understands actually doing discovery > > Then let it sit that way for a while, and we can work to make sure that other > clients with sahara support are also up to date with version discovery. > > There will eventually come a point where a deployer will decide they want to > change their catalog from /v1/{project_id} to / ... but by then we should > have all the clients able to understand discovery fully. > >> So, we either need to break the rules and create the >> `data-processingv2` type anyway, or we can create a new type just >> called, for example, `bigdata` which going forward can be used to >> discover either v1 or v2 without any interop concerns. > > > I think renaming to bigdata is less terrible than data-processing2 ... but > let's see if we can't get things to work the other day first - there's a lot > of churn otherwise. > >> This is not an aspect of OpenStack I know a lot about, so any guidance >> is appreciated. Once we figure out a way forward I will make sure >> patches get proposed to the service types authority repo. > > > Almost nobody does. :) But we can totally figure this one out. > > Monty > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev