Re: [openstack-dev] [sahara] [api] [sdks] [keystone] Sahara APIv2: service discovery

[Jeremy Freudberg]

I've been delayed in actually starting the grunt work on this. If
anyone else besides Monty is also able to chime in, feel free.

I'm a bit lost in trying to find examples of clients doing version
discovery + endpoint manipulation the "right way". If I could find a
good example life would be easier.

Stuff that constitutes the "right way":
- use keystoneauth for version discovery (should we be using
add_catalog_discover_hack [0] in Sahara's case?)
- actually using keystoneauth discovery features when creating the
client object (in my case, modifying [1])
- somehow putting the project id back in the URL (depending on at what
point in the process this happens, can that just be "%(project_id)s"
or must it be the actual project id, not sure)

All help appreciated.

Thanks!

[0] 
https://github.com/openstack/keystoneauth/blob/master/keystoneauth1/discover.py#L1224
[1] 
https://github.com/openstack/python-saharaclient/blob/master/saharaclient/api/client.py

On Thu, Nov 30, 2017 at 7:34 PM, Monty Taylor  wrote:
>
> On 11/30/2017 03:07 PM, Jeremy Freudberg wrote:
>>
>> Hi all,
>>
>> In the Sahara world, we are getting ready to expose our experimental
>> v2 API to real users and not just curious devs. Therefore we need to
>> start thinking about service/version discovery of this new API.
>
>
> \o/
>
>> Earlier this year, the service types authority was created, and one of
>> the things it asserted was that different service types for each API
>> version (like Cinder and Mistral did) is bad.
>>
>> So, it would entail that we should not adopt the `data-processingv2`
>> service type.
>
>
> Yes. Please don't... the service-types data has made its way into many places 
> now.
>
>> Unfortunately it's not so easy because Sahara API v1 relies on project
>> ID in the URL, and therefore is expected to be registered with the
>> project ID template in the Keystone service catalog. But API v2 does
>> not accept project ID in the URL.
>>
>> We don't want to break existing clients' ability to discover and use
>> Sahara among all clouds. So if we changed the expectation of the
>> endpoint for the current `dataprocessing` service type to never
>> contain project ID, some clients might get spooked. (Correct me if I'm
>> wrong)
>
>
> WELL - there's totally a way to do this that works, although it's gonna be 
> somewhat annoying.
>
> First and most importantly you need to update python-saharaclient to make 
> sure it can handle it an unversioned endpoint in the catalog (by doing 
> discovery) - and that if it finds an unversioned endpoint in the catalog it 
> knows to prepend project-id to the urls is sends. The easiest/best way to do 
> this it to make sure it's delegating version discovery to keystoneauth ... I 
> will be more than happy to help you get that updated.
>
> Then, for now, recommend that *new* deployments put the unversioned endpoint 
> into their catalog, but that existing deployments keep the v1 endpoint in the 
> catalog even if they upgrade sahara to a version that has v2 as well. (The 
> full description of version discovery describes how to get to a newer version 
> even if an older version is in the catalog, so people can opt-in to v2 if 
> it's there with no trouble)
>
> That gets us to a state where:
>
> - existing deployments with users using v1 are not broken
> - existing deployments that upgrade can have user's opt-in to v2 easily
> - new deployments will have both v1 and v2 - but users who want to use v1 
> will have to do so with a client that understands actually doing discovery
>
> Then let it sit that way for a while, and we can work to make sure that other 
> clients with sahara support are also up to date with version discovery.
>
> There will eventually come a point where a deployer will decide they want to 
> change their catalog from /v1/{project_id} to / ... but by then we should 
> have all the clients able to understand discovery fully.
>
>> So, we either need to break the rules and create the
>> `data-processingv2` type anyway, or we can create a new type just
>> called, for example, `bigdata` which going forward can be used to
>> discover either v1 or v2 without any interop concerns.
>
>
> I think renaming to bigdata is less terrible than data-processing2 ... but 
> let's see if we can't get things to work the other day first - there's a lot 
> of churn otherwise.
>
>> This is not an aspect of OpenStack I know a lot about, so any guidance
>> is appreciated. Once we figure out a way forward I will make sure
>> patches get proposed to the service types authority repo.
>
>
> Almost nobody does. :) But we can totally figure this one out.
>
> Monty
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [sahara] [api] [sdks] [keystone] Sahara APIv2: service discovery

[Monty Taylor]

On 12/01/2017 05:04 AM, Luigi Toscano wrote:

On Friday, 1 December 2017 01:34:36 CET Monty Taylor wrote:


First and most importantly you need to update python-saharaclient to
make sure it can handle it an unversioned endpoint in the catalog (by
doing discovery) - and that if it finds an unversioned endpoint in the
catalog it knows to prepend project-id to the urls is sends. The
easiest/best way to do this it to make sure it's delegating version
discovery to keystoneauth ... I will be more than happy to help you get
that updated.

Then, for now, recommend that *new* deployments put the unversioned
endpoint into their catalog, but that existing deployments keep the v1
endpoint in the catalog even if they upgrade sahara to a version that
has v2 as well. (The full description of version discovery describes how
to get to a newer version even if an older version is in the catalog, so
people can opt-in to v2 if it's there with no trouble)

That gets us to a state where:

- existing deployments with users using v1 are not broken
- existing deployments that upgrade can have user's opt-in to v2 easily
- new deployments will have both v1 and v2 - but users who want to use
v1 will have to do so with a client that understands actually doing
discovery


Does it work even if we would like to keep v1 as default for a while? v2, at
least in the first release, will be marked as experimental; hopefully it
should stabilize soon, but still.


Totally. In the version discovery document returned by sahara, keep v1 
listed as "CURRENT" and list v2 as "EXPERIMENTAL". Then, when you're 
ready to declare v2 as the recommended API, change v1 to "SUPPORTED" and 
v1 to "CURRENT".



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [sahara] [api] [sdks] [keystone] Sahara APIv2: service discovery

[Luigi Toscano]

On Friday, 1 December 2017 01:34:36 CET Monty Taylor wrote:

> First and most importantly you need to update python-saharaclient to
> make sure it can handle it an unversioned endpoint in the catalog (by
> doing discovery) - and that if it finds an unversioned endpoint in the
> catalog it knows to prepend project-id to the urls is sends. The
> easiest/best way to do this it to make sure it's delegating version
> discovery to keystoneauth ... I will be more than happy to help you get
> that updated.
> 
> Then, for now, recommend that *new* deployments put the unversioned
> endpoint into their catalog, but that existing deployments keep the v1
> endpoint in the catalog even if they upgrade sahara to a version that
> has v2 as well. (The full description of version discovery describes how
> to get to a newer version even if an older version is in the catalog, so
> people can opt-in to v2 if it's there with no trouble)
> 
> That gets us to a state where:
> 
> - existing deployments with users using v1 are not broken
> - existing deployments that upgrade can have user's opt-in to v2 easily
> - new deployments will have both v1 and v2 - but users who want to use
> v1 will have to do so with a client that understands actually doing
> discovery

Does it work even if we would like to keep v1 as default for a while? v2, at 
least in the first release, will be marked as experimental; hopefully it 
should stabilize soon, but still.

Ciao
-- 
Luigi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [sahara] [api] [sdks] [keystone] Sahara APIv2: service discovery

[Monty Taylor]

On 11/30/2017 03:07 PM, Jeremy Freudberg wrote:

Hi all,

In the Sahara world, we are getting ready to expose our experimental
v2 API to real users and not just curious devs. Therefore we need to
start thinking about service/version discovery of this new API.


\o/


Earlier this year, the service types authority was created, and one of
the things it asserted was that different service types for each API
version (like Cinder and Mistral did) is bad.

So, it would entail that we should not adopt the `data-processingv2`
service type.


Yes. Please don't... the service-types data has made its way into many 
places now.



Unfortunately it's not so easy because Sahara API v1 relies on project
ID in the URL, and therefore is expected to be registered with the
project ID template in the Keystone service catalog. But API v2 does
not accept project ID in the URL.

We don't want to break existing clients' ability to discover and use
Sahara among all clouds. So if we changed the expectation of the
endpoint for the current `dataprocessing` service type to never
contain project ID, some clients might get spooked. (Correct me if I'm
wrong)


WELL - there's totally a way to do this that works, although it's gonna 
be somewhat annoying.


First and most importantly you need to update python-saharaclient to 
make sure it can handle it an unversioned endpoint in the catalog (by 
doing discovery) - and that if it finds an unversioned endpoint in the 
catalog it knows to prepend project-id to the urls is sends. The 
easiest/best way to do this it to make sure it's delegating version 
discovery to keystoneauth ... I will be more than happy to help you get 
that updated.


Then, for now, recommend that *new* deployments put the unversioned 
endpoint into their catalog, but that existing deployments keep the v1 
endpoint in the catalog even if they upgrade sahara to a version that 
has v2 as well. (The full description of version discovery describes how 
to get to a newer version even if an older version is in the catalog, so 
people can opt-in to v2 if it's there with no trouble)


That gets us to a state where:

- existing deployments with users using v1 are not broken
- existing deployments that upgrade can have user's opt-in to v2 easily
- new deployments will have both v1 and v2 - but users who want to use 
v1 will have to do so with a client that understands actually doing 
discovery


Then let it sit that way for a while, and we can work to make sure that 
other clients with sahara support are also up to date with version 
discovery.


There will eventually come a point where a deployer will decide they 
want to change their catalog from /v1/{project_id} to / ... but by then 
we should have all the clients able to understand discovery fully.



So, we either need to break the rules and create the
`data-processingv2` type anyway, or we can create a new type just
called, for example, `bigdata` which going forward can be used to
discover either v1 or v2 without any interop concerns.


I think renaming to bigdata is less terrible than data-processing2 ... 
but let's see if we can't get things to work the other day first - 
there's a lot of churn otherwise.



This is not an aspect of OpenStack I know a lot about, so any guidance
is appreciated. Once we figure out a way forward I will make sure
patches get proposed to the service types authority repo.


Almost nobody does. :) But we can totally figure this one out.

Monty

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [sahara] [api] [sdks] [keystone] Sahara APIv2: service discovery

[Jeremy Freudberg]

Hi all,

In the Sahara world, we are getting ready to expose our experimental
v2 API to real users and not just curious devs. Therefore we need to
start thinking about service/version discovery of this new API.

Earlier this year, the service types authority was created, and one of
the things it asserted was that different service types for each API
version (like Cinder and Mistral did) is bad.

So, it would entail that we should not adopt the `data-processingv2`
service type.

Unfortunately it's not so easy because Sahara API v1 relies on project
ID in the URL, and therefore is expected to be registered with the
project ID template in the Keystone service catalog. But API v2 does
not accept project ID in the URL.

We don't want to break existing clients' ability to discover and use
Sahara among all clouds. So if we changed the expectation of the
endpoint for the current `dataprocessing` service type to never
contain project ID, some clients might get spooked. (Correct me if I'm
wrong)

So, we either need to break the rules and create the
`data-processingv2` type anyway, or we can create a new type just
called, for example, `bigdata` which going forward can be used to
discover either v1 or v2 without any interop concerns.

This is not an aspect of OpenStack I know a lot about, so any guidance
is appreciated. Once we figure out a way forward I will make sure
patches get proposed to the service types authority repo.

Best,
Jeremy

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev