Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-07 Thread Dolph Mathews 
On Wed, Aug 7, 2013 at 7:58 AM, Sam Harwell wrote:

> Please excuse me for being vague with many parts of this reply. Since I'm
> still learning the terminology used throughout this project, I chose to be
> non-specific rather than risk using the wrong name and distract from the
> points I'm trying to make.
>

Fair enough! I'll attempt to "translate" a bit below as there's already
some mixing of terminology in this thread.


>
> From a client perspective, the most important issue in writing a reliable
> application that is truly portable across implementations is ensuring that
> the API defines a way to determine whether or not a provider supports a
> particular optional feature. The precise manner in which that functionality
> is exposed does not matter so much. My only concern with consolidating
> feature discoverability into a single "endpoints" function, where users are
> expected to include standardized endpoints as well as non-standard
> endpoints ("extensions"), is the possibility of name collisions.


This seems to presume that an API extension must always present a discrete
endpoint, which may not be the case. A simple API extension may only add an
attribute to an existing API response, for example.

Our existing `GET /v#/extensions` satisfies the use case for your
"endpoints function," however. To avoid any possible confusion, this API
resource shouldn't be confused with Identity API's `GET /v3/endpoints`
which is specifically an API for managing raw service endpoints (it's worth
stressing that /v3/endpoints has no knowledge of the API extensions
presented by those services, nor should it).


> In this case, it helps to reserve certain names for use with standardized
> features (e.g. names starting with OS- could be reserved for optional
> behavior defined in the OpenStack specifications, and names starting with
> {Vendor}- could be reserved for optional behavior defined elsewhere).
>

That's exactly the convention that we follow, which I think alleviates your
concern for name collisions above.


>
> On the subject of "incrementing" an API version - this certainly makes
> sense for APIs that are linear. In practice, however, multiple
> implementations of similar features often produce aliased version numbers
> and/or overlapping version ranges, which makes incrementing the version
> number useless. This can be resolved by only using (and incrementing) API
> version numbers for the official, root-level specification. For a named
> extension, the "owner" of the extension acts as the root-level
> specification for the extension and should be the only one incrementing the
> version number. In cases where an API or extension has been altered from
> its original form, the alteration can be presented in a modular form, where
> the implementation supports the original versioned API under its originally
> published name and version, and offers the altered features as an extension
> with a new name. This allows the alterations to the core functionality to
> be linearly versioned independently from the core functionality itself.
>
> Thank you,
> Sam Harwell
>
> -Original Message-
> From: Jay Pipes [mailto:jaypi...@gmail.com]
> Sent: Tuesday, August 06, 2013 8:46 AM
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [Keystone] V3 Extensions Discoverability
>
> On 08/06/2013 01:19 AM, Jamie Lennox wrote:
> > Hi all,
> >
> > Partially in response to the trusts API review in keystoneclient
> > (https://review.openstack.org/#/c/39899/ ) and my work on keystone API
> > version discoverability (spell-check disagrees but I'm going to assume
> > that's a word - https://review.openstack.org/#/c/38414/ ) I was
> > thinking about how we should be able to know what/if an extension is
> > available. I even made a basic blueprint for how i think it should work:
> >
> https://blueprints.launchpad.net/python-keystoneclient/+spec/keystoneclient-extensionsand
>  then realized that GET /extensions is only a V2 API.
> >
> > Is this intentional? I was starting to make a review to add it to
> > identity-api but is there the intention that extensions should show up
> > within the endpoint APIs? There is no reason it couldn't work that way
> > and DELETE would just fail.
>
> I would hope that extensions would *not* show up in the endpoints API.
>
> Frankly, I'm not a fan of API extensions at all. I think they are silly
> and just promote an inconsistent and fractured user experience. I would
> highly prefer to just have a single API, versioned, with documentation
> online and in a versions/ resource that indicates what was changed, added

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-07 Thread Sam Harwell 
Please excuse me for being vague with many parts of this reply. Since I'm still 
learning the terminology used throughout this project, I chose to be 
non-specific rather than risk using the wrong name and distract from the points 
I'm trying to make.

>From a client perspective, the most important issue in writing a reliable 
>application that is truly portable across implementations is ensuring that the 
>API defines a way to determine whether or not a provider supports a particular 
>optional feature. The precise manner in which that functionality is exposed 
>does not matter so much. My only concern with consolidating feature 
>discoverability into a single "endpoints" function, where users are expected 
>to include standardized endpoints as well as non-standard endpoints 
>("extensions"), is the possibility of name collisions. In this case, it helps 
>to reserve certain names for use with standardized features (e.g. names 
>starting with OS- could be reserved for optional behavior defined in the 
>OpenStack specifications, and names starting with {Vendor}- could be reserved 
>for optional behavior defined elsewhere).

On the subject of "incrementing" an API version - this certainly makes sense 
for APIs that are linear. In practice, however, multiple implementations of 
similar features often produce aliased version numbers and/or overlapping 
version ranges, which makes incrementing the version number useless. This can 
be resolved by only using (and incrementing) API version numbers for the 
official, root-level specification. For a named extension, the "owner" of the 
extension acts as the root-level specification for the extension and should be 
the only one incrementing the version number. In cases where an API or 
extension has been altered from its original form, the alteration can be 
presented in a modular form, where the implementation supports the original 
versioned API under its originally published name and version, and offers the 
altered features as an extension with a new name. This allows the alterations 
to the core functionality to be linearly versioned independently from the core 
function
 ality itself.

Thank you,
Sam Harwell

-Original Message-
From: Jay Pipes [mailto:jaypi...@gmail.com] 
Sent: Tuesday, August 06, 2013 8:46 AM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

On 08/06/2013 01:19 AM, Jamie Lennox wrote:
> Hi all,
>
> Partially in response to the trusts API review in keystoneclient 
> (https://review.openstack.org/#/c/39899/ ) and my work on keystone API 
> version discoverability (spell-check disagrees but I'm going to assume 
> that's a word - https://review.openstack.org/#/c/38414/ ) I was 
> thinking about how we should be able to know what/if an extension is 
> available. I even made a basic blueprint for how i think it should work:
> https://blueprints.launchpad.net/python-keystoneclient/+spec/keystoneclient-extensions
>  and then realized that GET /extensions is only a V2 API.
>
> Is this intentional? I was starting to make a review to add it to 
> identity-api but is there the intention that extensions should show up 
> within the endpoint APIs? There is no reason it couldn't work that way 
> and DELETE would just fail.

I would hope that extensions would *not* show up in the endpoints API.

Frankly, I'm not a fan of API extensions at all. I think they are silly and 
just promote an inconsistent and fractured user experience. I would highly 
prefer to just have a single API, versioned, with documentation online and in a 
versions/ resource that indicates what was changed, added, and deleted in each 
version.

If some vendor wants to provide some special API resource that naturally 
belongs in a related API -- for instance, trusts in the OpenStack Identity API 
-- then the new resource should simply be added to the one and only Identity 
API, the version of the API incremented, and on we go.

API extensions are more hassle than anything else. Let us promote standards, 
not endless extensibility at the expense of usability.

Best,
-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread Jamie Lennox 
On Tue, 2013-08-06 at 11:17 -0400, Adam Young wrote:
> On 08/06/2013 10:54 AM, Dolph Mathews wrote:
> 
> > 
> > On Tue, Aug 6, 2013 at 9:28 AM, Jorge Williams
> >  wrote:
> > 
> > On Aug 6, 2013, at 8:36 AM, Adam Young wrote:
> > 
> > > On 08/06/2013 01:19 AM, Jamie Lennox wrote:
> > >> Hi all,
> > >>
> > >> Partially in response to the trusts API review in
> > keystoneclient
> > >> (https://review.openstack.org/#/c/39899/ ) and my work on
> > keystone API
> > >> version discoverability (spell-check disagrees but I'm
> > going to assume
> > >> that's a word - https://review.openstack.org/#/c/38414/ )
> > I was thinking
> > >> about how we should be able to know what/if an extension
> > is available. I
> > >> even made a basic blueprint for how i think it should
> > work:
> > >>
> > 
> > https://blueprints.launchpad.net/python-keystoneclient/+spec/keystoneclient-extensions
> >  and then realized that GET /extensions is only a V2 API.
> > >
> > > I'm not certain that the extensions should really be in
> > the v2 or v3.  It always seemed to me that Extensions should
> > be parallel to, and separate from, the core API.
> > 
> > 
> > 
> > I agree. Extensions should not be in core, but the mechanism
> > by which extensions are discovered should be part of the
> > core...right?
> > 
> > 
> > Agree. The fact that you call GET /v2.0/extensions or
> > GET /v3/extensions instead of GET /extensions just means that we can
> > iterate on the "extensions" response itself, not necessarily that
> > the extension *only* applies to particular version API being queried
> > (that's a different issue).
> 
> Agreed.  That makes sense.
> 
> 
> So the APIs should be:
> 
> v2.0/extensions
> or
> v3/extensions
> 
> but those should return links to:
> 
> extensions/some_extension

This was my thoughts as well, there is no reason for the extension to be
versioned behind our keystone API because we don't expect the extension
api to change with the core api. Extension discoverability should be
behind our API because we reserve the right to change how extensions are
discovered. 

I think it also somewhat answers my question that we should be providing
a /v3/extensions rather than putting these into /v3/endpoints.

> >  
> > 
> > -jOrGe W.
> > 
> > 
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > 
> > 
> > 
> > 
> > 
> > -- 
> > 
> > 
> > -Dolph 
> > 
> > 
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread David Chadwick 



On 06/08/2013 20:40, Clint Byrum wrote:

Agreed Jay. The successful extensible protocols like IMAP and SMTP are
merely allowing new arguments to existing fundamental functions.


But the key thing with these protocols is that they have a defined and 
standardised way of adding new extensions - any extension - to the 
protocol, which is followed by all the new extensions when they are 
defined. This is what needs to be documented by Keystone i.e. how to add 
any new type of extension.


regards

David

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread Jorge Williams 

On Aug 6, 2013, at 12:34 PM, Jay Pipes wrote:

> On 08/06/2013 01:21 PM, David Chadwick wrote:
>> 
>> 
>> On 06/08/2013 18:11, Jay Pipes wrote:
>>> What SMTP, DNS and LDAP extensions are in use by systems that need to
>>> interoperate in the same way that Keystone does? <-- This is a genuine
>>> question, not sarcasm. I'm truly curious.
>> 
>> Take SMTP for example. My Thunderbird client needs to know what
>> authentication extensions are implemented by the POP3 server and SMTP
>> server that it is talking to, in order to send and receive email in a
>> secure manner.
>> 
>> In the same way, once Keystone supports say federated login as an
>> extension, a client will need to know if this extension is supported or
>> not. If not, it wont be able to offer it to the end user. (It is not a
>> sensible design for a client to send an extension protocol message to a
>> server and get a 400 Bad Request response. This tells the client
>> nothing. 501 Not Implemented might be a more informative response, but
>> in this case the server has to know that an extension was requested and
>> we have to document that this is the standard response to an
>> unimplemented extension).
> 
> Ah, OK, so I think we're actually closer to one another than first glance. 
> So, I *entirely* agree that if API extensions are available/supported by an 
> API, then there should be an easy way to discover those extensions -- 
> /endpoints is perfectly fine.
> 
> I also agree that a *protocol* should have the flexibility, within its 
> bytestream construct, to extend its scope over time, *without needing to 
> change the underlying protocol*. So, for example, a protocol that leaves 
> itself some way of "growing" over time is, by nature, A Good Thing (tm).
> 
> However, I do *not* believe that resource additions to a REST-ful API 
> necessitate a new API "extension" that must be treated like something that is 
> fundamentally different from the existing resources published in the API.
> 
> Por ejemplo,
> 
> I do not believe the adding a /regions resource should require me to add an 
> API "extension" just to add the resource to the API. I believe we should be 
> able to propose the adding of the /regions resource, debate it, and then add 
> it to a v3.x Keystone API.
> 
> There isn't anything about a region resource that is fundamentally different 
> from some other resource managed by Keystone -- like domains or endpoints -- 
> and therefore I don't believe that adding a /regions resource endpoint should 
> require anything more than a bump in the version of the API.
> 


We should definitely have debates about what should be in core and what should 
be an extension.  And I also agree that some features that are currently 
defined as extensions should probably be built into core. A lot of factors can 
come into play in the decision..How general or niche  the feature is?  How much 
support it has in the community?  Whether or not it's possible for  all 
implementations / deployments to support it etc.   To say that something 
shouldn't be an extension simply because it's exposed as a resource though 
doesn't make sense to me.


> Hope this makes more sense,
> -jay
> 
> p.s. Despite my opinion that /regions resource addition should not be an 
> extension, I'm still submitting a proposed API extension for it ;)
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread Clint Byrum 
Excerpts from Jay Pipes's message of 2013-08-06 10:34:30 -0700:
> On 08/06/2013 01:21 PM, David Chadwick wrote:
> >
> >
> > On 06/08/2013 18:11, Jay Pipes wrote:
> >> What SMTP, DNS and LDAP extensions are in use by systems that need to
> >> interoperate in the same way that Keystone does? <-- This is a genuine
> >> question, not sarcasm. I'm truly curious.
> >
> > Take SMTP for example. My Thunderbird client needs to know what
> > authentication extensions are implemented by the POP3 server and SMTP
> > server that it is talking to, in order to send and receive email in a
> > secure manner.
> >
> > In the same way, once Keystone supports say federated login as an
> > extension, a client will need to know if this extension is supported or
> > not. If not, it wont be able to offer it to the end user. (It is not a
> > sensible design for a client to send an extension protocol message to a
> > server and get a 400 Bad Request response. This tells the client
> > nothing. 501 Not Implemented might be a more informative response, but
> > in this case the server has to know that an extension was requested and
> > we have to document that this is the standard response to an
> > unimplemented extension).
> 
> Ah, OK, so I think we're actually closer to one another than first 
> glance. So, I *entirely* agree that if API extensions are 
> available/supported by an API, then there should be an easy way to 
> discover those extensions -- /endpoints is perfectly fine.
> 
> I also agree that a *protocol* should have the flexibility, within its 
> bytestream construct, to extend its scope over time, *without needing to 
> change the underlying protocol*. So, for example, a protocol that leaves 
> itself some way of "growing" over time is, by nature, A Good Thing (tm).
> 
> However, I do *not* believe that resource additions to a REST-ful API 
> necessitate a new API "extension" that must be treated like something 
> that is fundamentally different from the existing resources published in 
> the API.
> 
> Por ejemplo,
> 
> I do not believe the adding a /regions resource should require me to add 
> an API "extension" just to add the resource to the API. I believe we 
> should be able to propose the adding of the /regions resource, debate 
> it, and then add it to a v3.x Keystone API.
> 
> There isn't anything about a region resource that is fundamentally 
> different from some other resource managed by Keystone -- like domains 
> or endpoints -- and therefore I don't believe that adding a /regions 
> resource endpoint should require anything more than a bump in the 
> version of the API.
> 

Agreed Jay. The successful extensible protocols like IMAP and SMTP are
merely allowing new arguments to existing fundamental functions.

HTTP allows different MIME types. There is no Content-Type extension to
the HTTP protocol. It also allows X-anything, but these are only ways
for a specific group to use the protocol as defined to communicate.

If there is something missing from keystone's model, it should be
added to the API. This does sound more like a cultural problem than a
technical one. Nobody should be afraid to add things to an API. I think
sometimes API writers are put into an ivory tower because they are also
held to higher standards for rigor and documentation. But we should all
be capable of applying that standard to ourselves at some point. 

It would help if we explicitly state that new API bits are encouraged,
rather than making the warm safe place "extension land".

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread Jay Pipes 

On 08/06/2013 01:21 PM, David Chadwick wrote:



On 06/08/2013 18:11, Jay Pipes wrote:

What SMTP, DNS and LDAP extensions are in use by systems that need to
interoperate in the same way that Keystone does? <-- This is a genuine
question, not sarcasm. I'm truly curious.


Take SMTP for example. My Thunderbird client needs to know what
authentication extensions are implemented by the POP3 server and SMTP
server that it is talking to, in order to send and receive email in a
secure manner.

In the same way, once Keystone supports say federated login as an
extension, a client will need to know if this extension is supported or
not. If not, it wont be able to offer it to the end user. (It is not a
sensible design for a client to send an extension protocol message to a
server and get a 400 Bad Request response. This tells the client
nothing. 501 Not Implemented might be a more informative response, but
in this case the server has to know that an extension was requested and
we have to document that this is the standard response to an
unimplemented extension).


Ah, OK, so I think we're actually closer to one another than first 
glance. So, I *entirely* agree that if API extensions are 
available/supported by an API, then there should be an easy way to 
discover those extensions -- /endpoints is perfectly fine.


I also agree that a *protocol* should have the flexibility, within its 
bytestream construct, to extend its scope over time, *without needing to 
change the underlying protocol*. So, for example, a protocol that leaves 
itself some way of "growing" over time is, by nature, A Good Thing (tm).


However, I do *not* believe that resource additions to a REST-ful API 
necessitate a new API "extension" that must be treated like something 
that is fundamentally different from the existing resources published in 
the API.


Por ejemplo,

I do not believe the adding a /regions resource should require me to add 
an API "extension" just to add the resource to the API. I believe we 
should be able to propose the adding of the /regions resource, debate 
it, and then add it to a v3.x Keystone API.


There isn't anything about a region resource that is fundamentally 
different from some other resource managed by Keystone -- like domains 
or endpoints -- and therefore I don't believe that adding a /regions 
resource endpoint should require anything more than a bump in the 
version of the API.


Hope this makes more sense,
-jay

p.s. Despite my opinion that /regions resource addition should not be an 
extension, I'm still submitting a proposed API extension for it ;)


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread David Chadwick 



On 06/08/2013 18:11, Jay Pipes wrote:

What SMTP, DNS and LDAP extensions are in use by systems that need to
interoperate in the same way that Keystone does? <-- This is a genuine
question, not sarcasm. I'm truly curious.


Take SMTP for example. My Thunderbird client needs to know what 
authentication extensions are implemented by the POP3 server and SMTP 
server that it is talking to, in order to send and receive email in a 
secure manner.


In the same way, once Keystone supports say federated login as an 
extension, a client will need to know if this extension is supported or 
not. If not, it wont be able to offer it to the end user. (It is not a 
sensible design for a client to send an extension protocol message to a 
server and get a 400 Bad Request response. This tells the client 
nothing. 501 Not Implemented might be a more informative response, but 
in this case the server has to know that an extension was requested and 
we have to document that this is the standard response to an 
unimplemented extension).


regards

David



Best,
-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread Jay Pipes 

On 08/06/2013 12:57 PM, David Chadwick wrote:



On 06/08/2013 16:53, Jay Pipes wrote:

On 08/06/2013 10:45 AM, David Chadwick wrote:



On 06/08/2013 14:46, Jay Pipes wrote:

API extensions are more hassle than anything else. Let us promote
standards, not endless extensibility at the expense of usability.


This is the crux of the issue. Everyone who participates in
standardisation meetings has their own agenda to follow: their
preferences, likes, dislikes, must have features, etc. This is why
standards end up with optional extensions.


Which standards are you referring to?


Virtually all standards from every standards organisations OASIS, IETF,
ISO etc.
e.g. SMTP is ubiquitous yet it has dozens of extensions.

> The DNS and

LDAP have multiple extensions as well. So you can and need to have
extensions and that does not stop a standard from gaining wide
acceptance and applicability.


What SMTP, DNS and LDAP extensions are in use by systems that need to 
interoperate in the same way that Keystone does? <-- This is a genuine 
question, not sarcasm. I'm truly curious.


Best,
-jay


  *Good* standards, like the HTTP or

ANSI SQL standards, just have a set of interfaces that correspond to a
version. It's only when vendors go outside of the standard to define
what they want when things get fuzzy.

Case in point: the HTTP extension framework:

http://tools.ietf.org/html/rfc2774.html

Last updated in the year 2000. Nobody uses or cares about it. Why?
Because it isn't a standard, and provides the ability for every Tom,
Dick, and Rackspace to reinvent their own HTTP interfaces.

It doesn't make sense. Then, or now.

The point of standards and standards committees is to come to a
compromise and develop a single standard. API Extensions are merely
punting on that responsibility in the name of "customization".

 > If you dont have them, then

you cannot get buy in from sufficient stakeholders. If you do have them,
then you end up with extensibility.

But actually extensibility in my opinion is a "must have" feature, since
no protocol or standard (or Keystone) remains static for ever, and new
features are continually being added to it. Therefore you must have a
way for clients to know what functionality the remote server currently
supports so that it can talk the correct protocol flavour to it.


Extensibility is only a must have for *implementations*, IMHO, not for
the *API*. API extensions are just a way around the hard work of
creating a good, standardized, well-documented API.


No, you need extensibility in the protocols as well.
Cases in point, SMTP and LDAP.

regards

David



Case in point: The Nova API extensions. How many of them are: a) not
documented at all, including the code itself, b) not documented in some
online document somewhere, and c) directly contradict the functionality
in other extensions?

Extensibility, at least in my view, belongs on the implementation/driver
layer. Keystone has done a good job keeping extensibility at its driver
layer so far. It's a shame it doesn't keep it there.

Best,
-jay




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread David Chadwick 



On 06/08/2013 16:53, Jay Pipes wrote:

On 08/06/2013 10:45 AM, David Chadwick wrote:



On 06/08/2013 14:46, Jay Pipes wrote:

API extensions are more hassle than anything else. Let us promote
standards, not endless extensibility at the expense of usability.


This is the crux of the issue. Everyone who participates in
standardisation meetings has their own agenda to follow: their
preferences, likes, dislikes, must have features, etc. This is why
standards end up with optional extensions.


Which standards are you referring to?


Virtually all standards from every standards organisations OASIS, IETF, 
ISO etc.
e.g. SMTP is ubiquitous yet it has dozens of extensions. The DNS and 
LDAP have multiple extensions as well. So you can and need to have 
extensions and that does not stop a standard from gaining wide 
acceptance and applicability.



 *Good* standards, like the HTTP or

ANSI SQL standards, just have a set of interfaces that correspond to a
version. It's only when vendors go outside of the standard to define
what they want when things get fuzzy.

Case in point: the HTTP extension framework:

http://tools.ietf.org/html/rfc2774.html

Last updated in the year 2000. Nobody uses or cares about it. Why?
Because it isn't a standard, and provides the ability for every Tom,
Dick, and Rackspace to reinvent their own HTTP interfaces.

It doesn't make sense. Then, or now.

The point of standards and standards committees is to come to a
compromise and develop a single standard. API Extensions are merely
punting on that responsibility in the name of "customization".

 > If you dont have them, then

you cannot get buy in from sufficient stakeholders. If you do have them,
then you end up with extensibility.

But actually extensibility in my opinion is a "must have" feature, since
no protocol or standard (or Keystone) remains static for ever, and new
features are continually being added to it. Therefore you must have a
way for clients to know what functionality the remote server currently
supports so that it can talk the correct protocol flavour to it.


Extensibility is only a must have for *implementations*, IMHO, not for
the *API*. API extensions are just a way around the hard work of
creating a good, standardized, well-documented API.


No, you need extensibility in the protocols as well.
Cases in point, SMTP and LDAP.

regards

David



Case in point: The Nova API extensions. How many of them are: a) not
documented at all, including the code itself, b) not documented in some
online document somewhere, and c) directly contradict the functionality
in other extensions?

Extensibility, at least in my view, belongs on the implementation/driver
layer. Keystone has done a good job keeping extensibility at its driver
layer so far. It's a shame it doesn't keep it there.

Best,
-jay



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread Dolph Mathews 
On Tue, Aug 6, 2013 at 10:53 AM, Jay Pipes  wrote:

> On 08/06/2013 10:45 AM, David Chadwick wrote:
>
>>
>>
>> On 06/08/2013 14:46, Jay Pipes wrote:
>>
>>> API extensions are more hassle than anything else. Let us promote
>>> standards, not endless extensibility at the expense of usability.
>>>
>>
>> This is the crux of the issue. Everyone who participates in
>> standardisation meetings has their own agenda to follow: their
>> preferences, likes, dislikes, must have features, etc. This is why
>> standards end up with optional extensions.
>>
>
> Which standards are you referring to? *Good* standards, like the HTTP or
> ANSI SQL standards, just have a set of interfaces that correspond to a
> version. It's only when vendors go outside of the standard to define what
> they want when things get fuzzy.
>
> Case in point: the HTTP extension framework:
>
> http://tools.ietf.org/html/**rfc2774.html
>
> Last updated in the year 2000. Nobody uses or cares about it. Why? Because
> it isn't a standard, and provides the ability for every Tom, Dick, and
> Rackspace to reinvent their own HTTP interfaces.
>
> It doesn't make sense. Then, or now.
>
> The point of standards and standards committees is to come to a compromise
> and develop a single standard. API Extensions are merely punting on that
> responsibility in the name of "customization".


First of all, I totally understand, appreciate and agree with your
sentiment. Extensions are very frequently painful, but I'd argue that they
don't have to be.


>
>
> > If you dont have them, then
>
>> you cannot get buy in from sufficient stakeholders. If you do have them,
>> then you end up with extensibility.
>>
>> But actually extensibility in my opinion is a "must have" feature, since
>> no protocol or standard (or Keystone) remains static for ever, and new
>> features are continually being added to it. Therefore you must have a
>> way for clients to know what functionality the remote server currently
>> supports so that it can talk the correct protocol flavour to it.
>>
>
> Extensibility is only a must have for *implementations*, IMHO, not for the
> *API*. API extensions are just a way around the hard work of creating a
> good, standardized, well-documented API.
>

I especially don't see an "API extension" as a way to avoid producing well
documented API's. For example, the accepted extensions to the v3 Identity
API are fully documented from use case through API behavior:


https://github.com/openstack/identity-api/tree/master/openstack-identity-api/v3/src/markdown


> Case in point: The Nova API extensions. How many of them are: a) not
> documented at all, including the code itself, b) not documented in some
> online document somewhere, and c) directly contradict the functionality in
> other extensions?
>

That makes me cringe... API extensions shouldn't be treated as hacks!
That's a cultural problem :(


>
> Extensibility, at least in my view, belongs on the implementation/driver
> layer. Keystone has done a good job keeping extensibility at its driver
> layer so far.


To be fair, extensibility at the driver layer is basically keystone's core
use case: allowing OpenStack to take advantage of your identity data,
wherever it is.


> It's a shame it doesn't keep it there.
>
>
> Best,
> -jay
>
>
> __**_
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.**org 
> http://lists.openstack.org/**cgi-bin/mailman/listinfo/**openstack-dev
>



-- 

-Dolph
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread Adam Young 

On 08/06/2013 11:53 AM, Jay Pipes wrote:

On 08/06/2013 10:45 AM, David Chadwick wrote:



On 06/08/2013 14:46, Jay Pipes wrote:

API extensions are more hassle than anything else. Let us promote
standards, not endless extensibility at the expense of usability.


This is the crux of the issue. Everyone who participates in
standardisation meetings has their own agenda to follow: their
preferences, likes, dislikes, must have features, etc. This is why
standards end up with optional extensions.


Which standards are you referring to? *Good* standards, like the HTTP 
or ANSI SQL standards, just have a set of interfaces that correspond 
to a version. It's only when vendors go outside of the standard to 
define what they want when things get fuzzy.


Case in point: the HTTP extension framework:

http://tools.ietf.org/html/rfc2774.html

Last updated in the year 2000. Nobody uses or cares about it. Why? 
Because it isn't a standard, and provides the ability for every Tom, 
Dick, and Rackspace to reinvent their own HTTP interfaces.


It doesn't make sense. Then, or now.

The point of standards and standards committees is to come to a 
compromise and develop a single standard. API Extensions are merely 
punting on that responsibility in the name of "customization".


> If you dont have them, then

you cannot get buy in from sufficient stakeholders. If you do have them,
then you end up with extensibility.

But actually extensibility in my opinion is a "must have" feature, since
no protocol or standard (or Keystone) remains static for ever, and new
features are continually being added to it. Therefore you must have a
way for clients to know what functionality the remote server currently
supports so that it can talk the correct protocol flavour to it.


Extensibility is only a must have for *implementations*, IMHO, not for 
the *API*. API extensions are just a way around the hard work of 
creating a good, standardized, well-documented API.


Case in point: The Nova API extensions. How many of them are: a) not 
documented at all, including the code itself, b) not documented in 
some online document somewhere, and c) directly contradict the 
functionality in other extensions?


Extensibility, at least in my view, belongs on the 
implementation/driver layer. Keystone has done a good job keeping 
extensibility at its driver layer so far. It's a shame it doesn't keep 
it there.


Interesting.  Dolph and I were just arguing on IRC if Certificates in 
support of PKI tokens belong in Extensions or in Core API.  While I was 
vehemently in support of core API, I could see his rationale for extensions.


Discoverability is one of the key mechanisms of the Web; the ability to 
deduce from the context where to go next.  However, that is much easier 
for humans than for automated processes.  Contracts, on the other hand, 
are bast for automated processes.






Best,
-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread Jay Pipes 

On 08/06/2013 10:45 AM, David Chadwick wrote:



On 06/08/2013 14:46, Jay Pipes wrote:

API extensions are more hassle than anything else. Let us promote
standards, not endless extensibility at the expense of usability.


This is the crux of the issue. Everyone who participates in
standardisation meetings has their own agenda to follow: their
preferences, likes, dislikes, must have features, etc. This is why
standards end up with optional extensions.


Which standards are you referring to? *Good* standards, like the HTTP or 
ANSI SQL standards, just have a set of interfaces that correspond to a 
version. It's only when vendors go outside of the standard to define 
what they want when things get fuzzy.


Case in point: the HTTP extension framework:

http://tools.ietf.org/html/rfc2774.html

Last updated in the year 2000. Nobody uses or cares about it. Why? 
Because it isn't a standard, and provides the ability for every Tom, 
Dick, and Rackspace to reinvent their own HTTP interfaces.


It doesn't make sense. Then, or now.

The point of standards and standards committees is to come to a 
compromise and develop a single standard. API Extensions are merely 
punting on that responsibility in the name of "customization".


> If you dont have them, then

you cannot get buy in from sufficient stakeholders. If you do have them,
then you end up with extensibility.

But actually extensibility in my opinion is a "must have" feature, since
no protocol or standard (or Keystone) remains static for ever, and new
features are continually being added to it. Therefore you must have a
way for clients to know what functionality the remote server currently
supports so that it can talk the correct protocol flavour to it.


Extensibility is only a must have for *implementations*, IMHO, not for 
the *API*. API extensions are just a way around the hard work of 
creating a good, standardized, well-documented API.


Case in point: The Nova API extensions. How many of them are: a) not 
documented at all, including the code itself, b) not documented in some 
online document somewhere, and c) directly contradict the functionality 
in other extensions?


Extensibility, at least in my view, belongs on the implementation/driver 
layer. Keystone has done a good job keeping extensibility at its driver 
layer so far. It's a shame it doesn't keep it there.


Best,
-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread Adam Young 

On 08/06/2013 10:54 AM, Dolph Mathews wrote:


On Tue, Aug 6, 2013 at 9:28 AM, Jorge Williams 
mailto:jorge.willi...@rackspace.com>> 
wrote:



On Aug 6, 2013, at 8:36 AM, Adam Young wrote:

> On 08/06/2013 01:19 AM, Jamie Lennox wrote:
>> Hi all,
>>
>> Partially in response to the trusts API review in keystoneclient
>> (https://review.openstack.org/#/c/39899/ ) and my work on
keystone API
>> version discoverability (spell-check disagrees but I'm going to
assume
>> that's a word - https://review.openstack.org/#/c/38414/ ) I was
thinking
>> about how we should be able to know what/if an extension is
available. I
>> even made a basic blueprint for how i think it should work:
>>

https://blueprints.launchpad.net/python-keystoneclient/+spec/keystoneclient-extensions
and then realized that GET /extensions is only a V2 API.
>
> I'm not certain that the extensions should really be in the v2
or v3.  It always seemed to me that Extensions should be parallel
to, and separate from, the core API.


I agree. Extensions should not be in core, but the mechanism by
which extensions are discovered should be part of the core...right?


Agree. The fact that you call GET /v2.0/extensions or GET 
/v3/extensions instead of GET /extensions just means that we can 
iterate on the "extensions" response itself, not necessarily that the 
extension *only* applies to particular version API being queried 
(that's a different issue).


Agreed.  That makes sense.


So the APIs should be:

v2.0/extensions
or
v3/extensions

but those should return links to:

extensions/some_extension



-jOrGe W.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--

-Dolph


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread Dolph Mathews 
On Tue, Aug 6, 2013 at 9:28 AM, Jorge Williams  wrote:

>
> On Aug 6, 2013, at 8:36 AM, Adam Young wrote:
>
> > On 08/06/2013 01:19 AM, Jamie Lennox wrote:
> >> Hi all,
> >>
> >> Partially in response to the trusts API review in keystoneclient
> >> (https://review.openstack.org/#/c/39899/ ) and my work on keystone API
> >> version discoverability (spell-check disagrees but I'm going to assume
> >> that's a word - https://review.openstack.org/#/c/38414/ ) I was
> thinking
> >> about how we should be able to know what/if an extension is available. I
> >> even made a basic blueprint for how i think it should work:
> >>
> https://blueprints.launchpad.net/python-keystoneclient/+spec/keystoneclient-extensionsand
>  then realized that GET /extensions is only a V2 API.
> >
> > I'm not certain that the extensions should really be in the v2 or v3.
>  It always seemed to me that Extensions should be parallel to, and separate
> from, the core API.
>
>
> I agree. Extensions should not be in core, but the mechanism by which
> extensions are discovered should be part of the core...right?
>

Agree. The fact that you call GET /v2.0/extensions or GET /v3/extensions
instead of GET /extensions just means that we can iterate on the
"extensions" response itself, not necessarily that the extension *only*
applies to particular version API being queried (that's a different issue).


>
> -jOrGe W.
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 

-Dolph
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread David Chadwick 



On 06/08/2013 14:46, Jay Pipes wrote:

API extensions are more hassle than anything else. Let us promote
standards, not endless extensibility at the expense of usability.


This is the crux of the issue. Everyone who participates in 
standardisation meetings has their own agenda to follow: their 
preferences, likes, dislikes, must have features, etc. This is why 
standards end up with optional extensions. If you dont have them, then 
you cannot get buy in from sufficient stakeholders. If you do have them, 
then you end up with extensibility.


But actually extensibility in my opinion is a "must have" feature, since 
no protocol or standard (or Keystone) remains static for ever, and new 
features are continually being added to it. Therefore you must have a 
way for clients to know what functionality the remote server currently 
supports so that it can talk the correct protocol flavour to it.


regards

David



Best,
-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread Jorge Williams 

On Aug 6, 2013, at 8:36 AM, Adam Young wrote:

> On 08/06/2013 01:19 AM, Jamie Lennox wrote:
>> Hi all,
>> 
>> Partially in response to the trusts API review in keystoneclient
>> (https://review.openstack.org/#/c/39899/ ) and my work on keystone API
>> version discoverability (spell-check disagrees but I'm going to assume
>> that's a word - https://review.openstack.org/#/c/38414/ ) I was thinking
>> about how we should be able to know what/if an extension is available. I
>> even made a basic blueprint for how i think it should work:
>> https://blueprints.launchpad.net/python-keystoneclient/+spec/keystoneclient-extensions
>>  and then realized that GET /extensions is only a V2 API.
> 
> I'm not certain that the extensions should really be in the v2 or v3.  It 
> always seemed to me that Extensions should be parallel to, and separate from, 
> the core API.


I agree. Extensions should not be in core, but the mechanism by which 
extensions are discovered should be part of the core...right?

-jOrGe W.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread David Chadwick 
as a general principle I would think it is a good idea for clients to be 
able to interrogate Keystone to determine what extensions it supports. 
Most protocols have some mechanism for determining what 
extensions/versions are supported by the server, and what optional 
features are implemented.


regards

David

On 06/08/2013 06:19, Jamie Lennox wrote:

Hi all,

Partially in response to the trusts API review in keystoneclient
(https://review.openstack.org/#/c/39899/ ) and my work on keystone API
version discoverability (spell-check disagrees but I'm going to assume
that's a word - https://review.openstack.org/#/c/38414/ ) I was thinking
about how we should be able to know what/if an extension is available. I
even made a basic blueprint for how i think it should work:
https://blueprints.launchpad.net/python-keystoneclient/+spec/keystoneclient-extensions
 and then realized that GET /extensions is only a V2 API.

Is this intentional? I was starting to make a review to add it to
identity-api but is there the intention that extensions should show up
within the endpoint APIs? There is no reason it couldn't work that way
and DELETE would just fail.

I am not convinced that it is a good idea though and I just want to
check if this is something that has been discussed or purposefully
handled this way or something we need to add.


Thanks,

Jamie


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread Jay Pipes 

On 08/06/2013 01:19 AM, Jamie Lennox wrote:

Hi all,

Partially in response to the trusts API review in keystoneclient
(https://review.openstack.org/#/c/39899/ ) and my work on keystone API
version discoverability (spell-check disagrees but I'm going to assume
that's a word - https://review.openstack.org/#/c/38414/ ) I was thinking
about how we should be able to know what/if an extension is available. I
even made a basic blueprint for how i think it should work:
https://blueprints.launchpad.net/python-keystoneclient/+spec/keystoneclient-extensions
 and then realized that GET /extensions is only a V2 API.

Is this intentional? I was starting to make a review to add it to
identity-api but is there the intention that extensions should show up
within the endpoint APIs? There is no reason it couldn't work that way
and DELETE would just fail.


I would hope that extensions would *not* show up in the endpoints API.

Frankly, I'm not a fan of API extensions at all. I think they are silly 
and just promote an inconsistent and fractured user experience. I would 
highly prefer to just have a single API, versioned, with documentation 
online and in a versions/ resource that indicates what was changed, 
added, and deleted in each version.


If some vendor wants to provide some special API resource that naturally 
belongs in a related API -- for instance, trusts in the OpenStack 
Identity API -- then the new resource should simply be added to the one 
and only Identity API, the version of the API incremented, and on we go.


API extensions are more hassle than anything else. Let us promote 
standards, not endless extensibility at the expense of usability.


Best,
-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-06 Thread Adam Young 

On 08/06/2013 01:19 AM, Jamie Lennox wrote:

Hi all,

Partially in response to the trusts API review in keystoneclient
(https://review.openstack.org/#/c/39899/ ) and my work on keystone API
version discoverability (spell-check disagrees but I'm going to assume
that's a word - https://review.openstack.org/#/c/38414/ ) I was thinking
about how we should be able to know what/if an extension is available. I
even made a basic blueprint for how i think it should work:
https://blueprints.launchpad.net/python-keystoneclient/+spec/keystoneclient-extensions
 and then realized that GET /extensions is only a V2 API.


I'm not certain that the extensions should really be in the v2 or v3.  
It always seemed to me that Extensions should be parallel to, and 
separate from, the core API.




Is this intentional? I was starting to make a review to add it to
identity-api but is there the intention that extensions should show up
within the endpoint APIs? There is no reason it couldn't work that way
and DELETE would just fail.

I am not convinced that it is a good idea though and I just want to
check if this is something that has been discussed or purposefully
handled this way or something we need to add.


Thanks,

Jamie


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Keystone] V3 Extensions Discoverability

2013-08-05 Thread Jamie Lennox 
Hi all, 

Partially in response to the trusts API review in keystoneclient
(https://review.openstack.org/#/c/39899/ ) and my work on keystone API
version discoverability (spell-check disagrees but I'm going to assume
that's a word - https://review.openstack.org/#/c/38414/ ) I was thinking
about how we should be able to know what/if an extension is available. I
even made a basic blueprint for how i think it should work:
https://blueprints.launchpad.net/python-keystoneclient/+spec/keystoneclient-extensions
 and then realized that GET /extensions is only a V2 API.

Is this intentional? I was starting to make a review to add it to
identity-api but is there the intention that extensions should show up
within the endpoint APIs? There is no reason it couldn't work that way
and DELETE would just fail. 

I am not convinced that it is a good idea though and I just want to
check if this is something that has been discussed or purposefully
handled this way or something we need to add.


Thanks, 

Jamie 


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev