On 08/04/2017 04:03 PM, Dean Troyer wrote:
On Fri, Aug 4, 2017 at 11:52 AM, Monty Taylor <mord...@inaugust.com> wrote:
[...]
* Both shade and python-openstackclient have investigated using openstacksdk
as their REST layer but were unable to because it puts some abstractions in
layers that make it impossible to do some of the advanced things we need.
OSC _does_ use the SDK for all Network API commands. That was a
mistake in the sense that we did it before SDK 1.0 was released and
still do not have 1.0.
Yah - although it got you out of the competing cliff issue, so that was
good.
* openstacksdk has internal implementations of things that exist at
different points in the stack. We just added full support for version
service and version discovery to keystoneauth, but openstacksdk has its own
layer for that so it both can't use the ksa implementation and is not
compliant with the API-WG consume guidelines.
This was intended to make it free from dependencies, when first
concieved, none of these other libs existed.
Oh - totally. It made total sense at the time ... it's just the
surrounding context has changed.
I am coming around to believe that we need to slice a couple more
things up so they can be composed as needed rather then bite off the
entire thing in once piece.
[...]
I'd propose we have the shade team adopt the python-openstacksdk codebase.
++
This is obviously an aggressive suggestion and essentially represents a
takeover of a project. We don't have the luxury of humans to work on things
that we once had, so I think as a community we should be realistic about the
benefits of consolidation and the downside to continuing to have 2 different
python SDKs.
++
I thought it would be natural for OSC to adopt the SDK someday if
Brian did not get around to making it official, but the current
circumstances make it clear that we (OSC) do not have the resources to
do this. This proposal is much better and leads to a natural
coalescence of the parallel goals and projects.
Doing that implies the following:
* Rework the underlying guts of openstacksdk to make it possible to replace
shade's REST layer with openstacksdk. openstacksdk still doesn't have a 1.0
release, so we can break the few things we'll need to break.
Sigh. OSC has been using the Network components of the SDK for a long
time in spite of SDK not being at 1.0. In retrospect that was a
mistake on my part but I believed at the time that 1.0 was close and
we had already ignored Network for far too long. We already have one
compatibility layer in the SDK due to the proxy refactor work that was
supposed to be the last thing before 1.0.
I don't think we need to break the things you're using, fwiw. In fact,
we can probably take it as a first-order requirement to not break OSC -
unless we find something SUPER intractable - and even then we should
talk about it first.
[...]
* Merge the two repos and retire one of them. Specifics on the mechanics of
this below, but this will either result in moving the resource and service
layer in openstacksdk into shade and adding appropriate attributes to the
shade.OpenStackCloud object, or moving the shade.OpenStackCloud into
something like openstack.cloud and making a shade backwards-compate shim. I
lean towards the first, as we've been telling devs "use shade to talk to
OpenStack" at hackathons and bootcamps and I'd rather avoid the messaging
shift. However, pointing to an SDK called "The Python OpenStack SDK" and
telling people to use it certainly has its benefits from a messaging
perspective.
I don't have a big concern about which repo is maintained. For OSC I
want what amounts to a low-level REST API, one example of which can be
found in openstackclient.api.*. Currently Shade is not quite the
right thing to back a CLI but now has the layer I want, and SDK does
not have that layer at all (it was proposed very early on and not
merged).
FWIW - now that we landed version discovery and microversion support in
keystoneauth - I'd say keystoneauth Adapter is actually now the
low-level REST API:
client = keystoneauth1.adapter.Adapter(
session=session,
service_type='compute',
region_name='DFW',
min_version='2',
max_version='2.latest')
endpoint_data = client.get_endpoint_data()
print(
"Microversion range is: ",
endpoint_data.min_microversion,
endpoint_data.max_microversion)
# want 2.31 of servers
server_response = client.get('/servers', microversion='2.31')
server = server_response.json()['servers'][0]
# want 2.14 of keypairs
keypair_response = client.get('/keypairs', microversion='2.14')
# Don't care on volume attachments - don't specify one.
volume_attachments_response = client.get(
'/servers/{id}/os-volume_attachments'.format(id=server['id']))
Is it better to have a single monolithic thing that has three
conceptual layers internally that can be individually consumed or to
have three things that get layered as needed?
It's a good question. I'm leaning to single repo for shade and sdk
mainly because since both do direct REST and dont' have other
dependencies, there's not really much difference from a composability
standpoint. If you just want the low-level SDK functions, use them. If
you want the hide-the-difference functions, use them - doesn't gain much
to have them split.
That said - if merging them is too much work, it's not super IMPORTANT
to merge them either.
* drop keystoneauth.Session subclass. It's over-riding things at the wrong
layer. keystoneauth Adapter is the thing it wants to be.
FWIW, OSC has this problem too...
Yah - I'm totally sorry I forgot to try to get your timing wrapper in to
ksa Session this pass so that you could drop that wrapper...
* suitability for python-openstackclient. Dean and Steve have been laying in
the groundwork for doing direct-REST in python-openstackclient because
python-*client are a mess from an end-user perspective and openstacksdk
isn't suitable. If we can sync on requirements hopefully we can produce
something that python-openstackclient can honestly use for that layer
instead of needing local code.
As I mentioned, we already use the Networking portions of SDK, even
without a 1.0, and it has bit us already a couple of times. It has
long been my plan to convert to using the SDK, but that was when I
believed there would also be a lower-level API exposed that did not
require all of the application-level goodness and abstractions.
I personally feel like splitting out the low-level REST API layers
into a stand-alone piece that shade, SDK and OSC can all benefit from
would be our best course, but then I have been wrong about this
layering thing in the past, so I throw it out there to have something
that can be used to push against to get what everyone else seems to
want.
Nod. I think that's ksa. We're doing almost nothing at the REST layer in
shade at this point - we're just using ksa Adapter. We have a wrapper
around it for two reasons:
- TaskManager support for nodepool
- translating ksa REST exceptions to shade exceptions for backwards compat
Then we do data normalization - but that's a 'hide cloud differences'
thing, not a thing that should be at that lower level.
rods actually just finished removing one additional thing we were doing,
which was stripping the top-level key from the results if there was one.
Turns out doing that really screws with pagination. whoops.
In any case, I agree with the need for a low-level REST interface.
Mostly since last we talked about it the missing pieces have hit ksa.
Looking at openstackclient.api.* though - we may be meaning different
things when we say "low level REST interface".
From the shade/sdk perspective, the extra metadata that's tracked for
each Resource object will, I _believe_ make it easier to add basic new
CRUD for each resource type. I could certainly be VERY wrong about that
though.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
