Re: [openstack-dev] [ara] Best practices on what to return from REST API ?

[David Moreau Simard]

On Wed, Aug 9, 2017 at 10:02 AM, Chris Dent  wrote:
>
> If people remember to actually read this and respond, you'll likely get
> as many different responses to this as there are options.
>

I expected as much but I very much appreciate your reply regardless!
\o/

>
> From there rather than /{collection} for list and
> /{collection}?id={id} for single entity I'd do:
>
> /{collection} # list
> /{collection}?filter=to_smaller_list # e.g. playbook_id
> /{collection}/id  # single entity
>

That makes sense.

>
> In the collection response only include the bare minimum of data, in
> the single entity include:
>
> - whatever is thought of the entity
> - anything that you think of as a child, represent as links
>   elsewhere
> - it is makes sense to have /{collection}/id/{child} and {child} is
>   actually some other /{collection}/id, consider making the child
>   URI a redirect (that is /{collection/id/{child} redirects to
>   /{childtype}/{child id}
> - if a child is actually children then you'll want to adjust the
>   grammar somewhat: /{collection}/id/{child collection}/id
>

I've read about HATEOAS [1] and it's something I will likely implement.
So, if I'm understanding correctly, you would perhaps only include
references to the childs but not their actual data ?

With the concept of HATEOAS, it would go something like:
# Example task
{
"id": 8,
"links": [
{
"href": "/api/v1/tasks/8",
"rel": "self"
}
],
"name": "Ansible >=2.2: include_role tasks",
"playbook_id": 5,
"results": [
{
"href": "/api/v1/results/1", # or /api/v1/tasks/8/results/1
"rel": "1"
},
{
"href": "/api/v1/results/2",
"rel": "2"
},
# ...
]
}

I think that would be manageable and it wasn't something I had considered.

> This provides you with good urls for things, straightforward ways of
> getting at data, and the capacity to spawn off concurrent requests
> in response to an initial query.
>
> This is essentially the same as your option 2, but with more
> structure to the URLs.

Thanks for the input !

[1]: https://spring.io/understanding/HATEOAS

David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ara] Best practices on what to return from REST API ?

[Chris Dent]

On Wed, 9 Aug 2017, David Moreau Simard wrote:


So I'm making what I think is good progress towards the python and
REST API implementation for ARA but now I have a question.
I've made the following API "GET" endpoints:


If people remember to actually read this and respond, you'll likely get
as many different responses to this as there are options.

Alot of it is preference, but you've mentioned that you are trying
to optimize for scale and concurrency so I think there are some things
you can do to maintain a reasonable grammar, hew to standards (both
real and the dreaded "best practice"), keep things simple, and be
fast (in aggregate). You've overcome the most important hurdle
already by enumerating your resources (as nouns):


- files (saved Ansible files)
- hosts (hosts involved in playbooks and their facts, if available)
- playbooks (data about the actual playbook and ansible parameters)
- plays (data about plays)
- results (actual results like 'ok', 'changed' as well as the data
from the Ansible module output)
- tasks (data about the task, like name, action and file path)



From there rather than /{collection} for list and

/{collection}?id={id} for single entity I'd do:

/{collection} # list
/{collection}?filter=to_smaller_list # e.g. playbook_id
/{collection}/id  # single entity

In the collection response only include the bare minimum of data, in
the single entity include:

- whatever is thought of the entity
- anything that you think of as a child, represent as links
  elsewhere
- it is makes sense to have /{collection}/id/{child} and {child} is
  actually some other /{collection}/id, consider making the child
  URI a redirect (that is /{collection/id/{child} redirects to
  /{childtype}/{child id}
- if a child is actually children then you'll want to adjust the
  grammar somewhat: /{collection}/id/{child collection}/id

This provides you with good urls for things, straightforward ways of
getting at data, and the capacity to spawn off concurrent requests
in response to an initial query.

This is essentially the same as your option 2, but with more
structure to the URLs.


--
Chris Dent  (⊙＿⊙') https://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ara] Best practices on what to return from REST API ?

[David Moreau Simard]

Hi,

So I'm making what I think is good progress towards the python and
REST API implementation for ARA but now I have a question.
I've made the following API "GET" endpoints:

- files (saved Ansible files)
- hosts (hosts involved in playbooks and their facts, if available)
- playbooks (data about the actual playbook and ansible parameters)
- plays (data about plays)
- results (actual results like 'ok', 'changed' as well as the data
from the Ansible module output)
- tasks (data about the task, like name, action and file path)

Each endpoint is self-sufficient and allows consumers to search
individual endpoints as required, for example:
(See attached links for actual and real output)

- GET /api/v1/plays # [1] List all plays
- GET /api/v1/plays?id=1 # [2] Get play with id 1
- GET /api/v1/plays?playbook_id=1 # [3] List all plays for playbook_id 1

The mechanism and logic is the same for all endpoints.
Where I get a bit doubtful is because some components have
relationships, like the following:

- files have file contents (file.content) [4]
- hosts have facts (host.facts) [5]
- tasks have results (task.results) [6]
- playbooks basically aggregate everything (playbook.files,
playbook.hosts, playbook.plays, playbook.results, playbook.tasks) so
it's potentially huge.

So, what is the best practice here ?
I'm thinking that returning all the available data on a LIST call (GET
/api/v1/playbooks) is nuts because of the potential amount of data
involved.

I'm torn between two different approaches but I would be happy to be
convinced there's a better way.

1) Only return "child" resources on specific call
So, when doing a GET on something that is expected to return a list
(GET /api/v1/playbooks), do not return child resources.
And, when doing a GET on the specific resource (GET
/api/v1/playbooks?id=1), display all the child resources.
This would be sort of analogous to how python-openstackclient has
different fields when doing "openstack server list" and "openstack
server show".

However, the amount of data for even a single playbook is likely very
significant at scale.

2) Require sub calls for different components
In this method, we would not return child resources when doing a list
call (GET /api/v1/playbooks) or when doing a specific call (GET
/api/v1/playbooks?id=1).
To get child resources, you would need to pick them up one by one, ex:

for playbook in playbooks:
   files = FileApi.get(playbook_id=playbook['id']
   hosts = HostApi.get(playbook_id=playbook['id']
   plays = PlayApi.get(playbook_id=playbook['id']
   [...]

I feel like approach #2 would be best -- for running at scale but also
in the context of loading data asynchronously and concurrently.

Any opinions ?

Thanks !

[1]: http://paste.openstack.org/raw/617868/
[2]: http://paste.openstack.org/raw/617869/
[3]: http://paste.openstack.org/raw/617870/
[4]: http://paste.openstack.org/raw/617871/
[5]: http://paste.openstack.org/raw/617872/
[6]: http://paste.openstack.org/raw/617873/

David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev