Re: [Pulp-dev] Single-Table Content API Changes, Performance Discussion

[Dennis Kliban]

On Mon, Nov 19, 2018 at 6:20 PM Daniel Alley  wrote:

> Some of the API changes that are required by single-table-content would be
> beneficial even if we didn't go forwards with the modelling changes.  For
> instance, currently we have single endpoints for each of
> repository_version/.../content/,  .../added_content/, and
> .../removed_content/ which mix content of all types together.  This makes
> it impossible for clients to expect the data returned to expect any
> particular schema.  What the single-table-content does is to provide
> separate query urls for each content type present in the repository
> version, which I believe is a usability win for us, and it's something we
> could implement without using any of the modelling changes.
>
>
The current behavior of the 'content' APIs is already causing a problem for
our OpenAPI 2.0 schema. OpenAPI 2.0 does not support polymorphic responses.
We are currently tracking problem with a bug[0]. The only way to resolve
this problem is to provide APIs that return heterogeneous types.

[0] https://pulp.plan.io/issues/4052


> Besides being a general update, I'd like to start a discussion to
> understand:  is changing the Pulp 3 API so that it's organized around
> content type URLs OK with everyone? This resolves the usability issues of
> returning mixed types. Are there any downsides with this approach?
>
> To clarify what I mean on that last point -- by "content type URLs" I mean
> that where you currently get back the url "
> /pulp/api/v3/repository_version/.../content/" under the "_content" field
> on a repoversion, you would instead get back something like
>
> { "pulp_file.filecontent":
> "/pulp/api/v3/content/file/files/?repository_version=.. }
>

I am +1 to making this change to our REST API.


>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Single-Table Content API Changes, Performance Discussion

[David Davis]

I’m also +1 to the single-content API changes.

David


On Tue, Nov 20, 2018 at 12:32 PM Dennis Kliban  wrote:

> On Mon, Nov 19, 2018 at 6:20 PM Daniel Alley  wrote:
>
>> Some of the API changes that are required by single-table-content would
>> be beneficial even if we didn't go forwards with the modelling changes.
>> For instance, currently we have single endpoints for each of
>> repository_version/.../content/,  .../added_content/, and
>> .../removed_content/ which mix content of all types together.  This makes
>> it impossible for clients to expect the data returned to expect any
>> particular schema.  What the single-table-content does is to provide
>> separate query urls for each content type present in the repository
>> version, which I believe is a usability win for us, and it's something we
>> could implement without using any of the modelling changes.
>>
>>
> The current behavior of the 'content' APIs is already causing a problem
> for our OpenAPI 2.0 schema. OpenAPI 2.0 does not support polymorphic
> responses. We are currently tracking problem with a bug[0]. The only way to
> resolve this problem is to provide APIs that return heterogeneous types.
>
> [0] https://pulp.plan.io/issues/4052
>
>
>> Besides being a general update, I'd like to start a discussion to
>> understand:  is changing the Pulp 3 API so that it's organized around
>> content type URLs OK with everyone? This resolves the usability issues of
>> returning mixed types. Are there any downsides with this approach?
>>
>> To clarify what I mean on that last point -- by "content type URLs" I
>> mean that where you currently get back the url "
>> /pulp/api/v3/repository_version/.../content/" under the "_content" field
>> on a repoversion, you would instead get back something like
>>
>> { "pulp_file.filecontent":
>> "/pulp/api/v3/content/file/files/?repository_version=.. }
>>
>
> I am +1 to making this change to our REST API.
>
>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Single-Table Content API Changes, Performance Discussion

[Jeff Ortel]

On 11/20/18 11:31 AM, Dennis Kliban wrote:
On Mon, Nov 19, 2018 at 6:20 PM Daniel Alley > wrote:



Some of the API changes that are required by single-table-content
would be beneficial even if we didn't go forwards with the
modelling changes. For instance, currently we have single
endpoints for each of repository_version/.../content/,
.../added_content/, and .../removed_content/ which mix content of
all types together.  This makes it impossible for clients to
expect the data returned to expect any particular schema.  What
the single-table-content does is to provide separate query urls
for each content type present in the repository version, which I
believe is a usability win for us, and it's something we could
implement without using any of the modelling changes.


The current behavior of the 'content' APIs is already causing a 
problem for our OpenAPI 2.0 schema. OpenAPI 2.0 does not support 
polymorphic responses. We are currently tracking problem with a 
bug[0]. The only way to resolve this problem is to provide APIs that 
return heterogeneous types.


[0] https://pulp.plan.io/issues/4052

Besides being a general update, I'd like to start a discussionto
understand:  is changing the Pulp 3 API so that it's organized
around content type URLs OK with everyone? This resolves the
usability issues of returning mixed types. Are there any downsides
with this approach?

To clarify what I mean on that last point -- by "content type
URLs" I mean that where you currently get back the url
"/pulp/api/v3/repository_version/.../content/" under the
"_content" field on a repoversion, you would instead get back
something like

{ "pulp_file.filecontent":
"/pulp/api/v3/content/file/files/?repository_version=.. }


I am +1 to making this change to our REST API.


+1



___
Pulp-dev mailing list
Pulp-dev@redhat.com 
https://www.redhat.com/mailman/listinfo/pulp-dev


___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev


___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Single-Table Content API Changes, Performance Discussion

[Dana Walker]

+1

Dana Walker

Associate Software Engineer

Red Hat





On Mon, Nov 26, 2018 at 10:55 AM Jeff Ortel  wrote:

>
>
> On 11/20/18 11:31 AM, Dennis Kliban wrote:
>
> On Mon, Nov 19, 2018 at 6:20 PM Daniel Alley  wrote:
>
>>
>> Some of the API changes that are required by single-table-content would
>> be beneficial even if we didn't go forwards with the modelling changes.
>> For instance, currently we have single endpoints for each of
>> repository_version/.../content/,  .../added_content/, and
>> .../removed_content/ which mix content of all types together.  This makes
>> it impossible for clients to expect the data returned to expect any
>> particular schema.  What the single-table-content does is to provide
>> separate query urls for each content type present in the repository
>> version, which I believe is a usability win for us, and it's something we
>> could implement without using any of the modelling changes.
>>
>>
> The current behavior of the 'content' APIs is already causing a problem
> for our OpenAPI 2.0 schema. OpenAPI 2.0 does not support polymorphic
> responses. We are currently tracking problem with a bug[0]. The only way to
> resolve this problem is to provide APIs that return heterogeneous types.
>
> [0] https://pulp.plan.io/issues/4052
>
>
>> Besides being a general update, I'd like to start a discussion to
>> understand:  is changing the Pulp 3 API so that it's organized around
>> content type URLs OK with everyone? This resolves the usability issues of
>> returning mixed types. Are there any downsides with this approach?
>>
>> To clarify what I mean on that last point -- by "content type URLs" I
>> mean that where you currently get back the url "
>> /pulp/api/v3/repository_version/.../content/" under the "_content" field
>> on a repoversion, you would instead get back something like
>>
>> { "pulp_file.filecontent":
>> "/pulp/api/v3/content/file/files/?repository_version=.. }
>>
>
> I am +1 to making this change to our REST API.
>
>
> +1
>
>
>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>
> ___
> Pulp-dev mailing 
> listPulp-dev@redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Single-Table Content API Changes, Performance Discussion

[Brian Bouterse]

I've done cprofile and code analysis and I don't think we can make the
single-content branches as fast as 'master' currently is. I posted the
reasoning for this analysis here [0]. Per the previous discussion, since
this performance improvement actually slows things down, we should just
close these branches. I believe we also want to adopt the API changes
@dalley posted [1][2] which need review and testing.

Any other ideas on how to make this work better are welcome. As it is now
I'll just say thanks to everyone who contributed to this effort, and a
special thank you to @dalley for doing so much coding and days worth of
effort on it.

[0]: https://pulp.plan.io/issues/3770#note-17
[1]: https://github.com/pulp/pulp/pull/3774
[2]: https://github.com/pulp/pulp_file/pull/133



On Tue, Nov 27, 2018 at 5:29 PM Dana Walker  wrote:

> +1
>
> Dana Walker
>
> Associate Software Engineer
>
> Red Hat
>
> 
> 
>
>
> On Mon, Nov 26, 2018 at 10:55 AM Jeff Ortel  wrote:
>
>>
>>
>> On 11/20/18 11:31 AM, Dennis Kliban wrote:
>>
>> On Mon, Nov 19, 2018 at 6:20 PM Daniel Alley  wrote:
>>
>>>
>>> Some of the API changes that are required by single-table-content would
>>> be beneficial even if we didn't go forwards with the modelling changes.
>>> For instance, currently we have single endpoints for each of
>>> repository_version/.../content/,  .../added_content/, and
>>> .../removed_content/ which mix content of all types together.  This makes
>>> it impossible for clients to expect the data returned to expect any
>>> particular schema.  What the single-table-content does is to provide
>>> separate query urls for each content type present in the repository
>>> version, which I believe is a usability win for us, and it's something we
>>> could implement without using any of the modelling changes.
>>>
>>>
>> The current behavior of the 'content' APIs is already causing a problem
>> for our OpenAPI 2.0 schema. OpenAPI 2.0 does not support polymorphic
>> responses. We are currently tracking problem with a bug[0]. The only way to
>> resolve this problem is to provide APIs that return heterogeneous types.
>>
>> [0] https://pulp.plan.io/issues/4052
>>
>>
>>> Besides being a general update, I'd like to start a discussion to
>>> understand:  is changing the Pulp 3 API so that it's organized around
>>> content type URLs OK with everyone? This resolves the usability issues of
>>> returning mixed types. Are there any downsides with this approach?
>>>
>>> To clarify what I mean on that last point -- by "content type URLs" I
>>> mean that where you currently get back the url "
>>> /pulp/api/v3/repository_version/.../content/" under the "_content"
>>> field on a repoversion, you would instead get back something like
>>>
>>> { "pulp_file.filecontent":
>>> "/pulp/api/v3/content/file/files/?repository_version=.. }
>>>
>>
>> I am +1 to making this change to our REST API.
>>
>>
>> +1
>>
>>
>>
>>>
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>
>> ___
>> Pulp-dev mailing 
>> listPulp-dev@redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Single-Table Content API Changes, Performance Discussion

[Dennis Kliban]

On Tue, Nov 20, 2018 at 12:31 PM Dennis Kliban  wrote:

> On Mon, Nov 19, 2018 at 6:20 PM Daniel Alley  wrote:
>
>> Some of the API changes that are required by single-table-content would
>> be beneficial even if we didn't go forwards with the modelling changes.
>> For instance, currently we have single endpoints for each of
>> repository_version/.../content/,  .../added_content/, and
>> .../removed_content/ which mix content of all types together.  This makes
>> it impossible for clients to expect the data returned to expect any
>> particular schema.  What the single-table-content does is to provide
>> separate query urls for each content type present in the repository
>> version, which I believe is a usability win for us, and it's something we
>> could implement without using any of the modelling changes.
>>
>>
> The current behavior of the 'content' APIs is already causing a problem
> for our OpenAPI 2.0 schema. OpenAPI 2.0 does not support polymorphic
> responses. We are currently tracking problem with a bug[0]. The only way to
> resolve this problem is to provide APIs that return heterogeneous types.
>
> [0] https://pulp.plan.io/issues/4052
>
>
>> Besides being a general update, I'd like to start a discussion to
>> understand:  is changing the Pulp 3 API so that it's organized around
>> content type URLs OK with everyone? This resolves the usability issues of
>> returning mixed types. Are there any downsides with this approach?
>>
>> To clarify what I mean on that last point -- by "content type URLs" I
>> mean that where you currently get back the url "
>> /pulp/api/v3/repository_version/.../content/" under the "_content" field
>> on a repoversion, you would instead get back something like
>>
>> { "pulp_file.filecontent":
>> "/pulp/api/v3/content/file/files/?repository_version=.. }
>>
>
> I am +1 to making this change to our REST API.
>

Thank you Daniel for putting together the patches[0,1] to make these
changes possible. I've had a chance to try out the Python bindings. When
using the bindings, I discovered that I could not do anything with the URLs
returned for each content type added or removed. Making the request to
those URLs requires making a call that looks like this:

api.content_file_files_list(repository_version_added=repository_version_1.href)

What if instead the API returned the number of each content type added or
removed. So a repository version response would look like:

{'base_version': None,
 'content_added': {'pulp_file.file': 4},
 'content_removed': {'pulp_file.file': 1},
 'content_summary': {'pulp_file.file': '3'},
 'created': datetime.datetime(2018, 12, 5, 23, 34, 26, 948749,
tzinfo=tzlocal()),
 'href': '/pulp/api/v3/repositories/4/versions/1/',
 'number': 1}

Thoughts?

[0] https://github.com/pulp/pulp/pull/3774
[1] https://github.com/pulp/pulp_file/pull/133


>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Single-Table Content API Changes, Performance Discussion

[Daniel Alley]

I'm not necessarily against this but I'll recap some points I made on IRC:

The burden of knowing where to go to get that information would be pushed
off onto the API user.  If we're not returning the URL, then anyone using
the API must know that they need to query /pulp/api/v3/content/file/files/
(and likewise for every other content type), and that they need to use a
filter for repository_version=... or repository_version_added=... and so so
on.

I'm not sure how that would work, how that knowledge would be provided or
if it's something that can be hardcoded into the bindings.  If you think
that's possible, then I'm open to it.



On Thu, Dec 6, 2018 at 4:53 PM Dennis Kliban  wrote:

> On Tue, Nov 20, 2018 at 12:31 PM Dennis Kliban  wrote:
>
>> On Mon, Nov 19, 2018 at 6:20 PM Daniel Alley  wrote:
>>
>>> Some of the API changes that are required by single-table-content would
>>> be beneficial even if we didn't go forwards with the modelling changes.
>>> For instance, currently we have single endpoints for each of
>>> repository_version/.../content/,  .../added_content/, and
>>> .../removed_content/ which mix content of all types together.  This makes
>>> it impossible for clients to expect the data returned to expect any
>>> particular schema.  What the single-table-content does is to provide
>>> separate query urls for each content type present in the repository
>>> version, which I believe is a usability win for us, and it's something we
>>> could implement without using any of the modelling changes.
>>>
>>>
>> The current behavior of the 'content' APIs is already causing a problem
>> for our OpenAPI 2.0 schema. OpenAPI 2.0 does not support polymorphic
>> responses. We are currently tracking problem with a bug[0]. The only way to
>> resolve this problem is to provide APIs that return heterogeneous types.
>>
>> [0] https://pulp.plan.io/issues/4052
>>
>>
>>> Besides being a general update, I'd like to start a discussion to
>>> understand:  is changing the Pulp 3 API so that it's organized around
>>> content type URLs OK with everyone? This resolves the usability issues of
>>> returning mixed types. Are there any downsides with this approach?
>>>
>>> To clarify what I mean on that last point -- by "content type URLs" I
>>> mean that where you currently get back the url "
>>> /pulp/api/v3/repository_version/.../content/" under the "_content"
>>> field on a repoversion, you would instead get back something like
>>>
>>> { "pulp_file.filecontent":
>>> "/pulp/api/v3/content/file/files/?repository_version=.. }
>>>
>>
>> I am +1 to making this change to our REST API.
>>
>
> Thank you Daniel for putting together the patches[0,1] to make these
> changes possible. I've had a chance to try out the Python bindings. When
> using the bindings, I discovered that I could not do anything with the URLs
> returned for each content type added or removed. Making the request to
> those URLs requires making a call that looks like this:
>
>
> api.content_file_files_list(repository_version_added=repository_version_1.href)
>
> What if instead the API returned the number of each content type added or
> removed. So a repository version response would look like:
>
> {'base_version': None,
>  'content_added': {'pulp_file.file': 4},
>  'content_removed': {'pulp_file.file': 1},
>  'content_summary': {'pulp_file.file': '3'},
>  'created': datetime.datetime(2018, 12, 5, 23, 34, 26, 948749,
> tzinfo=tzlocal()),
>  'href': '/pulp/api/v3/repositories/4/versions/1/',
>  'number': 1}
>
> Thoughts?
>
> [0] https://github.com/pulp/pulp/pull/3774
> [1] https://github.com/pulp/pulp_file/pull/133
>
>
>>
>>>
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Single-Table Content API Changes, Performance Discussion

[Dennis Kliban]

On Thu, Dec 6, 2018 at 6:04 PM Daniel Alley  wrote:

> I'm not necessarily against this but I'll recap some points I made on IRC:
>
> The burden of knowing where to go to get that information would be pushed
> off onto the API user.  If we're not returning the URL, then anyone using
> the API must know that they need to query /pulp/api/v3/content/file/files/
> (and likewise for every other content type), and that they need to use a
> filter for repository_version=... or repository_version_added=... and so so
> on.
>
> I'm not sure how that would work, how that knowledge would be provided or
> if it's something that can be hardcoded into the bindings.  If you think
> that's possible, then I'm open to it.
>
>
The bindings that get generated provide documentation about the available
filters. The following doc block is generated in the bindings right now:

def content_file_files_list(self, **kwargs):  # noqa: E501
"""content_file_files_list  # noqa: E501

ViewSet for FileContent.  # noqa: E501
This method makes a synchronous HTTP request by default. To make an
asynchronous HTTP request, please pass async=True
>>> thread = api.content_file_files_list(async=True)
>>> result = thread.get()

:param async bool
:param str relative_path: Filter results where relative_path
matches value
:param str digest: Filter results where digest matches value
:param str repository_version: Repository Version referenced by HREF
:param str repository_version_added: Repository Version referenced
by HREF
:param str repository_version_removed: Repository Version
referenced by HREF
:param int page: A page number within the paginated result set.
:param int page_size: Number of results to return per page.
:return: InlineResponse2001
 If the method is called asynchronously,
 returns the request thread.
"""

The bindings user would know that when listing all the file content that
she can provide keyword arguments to filter that content. Does that
alleviate the concern?


>
>
> On Thu, Dec 6, 2018 at 4:53 PM Dennis Kliban  wrote:
>
>> On Tue, Nov 20, 2018 at 12:31 PM Dennis Kliban 
>> wrote:
>>
>>> On Mon, Nov 19, 2018 at 6:20 PM Daniel Alley  wrote:
>>>
 Some of the API changes that are required by single-table-content would
 be beneficial even if we didn't go forwards with the modelling changes.
 For instance, currently we have single endpoints for each of
 repository_version/.../content/,  .../added_content/, and
 .../removed_content/ which mix content of all types together.  This makes
 it impossible for clients to expect the data returned to expect any
 particular schema.  What the single-table-content does is to provide
 separate query urls for each content type present in the repository
 version, which I believe is a usability win for us, and it's something we
 could implement without using any of the modelling changes.


>>> The current behavior of the 'content' APIs is already causing a problem
>>> for our OpenAPI 2.0 schema. OpenAPI 2.0 does not support polymorphic
>>> responses. We are currently tracking problem with a bug[0]. The only way to
>>> resolve this problem is to provide APIs that return heterogeneous types.
>>>
>>> [0] https://pulp.plan.io/issues/4052
>>>
>>>
 Besides being a general update, I'd like to start a discussion to
 understand:  is changing the Pulp 3 API so that it's organized around
 content type URLs OK with everyone? This resolves the usability issues of
 returning mixed types. Are there any downsides with this approach?

 To clarify what I mean on that last point -- by "content type URLs" I
 mean that where you currently get back the url "
 /pulp/api/v3/repository_version/.../content/" under the "_content"
 field on a repoversion, you would instead get back something like

 { "pulp_file.filecontent":
 "/pulp/api/v3/content/file/files/?repository_version=.. }

>>>
>>> I am +1 to making this change to our REST API.
>>>
>>
>> Thank you Daniel for putting together the patches[0,1] to make these
>> changes possible. I've had a chance to try out the Python bindings. When
>> using the bindings, I discovered that I could not do anything with the URLs
>> returned for each content type added or removed. Making the request to
>> those URLs requires making a call that looks like this:
>>
>>
>> api.content_file_files_list(repository_version_added=repository_version_1.href)
>>
>> What if instead the API returned the number of each content type added or
>> removed. So a repository version response would look like:
>>
>> {'base_version': None,
>>  'content_added': {'pulp_file.file': 4},
>>  'content_removed': {'pulp_file.file': 1},
>>  'content_summary': {'pulp_file.file': '3'},
>>  'created': datetime.datetime(2018, 12, 5, 23, 34, 26, 948749,
>> tzinfo=tzlocal(

Re: [Pulp-dev] Single-Table Content API Changes, Performance Discussion

[Jeff Ortel]

+1 to counts instead of URLs.  The URLs are documented and can be 
constructed to listing them on the serialized version does not seem to 
add much value.  The counts would likely provide more useful information 
and consistent with the summary counts.


On 12/7/18 1:30 PM, Dennis Kliban wrote:
What if instead the API returned the number of each content type added 
or removed. So a repository version response would look like:


{'base_version': None,
 'content_added': {'pulp_file.file': 4},
 'content_removed': {'pulp_file.file': 1},
 'content_summary': {'pulp_file.file': '3'},
 'created': datetime.datetime(2018, 12, 5, 23, 34, 26, 948749, 
tzinfo=tzlocal()),

 'href': '/pulp/api/v3/repositories/4/versions/1/',
 'number': 1}

Thoughts?


___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Single-Table Content API Changes, Performance Discussion

[Jeff Ortel]

On 12/10/18 1:06 PM, Jeff Ortel wrote:
+1 to counts instead of URLs.  The URLs are documented and can be 
constructed to listing them on the serialized version does not seem to 
add much value.  The counts would likely provide more useful 
information and consistent with the summary counts.


Just thought of something.  The URLs for specific content types are at 
the discretion of the plugin writer so now I'm not convinced the user 
has a way to reliably construct the URLs themselves.




On 12/7/18 1:30 PM, Dennis Kliban wrote:
What if instead the API returned the number of each content type 
added or removed. So a repository version response would look like:


{'base_version': None,
 'content_added': {'pulp_file.file': 4},
 'content_removed': {'pulp_file.file': 1},
 'content_summary': {'pulp_file.file': '3'},
 'created': datetime.datetime(2018, 12, 5, 23, 34, 26, 948749, 
tzinfo=tzlocal()),

 'href': '/pulp/api/v3/repositories/4/versions/1/',
 'number': 1}

Thoughts?




___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Single-Table Content API Changes, Performance Discussion

[Daniel Alley]

>
> Just thought of something.  The URLs for specific content types are at
> the discretion of the plugin writer so now I'm not convinced the user
> has a way to reliably construct the URLs themselves.


Yup, this is my view.  The counterargument Dennis has been making is that
the user could either A) use the live API docs to find out what URL to hit,
B) find it in the hosted docs, or C) use the bindings generated from the
schema, the name of the function is documented and you don't need to care
about the URL.

I suppose it depends on exactly how frequently users actually need to
view/search on the content present in a repository version.  If it's a rare
need, then maybe that extra friction is OK.  If it is common, it could be a
pain point -- or perhaps people will just memorize all the endpoints they
need to use and it won't be a big deal, I don't really know.


On Wed, Dec 12, 2018 at 1:14 PM Jeff Ortel  wrote:

> On 12/10/18 1:06 PM, Jeff Ortel wrote:
> > +1 to counts instead of URLs.  The URLs are documented and can be
> > constructed to listing them on the serialized version does not seem to
> > add much value.  The counts would likely provide more useful
> > information and consistent with the summary counts.
>
> Just thought of something.  The URLs for specific content types are at
> the discretion of the plugin writer so now I'm not convinced the user
> has a way to reliably construct the URLs themselves.
>
> >
> > On 12/7/18 1:30 PM, Dennis Kliban wrote:
> >> What if instead the API returned the number of each content type
> >> added or removed. So a repository version response would look like:
> >>
> >> {'base_version': None,
> >>  'content_added': {'pulp_file.file': 4},
> >>  'content_removed': {'pulp_file.file': 1},
> >>  'content_summary': {'pulp_file.file': '3'},
> >>  'created': datetime.datetime(2018, 12, 5, 23, 34, 26, 948749,
> >> tzinfo=tzlocal()),
> >>  'href': '/pulp/api/v3/repositories/4/versions/1/',
> >>  'number': 1}
> >>
> >> Thoughts?
> >
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Single-Table Content API Changes, Performance Discussion

[Brian Bouterse]

I commented on the PR that I think we should include the URLs and here's
the reasoning:
https://github.com/pulp/pulp/pull/3774#issuecomment-446633354

On Wed, Dec 12, 2018 at 5:29 PM Daniel Alley  wrote:

> Just thought of something.  The URLs for specific content types are at
>> the discretion of the plugin writer so now I'm not convinced the user
>> has a way to reliably construct the URLs themselves.
>
>
> Yup, this is my view.  The counterargument Dennis has been making is that
> the user could either A) use the live API docs to find out what URL to hit,
> B) find it in the hosted docs, or C) use the bindings generated from the
> schema, the name of the function is documented and you don't need to care
> about the URL.
>
> I suppose it depends on exactly how frequently users actually need to
> view/search on the content present in a repository version.  If it's a rare
> need, then maybe that extra friction is OK.  If it is common, it could be a
> pain point -- or perhaps people will just memorize all the endpoints they
> need to use and it won't be a big deal, I don't really know.
>
>
> On Wed, Dec 12, 2018 at 1:14 PM Jeff Ortel  wrote:
>
>> On 12/10/18 1:06 PM, Jeff Ortel wrote:
>> > +1 to counts instead of URLs.  The URLs are documented and can be
>> > constructed to listing them on the serialized version does not seem to
>> > add much value.  The counts would likely provide more useful
>> > information and consistent with the summary counts.
>>
>> Just thought of something.  The URLs for specific content types are at
>> the discretion of the plugin writer so now I'm not convinced the user
>> has a way to reliably construct the URLs themselves.
>>
>> >
>> > On 12/7/18 1:30 PM, Dennis Kliban wrote:
>> >> What if instead the API returned the number of each content type
>> >> added or removed. So a repository version response would look like:
>> >>
>> >> {'base_version': None,
>> >>  'content_added': {'pulp_file.file': 4},
>> >>  'content_removed': {'pulp_file.file': 1},
>> >>  'content_summary': {'pulp_file.file': '3'},
>> >>  'created': datetime.datetime(2018, 12, 5, 23, 34, 26, 948749,
>> >> tzinfo=tzlocal()),
>> >>  'href': '/pulp/api/v3/repositories/4/versions/1/',
>> >>  'number': 1}
>> >>
>> >> Thoughts?
>> >
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Single-Table Content API Changes, Performance Discussion

[Dana Walker]

I agree that the "human usable" route makes sense.

We want to make this as easy to use as possible.

--Dana

Dana Walker

Associate Software Engineer

Red Hat





On Wed, Dec 12, 2018 at 5:51 PM Brian Bouterse  wrote:

> I commented on the PR that I think we should include the URLs and here's
> the reasoning:
> https://github.com/pulp/pulp/pull/3774#issuecomment-446633354
>
> On Wed, Dec 12, 2018 at 5:29 PM Daniel Alley  wrote:
>
>> Just thought of something.  The URLs for specific content types are at
>>> the discretion of the plugin writer so now I'm not convinced the user
>>> has a way to reliably construct the URLs themselves.
>>
>>
>> Yup, this is my view.  The counterargument Dennis has been making is that
>> the user could either A) use the live API docs to find out what URL to hit,
>> B) find it in the hosted docs, or C) use the bindings generated from the
>> schema, the name of the function is documented and you don't need to care
>> about the URL.
>>
>> I suppose it depends on exactly how frequently users actually need to
>> view/search on the content present in a repository version.  If it's a rare
>> need, then maybe that extra friction is OK.  If it is common, it could be a
>> pain point -- or perhaps people will just memorize all the endpoints they
>> need to use and it won't be a big deal, I don't really know.
>>
>>
>> On Wed, Dec 12, 2018 at 1:14 PM Jeff Ortel  wrote:
>>
>>> On 12/10/18 1:06 PM, Jeff Ortel wrote:
>>> > +1 to counts instead of URLs.  The URLs are documented and can be
>>> > constructed to listing them on the serialized version does not seem to
>>> > add much value.  The counts would likely provide more useful
>>> > information and consistent with the summary counts.
>>>
>>> Just thought of something.  The URLs for specific content types are at
>>> the discretion of the plugin writer so now I'm not convinced the user
>>> has a way to reliably construct the URLs themselves.
>>>
>>> >
>>> > On 12/7/18 1:30 PM, Dennis Kliban wrote:
>>> >> What if instead the API returned the number of each content type
>>> >> added or removed. So a repository version response would look like:
>>> >>
>>> >> {'base_version': None,
>>> >>  'content_added': {'pulp_file.file': 4},
>>> >>  'content_removed': {'pulp_file.file': 1},
>>> >>  'content_summary': {'pulp_file.file': '3'},
>>> >>  'created': datetime.datetime(2018, 12, 5, 23, 34, 26, 948749,
>>> >> tzinfo=tzlocal()),
>>> >>  'href': '/pulp/api/v3/repositories/4/versions/1/',
>>> >>  'number': 1}
>>> >>
>>> >> Thoughts?
>>> >
>>>
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Single-Table Content API Changes, Performance Discussion

[David Davis]

How are these URLs being populated? What if the plugin writer puts her
content URLs at a different place like /pulp_rpm/content/packages or
something?

David


On Wed, Dec 12, 2018 at 8:38 PM Dana Walker  wrote:

> I agree that the "human usable" route makes sense.
>
> We want to make this as easy to use as possible.
>
> --Dana
>
> Dana Walker
>
> Associate Software Engineer
>
> Red Hat
>
> 
> 
>
>
> On Wed, Dec 12, 2018 at 5:51 PM Brian Bouterse 
> wrote:
>
>> I commented on the PR that I think we should include the URLs and here's
>> the reasoning:
>> https://github.com/pulp/pulp/pull/3774#issuecomment-446633354
>>
>> On Wed, Dec 12, 2018 at 5:29 PM Daniel Alley  wrote:
>>
>>> Just thought of something.  The URLs for specific content types are at
 the discretion of the plugin writer so now I'm not convinced the user
 has a way to reliably construct the URLs themselves.
>>>
>>>
>>> Yup, this is my view.  The counterargument Dennis has been making is
>>> that the user could either A) use the live API docs to find out what URL to
>>> hit, B) find it in the hosted docs, or C) use the bindings generated from
>>> the schema, the name of the function is documented and you don't need to
>>> care about the URL.
>>>
>>> I suppose it depends on exactly how frequently users actually need to
>>> view/search on the content present in a repository version.  If it's a rare
>>> need, then maybe that extra friction is OK.  If it is common, it could be a
>>> pain point -- or perhaps people will just memorize all the endpoints they
>>> need to use and it won't be a big deal, I don't really know.
>>>
>>>
>>> On Wed, Dec 12, 2018 at 1:14 PM Jeff Ortel  wrote:
>>>
 On 12/10/18 1:06 PM, Jeff Ortel wrote:
 > +1 to counts instead of URLs.  The URLs are documented and can be
 > constructed to listing them on the serialized version does not seem
 to
 > add much value.  The counts would likely provide more useful
 > information and consistent with the summary counts.

 Just thought of something.  The URLs for specific content types are at
 the discretion of the plugin writer so now I'm not convinced the user
 has a way to reliably construct the URLs themselves.

 >
 > On 12/7/18 1:30 PM, Dennis Kliban wrote:
 >> What if instead the API returned the number of each content type
 >> added or removed. So a repository version response would look like:
 >>
 >> {'base_version': None,
 >>  'content_added': {'pulp_file.file': 4},
 >>  'content_removed': {'pulp_file.file': 1},
 >>  'content_summary': {'pulp_file.file': '3'},
 >>  'created': datetime.datetime(2018, 12, 5, 23, 34, 26, 948749,
 >> tzinfo=tzlocal()),
 >>  'href': '/pulp/api/v3/repositories/4/versions/1/',
 >>  'number': 1}
 >>
 >> Thoughts?
 >

 ___
 Pulp-dev mailing list
 Pulp-dev@redhat.com
 https://www.redhat.com/mailman/listinfo/pulp-dev

>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Single-Table Content API Changes, Performance Discussion

[Dennis Kliban]

Let's continue with hte original proposal and keep the URLs.

On Wed, Dec 12, 2018 at 5:51 PM Brian Bouterse  wrote:

> I commented on the PR that I think we should include the URLs and here's
> the reasoning:
> https://github.com/pulp/pulp/pull/3774#issuecomment-446633354
>
> On Wed, Dec 12, 2018 at 5:29 PM Daniel Alley  wrote:
>
>> Just thought of something.  The URLs for specific content types are at
>>> the discretion of the plugin writer so now I'm not convinced the user
>>> has a way to reliably construct the URLs themselves.
>>
>>
>> Yup, this is my view.  The counterargument Dennis has been making is that
>> the user could either A) use the live API docs to find out what URL to hit,
>> B) find it in the hosted docs, or C) use the bindings generated from the
>> schema, the name of the function is documented and you don't need to care
>> about the URL.
>>
>> I suppose it depends on exactly how frequently users actually need to
>> view/search on the content present in a repository version.  If it's a rare
>> need, then maybe that extra friction is OK.  If it is common, it could be a
>> pain point -- or perhaps people will just memorize all the endpoints they
>> need to use and it won't be a big deal, I don't really know.
>>
>>
>> On Wed, Dec 12, 2018 at 1:14 PM Jeff Ortel  wrote:
>>
>>> On 12/10/18 1:06 PM, Jeff Ortel wrote:
>>> > +1 to counts instead of URLs.  The URLs are documented and can be
>>> > constructed to listing them on the serialized version does not seem to
>>> > add much value.  The counts would likely provide more useful
>>> > information and consistent with the summary counts.
>>>
>>> Just thought of something.  The URLs for specific content types are at
>>> the discretion of the plugin writer so now I'm not convinced the user
>>> has a way to reliably construct the URLs themselves.
>>>
>>> >
>>> > On 12/7/18 1:30 PM, Dennis Kliban wrote:
>>> >> What if instead the API returned the number of each content type
>>> >> added or removed. So a repository version response would look like:
>>> >>
>>> >> {'base_version': None,
>>> >>  'content_added': {'pulp_file.file': 4},
>>> >>  'content_removed': {'pulp_file.file': 1},
>>> >>  'content_summary': {'pulp_file.file': '3'},
>>> >>  'created': datetime.datetime(2018, 12, 5, 23, 34, 26, 948749,
>>> >> tzinfo=tzlocal()),
>>> >>  'href': '/pulp/api/v3/repositories/4/versions/1/',
>>> >>  'number': 1}
>>> >>
>>> >> Thoughts?
>>> >
>>>
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Single-Table Content API Changes, Performance Discussion

[Daniel Alley]

@David Davis  The URL generation code doesn't make any assumptions, it
looks up the view name for the model and then fetches the URL associated
with that view using reverse().  It doesn't matter where the plugin writer
puts their URLs.

@Dennis Do we still want to have counts for added and removed?  It might be
useful.

On Thu, Dec 13, 2018 at 8:18 AM Dennis Kliban  wrote:

> Let's continue with hte original proposal and keep the URLs.
>
> On Wed, Dec 12, 2018 at 5:51 PM Brian Bouterse 
> wrote:
>
>> I commented on the PR that I think we should include the URLs and here's
>> the reasoning:
>> https://github.com/pulp/pulp/pull/3774#issuecomment-446633354
>>
>> On Wed, Dec 12, 2018 at 5:29 PM Daniel Alley  wrote:
>>
>>> Just thought of something.  The URLs for specific content types are at
 the discretion of the plugin writer so now I'm not convinced the user
 has a way to reliably construct the URLs themselves.
>>>
>>>
>>> Yup, this is my view.  The counterargument Dennis has been making is
>>> that the user could either A) use the live API docs to find out what URL to
>>> hit, B) find it in the hosted docs, or C) use the bindings generated from
>>> the schema, the name of the function is documented and you don't need to
>>> care about the URL.
>>>
>>> I suppose it depends on exactly how frequently users actually need to
>>> view/search on the content present in a repository version.  If it's a rare
>>> need, then maybe that extra friction is OK.  If it is common, it could be a
>>> pain point -- or perhaps people will just memorize all the endpoints they
>>> need to use and it won't be a big deal, I don't really know.
>>>
>>>
>>> On Wed, Dec 12, 2018 at 1:14 PM Jeff Ortel  wrote:
>>>
 On 12/10/18 1:06 PM, Jeff Ortel wrote:
 > +1 to counts instead of URLs.  The URLs are documented and can be
 > constructed to listing them on the serialized version does not seem
 to
 > add much value.  The counts would likely provide more useful
 > information and consistent with the summary counts.

 Just thought of something.  The URLs for specific content types are at
 the discretion of the plugin writer so now I'm not convinced the user
 has a way to reliably construct the URLs themselves.

 >
 > On 12/7/18 1:30 PM, Dennis Kliban wrote:
 >> What if instead the API returned the number of each content type
 >> added or removed. So a repository version response would look like:
 >>
 >> {'base_version': None,
 >>  'content_added': {'pulp_file.file': 4},
 >>  'content_removed': {'pulp_file.file': 1},
 >>  'content_summary': {'pulp_file.file': '3'},
 >>  'created': datetime.datetime(2018, 12, 5, 23, 34, 26, 948749,
 >> tzinfo=tzlocal()),
 >>  'href': '/pulp/api/v3/repositories/4/versions/1/',
 >>  'number': 1}
 >>
 >> Thoughts?
 >

 ___
 Pulp-dev mailing list
 Pulp-dev@redhat.com
 https://www.redhat.com/mailman/listinfo/pulp-dev

>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev