Re: [Pulp-dev] creating repository version resources using a single REST endpoint

[Dennis Kliban]

On Mon, Jan 8, 2018 at 2:36 PM, David Davis  wrote:

> How would REST API users discover the possible values for ‘operation’? I
> guess we could put it in the help text for the field.
>
> I’m unsure of the value of having an operation param. I think I prefer the
> idea of just having users supply importer or add/remove_content (but not
> both) or having two separate endpoints.
>
>
The value is that this will be a single endpoint for creating a repository
version. The plugin writers will not need to add REST API endpoints when
implementing a complex manipulation of content in a repository version.
They will simply create a task and we'll dispatch this task when a user
requests to create a new repository version using this task.


>
> David
>
> On Mon, Jan 8, 2018 at 1:45 PM, Dennis Kliban  wrote:
>
>> Enable users to POST to /api/v3/repositories/123abc456/versions/ with
>> one required parameter 'operation'. This parameter would be an identifier
>> for a task Pulp would run to create a new version. Any additional
>> parameters passed in by the API user would be passed along to the task.
>>
>> pulpcore would provide the 'sync' task and the 'add_remove' task. 'sync'
>> would accept an 'importer'. 'add_remove' would accept 'remove_content' and
>> 'add_content'.
>>
>> Each plugin could provide any number of tasks for creating a repository
>> version.
>>
>> pulpcore would always create the new repository version, hand it to the
>> plugin code, and then mark it as complete after plugin code runs
>> successfully. Alleviating the plugin writer of these concern.
>>
>> REST API users would always use the same end point to create a repository
>> version. Plugin writers wouldn't have to worry about creating repository
>> versions and managing the 'complete' state.
>>
>> What do you all think?
>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] creating repository version resources using a single REST endpoint

[Dennis Kliban]

On Mon, Jan 8, 2018 at 2:39 PM, Austin Macdonald 
wrote:

> I like the concept of single REST endpoint that is responsible for all the
> ways to create a RepositoryVersion, but I don't quite understand how this
> would work. Since the endpoint is purely pulpcore, how can the
> RepositoryVersionViewSet import the plugin defined tasks that correspond to
> the action specified by the user? The only way I see is to force plugin
> writers to define all their tasks as methods on the Importer or Publisher,
> which brings us back to the circular import problem.
> https://pulp.plan.io/issues/3074
>
>
Plugin writers would need to define the tasks inside the tasks module of
their django app. pulpcore would then be able to discover the tasks defined
by the plugin at startup. The 'operation' could be name spaced by the
plugin name. Any tasks discovered in pulpcore would have pulpcore prepended
to the operation name. e.g.: pulpcore.sync or pulp_rpm.deep_copy

This would also address the circular import problem by moving the code that
performs a sync outside the Importer. However, this would require the
plugin writer to instantiate an Importer based on an 'href' passed in as an
argument. And only then could the importer be used to drive the API.


> Also, I think it would be a little unusual that the possible actions
> specified in the POST body to a pulpcore endpoint would vary depending on
> the plugin it is being used with. How would we document how to use this
> endpoint?
>
>
The endpoint would have a limited number of operations listed in our hosted
docs. However, the rest API docs on each Pulp installation should be able
to provide the user with a list of all available options.


> On Mon, Jan 8, 2018 at 1:45 PM, Dennis Kliban  wrote:
>
>> Enable users to POST to /api/v3/repositories/123abc456/versions/ with
>> one required parameter 'operation'. This parameter would be an identifier
>> for a task Pulp would run to create a new version. Any additional
>> parameters passed in by the API user would be passed along to the task.
>>
>> pulpcore would provide the 'sync' task and the 'add_remove' task. 'sync'
>> would accept an 'importer'. 'add_remove' would accept 'remove_content' and
>> 'add_content'.
>>
>> Each plugin could provide any number of tasks for creating a repository
>> version.
>>
>> pulpcore would always create the new repository version, hand it to the
>> plugin code, and then mark it as complete after plugin code runs
>> successfully. Alleviating the plugin writer of these concern.
>>
>> REST API users would always use the same end point to create a repository
>> version. Plugin writers wouldn't have to worry about creating repository
>> versions and managing the 'complete' state.
>>
>> What do you all think?
>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] creating repository version resources using a single REST endpoint

[David Davis]

How would REST API users discover the possible values for ‘operation’? I
guess we could put it in the help text for the field.

I’m unsure of the value of having an operation param. I think I prefer the
idea of just having users supply importer or add/remove_content (but not
both) or having two separate endpoints.


David

On Mon, Jan 8, 2018 at 1:45 PM, Dennis Kliban  wrote:

> Enable users to POST to /api/v3/repositories/123abc456/versions/ with one
> required parameter 'operation'. This parameter would be an identifier for a
> task Pulp would run to create a new version. Any additional parameters
> passed in by the API user would be passed along to the task.
>
> pulpcore would provide the 'sync' task and the 'add_remove' task. 'sync'
> would accept an 'importer'. 'add_remove' would accept 'remove_content' and
> 'add_content'.
>
> Each plugin could provide any number of tasks for creating a repository
> version.
>
> pulpcore would always create the new repository version, hand it to the
> plugin code, and then mark it as complete after plugin code runs
> successfully. Alleviating the plugin writer of these concern.
>
> REST API users would always use the same end point to create a repository
> version. Plugin writers wouldn't have to worry about creating repository
> versions and managing the 'complete' state.
>
> What do you all think?
>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] creating repository version resources using a single REST endpoint

[Austin Macdonald]

I like the concept of single REST endpoint that is responsible for all the
ways to create a RepositoryVersion, but I don't quite understand how this
would work. Since the endpoint is purely pulpcore, how can the
RepositoryVersionViewSet import the plugin defined tasks that correspond to
the action specified by the user? The only way I see is to force plugin
writers to define all their tasks as methods on the Importer or Publisher,
which brings us back to the circular import problem.
https://pulp.plan.io/issues/3074

Also, I think it would be a little unusual that the possible actions
specified in the POST body to a pulpcore endpoint would vary depending on
the plugin it is being used with. How would we document how to use this
endpoint?

On Mon, Jan 8, 2018 at 1:45 PM, Dennis Kliban  wrote:

> Enable users to POST to /api/v3/repositories/123abc456/versions/ with one
> required parameter 'operation'. This parameter would be an identifier for a
> task Pulp would run to create a new version. Any additional parameters
> passed in by the API user would be passed along to the task.
>
> pulpcore would provide the 'sync' task and the 'add_remove' task. 'sync'
> would accept an 'importer'. 'add_remove' would accept 'remove_content' and
> 'add_content'.
>
> Each plugin could provide any number of tasks for creating a repository
> version.
>
> pulpcore would always create the new repository version, hand it to the
> plugin code, and then mark it as complete after plugin code runs
> successfully. Alleviating the plugin writer of these concern.
>
> REST API users would always use the same end point to create a repository
> version. Plugin writers wouldn't have to worry about creating repository
> versions and managing the 'complete' state.
>
> What do you all think?
>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Stop distributing gofer

[Patrick Creech]

On Mon, 2018-01-08 at 09:08 -0600, Jeff Ortel wrote:
> 
> Thoughts?
> 
Sounds good from a releng pov for upstream, with the caveat of what to do about 
EL5?  We technically
still have client bits published for that distro.

signature.asc
Description: This is a digitally signed message part
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] creating repository version resources using a single REST endpoint

[Dennis Kliban]

Enable users to POST to /api/v3/repositories/123abc456/versions/ with one
required parameter 'operation'. This parameter would be an identifier for a
task Pulp would run to create a new version. Any additional parameters
passed in by the API user would be passed along to the task.

pulpcore would provide the 'sync' task and the 'add_remove' task. 'sync'
would accept an 'importer'. 'add_remove' would accept 'remove_content' and
'add_content'.

Each plugin could provide any number of tasks for creating a repository
version.

pulpcore would always create the new repository version, hand it to the
plugin code, and then mark it as complete after plugin code runs
successfully. Alleviating the plugin writer of these concern.

REST API users would always use the same end point to create a repository
version. Plugin writers wouldn't have to worry about creating repository
versions and managing the 'complete' state.

What do you all think?
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Github keywords

[Daniel Alley]

Even if we don't change this, It's something we should keep in mind since
our PR #s are currently in a spot where they may frequently overlap with
issue #s.  At some point they'll diverge again and it won't be so much of
an issue, but currently it is.

I don't know precisely how the redmine integration works, but perhaps we
could change the syntax to "closes redmine #"?

On Mon, Jan 8, 2018 at 10:41 AM, David Davis  wrote:

> Today, I had a PR[0] that was accidentally closed by another PR[1]. The
> issue is that the format we’re using attach our commits to redmine issues
> (e.g. 'fixes #1234’) is the exact same format as what Github uses[2].
>
> I’m wondering if we should change our format. I hate to do so but at the
> same time, I worry about PRs getting closed by accident in the future.
> Thoughts?
>
> [0] https://github.com/pulp/pulp/pull/3245
> [1] https://github.com/pulp/
> pulp/pull/3252
> [2] https://help.github.com/articles/closing-issues-using-keywords/
>
> David
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] 500 error during logrotate

[Jonathon Turel]

Hi team,

I'd like some input on this bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1512426

The httpd error log there shows what is wrong. I would have expected
the request to indeed be handled gracefully but that doesn't seem to
be the case.

Interestingly, I wasn't able to reproduce the problem with a different
API: /pulp/api/v2/users/ so maybe this problem is somehow specific to
publishing a repo with a distributor ID.

We're thinking of putting some across-the-board retry logic into
runcible to get around this, but obviously that isn't preferred. This
feels like a WSGI or Pulp problem.

Thanks for taking a look & let me know if I can provide more details.

Jonathon

(let me know if there's a better list for this discussion)

___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] Github keywords

[David Davis]

Today, I had a PR[0] that was accidentally closed by another PR[1]. The
issue is that the format we’re using attach our commits to redmine issues
(e.g. 'fixes #1234’) is the exact same format as what Github uses[2].

I’m wondering if we should change our format. I hate to do so but at the
same time, I worry about PRs getting closed by accident in the future.
Thoughts?

[0] https://github.com/pulp/pulp/pull/3245
[1]
https://github.com/pulp/pulp/pull/3252
[2] https://help.github.com/articles/closing-issues-using-keywords/

David
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] Stop distributing gofer

[Jeff Ortel]

The gofer package is distributed in Fedora and Copr[1] for EL6 & EL7.  
Gofer is an external (to the project) dependency much like Celery.  I 
propose we stop distributing gofer and update our documentation to refer 
users to the existing sources.


Thoughts?


[1] https://copr.fedorainfracloud.org/coprs/jortel/gofer/
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev