Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

I have updated #3033 with my understanding of the consensus reached on this thread. Please review and continue any outstanding discussion by posting comments. https://pulp.plan.io/issues/3033 signature.asc Description: OpenPGP digital signature ___

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On 11/02/2017 11:48 AM, Brian Bouterse wrote: > Yes what you've written is the idea. An alternative implementation would be > for the option field to be json > field which postgres supports now, but either way works. Those 'options' are > used during publishing as a > mechanism to allow arbitrary

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

Yes what you've written is the idea. An alternative implementation would be for the option field to be json field which postgres supports now, but either way works. Those 'options' are used during publishing as a mechanism to allow arbitrary options to be passed to the the publisher, and later they

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On 11/01/2017 12:52 PM, Brian Bouterse wrote: > +1 exactly to what @daviddavis said about 202 because normal Okay. I'm sold on this point. > > I also think we can support one-time publish params w/ their call to POST > /.../publication/. Assuming we want > to store those options to recalled

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

+1 exactly to what @daviddavis said about 202 because normal I also think we can support one-time publish params w/ their call to POST /.../publication/. Assuming we want to store those options to recalled (but not used) later, submitting that data to the publication endpoint is probably the best

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

Calling a create endpoint and getting back a 202 with a way to monitor the status of the request is pretty common practice in REST APIs[0]. I think it would be intuitive to users familiar with REST APIs that have async operations. I think we could support allowing users to submit one-time publish

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On 11/01/2017 09:16 AM, Brian Bouterse wrote: > Thanks for the response. Let's not move forward until we have more agreement > in this area. I've written some > responses inline. > > On Wed, Nov 1, 2017 at 9:05 AM, Jeff Ortel > wrote: > > I'm not yet convinced ab

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

Thanks for the response. Let's not move forward until we have more agreement in this area. I've written some responses inline. On Wed, Nov 1, 2017 at 9:05 AM, Jeff Ortel wrote: > I'm not yet convinced about the proposed URL change for publishing. Can > you help me understand why a POST to > the

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

I'm not yet convinced about the proposed URL change for publishing. Can you help me understand why a POST to the publications collection is more appropriate than the a POST to a publisher? A POST to the publications/ collection means the POST body should define the publication to be created. Ri

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

@dkliban, I'm +1 on that. @all, Please jump in if this is not the best direction for us to go. On Tue, Oct 31, 2017 at 3:55 PM, Dennis Kliban wrote: > On Tue, Oct 31, 2017 at 3:52 PM, Brian Bouterse > wrote: > >> Would that return the 202 w/ a link to the task because the publication >> hasn't

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On Tue, Oct 31, 2017 at 3:52 PM, Brian Bouterse wrote: > Would that return the 202 w/ a link to the task because the publication > hasn't been created yet? Then using the created_resources they can see what > was created, and in the event of failure the task fails and there are no > created_resou

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

Would that return the 202 w/ a link to the task because the publication hasn't been created yet? Then using the created_resources they can see what was created, and in the event of failure the task fails and there are no created_resources. @dkliban is ^ the idea? On Tue, Oct 31, 2017 at 3:48 PM,

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On Tue, Oct 31, 2017 at 3:40 PM, Brian Bouterse wrote: > +1 to updating #3033 to have a created_resources attribute which would be > a list of GenericForeignKeys. It also needs docs, but I'm not entirely sure > where. > > If we're going to introduce the above attribute, I think having the > contr

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

+1 to updating #3033 to have a created_resources attribute which would be a list of GenericForeignKeys. It also needs docs, but I'm not entirely sure where. If we're going to introduce the above attribute, I think having the controller endpoint as-is would be the most usable. @dkliban do you see v

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

Personally I am not opposed to the url endpoint you suggest. It also seems like there is some consensus around adding a ‘created resources’ relationship to Task or at least prototyping that out to see what it would look like. If no one disagrees, should I update issue #3033 with those two items?

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On Wed, Oct 25, 2017 at 11:24 AM, David Davis wrote: > I don’t know that the ambiguity around whether a task has a publication or > not is a big deal. If I call the publication endpoint, I’d expect a > publication task which either has 1 publication or 0 (if the publication > failed) attached to

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

Perhaps the ambiguity problem isn't a big deal. Let's continue to explore adding "generic" links to the task status. I wonder what a link or a HAL would look like w/ the swagger coreAPI. What does that look like? A proof of concept of this would be great. What is the user experience like for coreAP

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On 10/25/2017 10:24 AM, David Davis wrote: > I don’t know that the ambiguity around whether a task has a publication or > not is a big deal. If I call the > publication endpoint, I’d expect a publication task which either has 1 > publication or 0 (if the publication > failed) attached to it. >

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

I don’t know that the ambiguity around whether a task has a publication or not is a big deal. If I call the publication endpoint, I’d expect a publication task which either has 1 publication or 0 (if the publication failed) attached to it. In terms of ambiguity, I see a worse problem around adding

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On Tue, Oct 24, 2017 at 10:00 PM, Michael Hrivnak wrote: > > > On Tue, Oct 24, 2017 at 2:11 PM, Brian Bouterse > wrote: > >> Thanks everyone for all the discussion! I'll try to recap the problem and >> some of the solutions I've heard. I'll also share some of my perspective on >> them too. >> >>

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On Tue, Oct 24, 2017 at 2:11 PM, Brian Bouterse wrote: > Thanks everyone for all the discussion! I'll try to recap the problem and > some of the solutions I've heard. I'll also share some of my perspective on > them too. > > What problem are we solving? > When a user calls "publish" (the action A

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

I was just reviewing the Task search API in Pulp 3 we designed a few months ago. Two of the requirements [0] were "As a user of the task search API I want to search for all tasks that operated on repo zoo” and "As a user of the task search API I want to search all publish tasks performed by a parti

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

> 2) Have the user find the publication via query that sorts on time and filters only for a specific publisher. This could be fragile because with a multi-user system and no hard references between publications and tasks, answering the question "which is the publication for me" is hard because anot

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

Thanks everyone for all the discussion! I'll try to recap the problem and some of the solutions I've heard. I'll also share some of my perspective on them too. What problem are we solving? When a user calls "publish" (the action API endpoint) they get a 202 w/ a link to the task. That task will pr

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

> > As an example, HAL is one common way to do this: > > http://stateless.co/hal_specification.html HAL looks neat. It's a reasonably well defined standard for defining HTTP responses, in much the same vein as JSON schema. If it's adopted, the one thing I'd suggest is to avoid the use of CURIEs.

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On 10/23/2017 06:14 PM, Dennis Kliban wrote: > On Mon, Oct 23, 2017 at 3:20 PM, Michael Hrivnak > wrote: > > > > On Mon, Oct 23, 2017 at 12:30 PM, Dennis Kliban > wrote: > > On Mon, Oct 23, 2017 at 10:56 AM, Jeff Ortel

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On Mon, Oct 23, 2017 at 3:20 PM, Michael Hrivnak wrote: > > > On Mon, Oct 23, 2017 at 12:30 PM, Dennis Kliban > wrote: > >> On Mon, Oct 23, 2017 at 10:56 AM, Jeff Ortel wrote: >> >>> This is interesting. >>> >>> Some thoughts: >>> >>> If adopted, I propose the publication task create the public

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

> > I also think it's valuable to let users name their publications. However, we should avoid forcing users to > > form URLs to resources on their own. Jeremy put it well in his response. > > I was not aware that was suggested. I think I just read things the wrong way. Sorry for the noise. ___

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On 10/23/2017 10:50 AM, Dennis Kliban wrote: > > We should probably reevaluate the value of 'auto_update' on a Distribution. > Information about distributions > that need to be updated can be passed in the body of the POST to > publications/. This way the user explicitly > instructs Pulp to pe

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On 10/23/2017 11:30 AM, Dennis Kliban wrote: > That is exactly what I had in mind. Though the field can be NULL if the task > has been removed from the > database already. This way a serialized version of a Publication would > provide a reference to a task that can > be tracked to see if the pu

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On Mon, Oct 23, 2017 at 12:30 PM, Dennis Kliban wrote: > On Mon, Oct 23, 2017 at 10:56 AM, Jeff Ortel wrote: > >> This is interesting. >> >> Some thoughts: >> >> If adopted, I propose the publication task create the publication and >> pass to the publisher which would >> require a change in the

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

Agreed on publications not having 'natural key's for the same reasons you outline. The tasking system has a progress reporting feature to report progress, but it's not a place for general reporting. The reasoning is outlined in the notes from @ichimonji10 and myself earlier. >From a high level, w

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On Mon, Oct 23, 2017 at 11:43 AM, Brian Bouterse wrote: > There is a lot of good discussion here. See inline about the motivation to > not add additional fields to the task model. > > On Mon, Oct 23, 2017 at 10:55 AM, Michael Hrivnak > wrote: > >> Unless the publication can be created before the

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On Mon, Oct 23, 2017 at 11:19 AM, Jeremy Audet wrote: > > Once the task has completed, the user should be free to forget the task > ID and be able to use natural keys to find and inspect resources that got > created/updated. > > I sure hope we won't be asking the user to build URLs themselves. Th

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On Mon, Oct 23, 2017 at 11:06 AM, Jeff Ortel wrote: > > On 10/23/2017 09:55 AM, Michael Hrivnak wrote: > > > > A task status should not include an exhaustive list of every resource > created. For example, a publish task > > should not include a reference to every metadata artifact it made. It > w

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On Mon, Oct 23, 2017 at 10:56 AM, Jeff Ortel wrote: > This is interesting. > > Some thoughts: > > If adopted, I propose the publication task create the publication and pass > to the publisher which would > require a change in the plugin API - Publisher.publish(publication). If > the publisher fa

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On Mon, Oct 23, 2017 at 10:55 AM, Michael Hrivnak wrote: > Unless the publication can be created before the response is returned, the > response code will need to still be 202. > > As for the path, either way seems workable, although I have two > hesitations about POSTing to publications/. > > 1)

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

There is a lot of good discussion here. See inline about the motivation to not add additional fields to the task model. On Mon, Oct 23, 2017 at 10:55 AM, Michael Hrivnak wrote: > Unless the publication can be created before the response is returned, the > response code will need to still be 202.

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

> > Normally in REST when a user creates a resource via POST to a collection >> endpoint, they are expected to provide a representation of the new >> resource, even if it is only partial. >> > >> > No, not true. > Sorry, Michael. What you said *is* true. I'm just aiming to add some additional dept

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

> Once the task has completed, the user should be free to forget the task ID and be able to use natural keys to find and inspect resources that got created/updated. I sure hope we won't be asking the user to build URLs themselves. That promotes a much tighter coupling between clients and API versi

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

> 'example' is an identifier for the publisher type So you might have two "rpm" publishers, for example? OK. > Normally in REST when a user creates a resource via POST to a collection endpoint, they are expected to provide a representation of the new resource, even if it is only partial. No, not

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On 10/19/2017 02:27 PM, Dennis Kliban wrote: > > This work would probably be done by whoever picks up issue 3033[1]. If adopted, I think this should be a separate story. > > [0] https://pulp.plan.io/issues/3035 > [1] https://pulp.plan.io/issues/3033 signature.asc Description: OpenPGP digita

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On 10/23/2017 09:55 AM, Michael Hrivnak wrote: > > A task status should not include an exhaustive list of every resource > created. For example, a publish task > should not include a reference to every metadata artifact it made. It would > be sufficient to include a > reference to the publicati

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

On Mon, Oct 23, 2017 at 10:40 AM, Jeremy Audet wrote: > > http://localhost:8000/api/v3/repositories/foo/publishers/exa > mple/bar/publications/ > > > > AFAIK, the form of this URL is: scheme://netloc/api/v3/ > re

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

This is interesting. Some thoughts: If adopted, I propose the publication task create the publication and pass to the publisher which would require a change in the plugin API - Publisher.publish(publication). If the publisher fails, I think the publication should be deleted. On 10/19/2017 02:

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

Unless the publication can be created before the response is returned, the response code will need to still be 202. As for the path, either way seems workable, although I have two hesitations about POSTing to publications/. 1) Normally in REST when a user creates a resource via POST to a collecti

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

> http://localhost:8000/api/v3/repositories/foo/publishers/ example/bar/publications/ AFAIK, the form of this URL is: scheme://netloc/api/v3/repositories/{repository_id}/publishers/example/{publisher_id}/publicati

Re: [Pulp-dev] [pulp 3] proposed change to publishing REST api

That seems sensible, and in line with REST's mantra of "nouns in resource URLs, not verbs". On Thu, Oct 19, 2017 at 3:27 PM, Dennis Kliban wrote: > @jortel and I have been discussing[0] how a user should find out what > publication was created after a request is made to > http://localhost:8000/a

[Pulp-dev] [pulp 3] proposed change to publishing REST api

@jortel and I have been discussing[0] how a user should find out what publication was created after a request is made to http://localhost:8000/api/v3/repositories/foo/publishers/example/bar/publish/ I propose that we get rid of the above URL from our REST API and add ability to POST to http://loca