On 11/01/2017 12:52 PM, Brian Bouterse wrote: > +1 exactly to what @daviddavis said about 202 because normal
Okay. I'm sold on this point.
>
> I also think we can support one-time publish params w/ their call to POST
> /.../publication/. Assuming we want
> to store those options to recalled (but not used) later, submitting that data
> to the publication endpoint is
> probably the best option. It won't work if we try to store the one-time
> options from many publishes on a
> single publisher object, but we can store each one-time set of options on
> their respective publication
> objects. I think we do want to store them because there is likely a use case
> where 'as a user I can recall the
> one-time options used for any given publication.'
Interesting.
Is the thinking that options would be passed in the POST body as part of the
publication document?
{
"options": {
"incremental": true,
}
}
And stored on the publication using generic key/value table like?
Publication(Model):
...
options (GenericKeyValueRelation): Publisher options.
...
>
> -Brian
>
>
>
> On Wed, Nov 1, 2017 at 1:20 PM, David Davis <[email protected]
> <mailto:[email protected]>> wrote:
>
> Calling a create endpoint and getting back a 202 with a way to monitor
> the status of the request is pretty
> common practice in REST APIs[0]. I think it would be intuitive to users
> familiar with REST APIs that have
> async operations.
>
> I think we could support allowing users to submit one-time publish params
> with their call to POST
> /../publications. It’s not great option but it’s an option.
>
> [0] http://jsonapi.org/format/#crud-creating-responses-202
> <http://jsonapi.org/format/#crud-creating-responses-202>
>
>
> David
>
> On Wed, Nov 1, 2017 at 10:58 AM, Jeff Ortel <[email protected]
> <mailto:[email protected]>> wrote:
>
>
>
> On 11/01/2017 09:16 AM, Brian Bouterse wrote:
> > Thanks for the response. Let's not move forward until we have more
> agreement in this area. I've written some
> > responses inline.
> >
> > On Wed, Nov 1, 2017 at 9:05 AM, Jeff Ortel <[email protected]
> <mailto:[email protected]> <mailto:[email protected]
> <mailto:[email protected]>>> wrote:
> >
> > I'm not yet convinced about the proposed URL change for
> publishing. Can you help me understand why a POST to
> > the publications collection is more appropriate than the a POST
> to a publisher?
> >
> >
> > I believe the thinking is: REST suggests that POSTing to a resource
> is expected to create a new resource of
> > that type. So assume a users knows REST and they know they want to
> get a Publication created in Pulp, they
> > know exactly how to do that just by knowing REST. In the case of a
> POST to a publisher url with a special
> > 'publish' keyword on the end of it (the controller endpoint), they
> only way a user could know to do that is by
> > reading our docs. Both approaches would work, but I believe the
> former is more aligned with REST which means
> > users can do more without having to read Pulp docs.
>
> I believe that if the user is experienced with REST, they will be
> expecting a 201 and an href to the
> created
> resource to be returned instead of a 202 and a task href. Further,
> they will expect the resource to be
> created as defined in the POST body. I think the side-effect of
> running the publisher to actually
> create the
> resource will be unexpected. The user would still need to read the
> docs to understand what's
> happening. Not
> saying this is all together wrong, just challenging the assertion
> that this would be more intuitive to
> the user.
>
> >
> >
> >
> > A POST to the publications/ collection means the POST body
> should define the publication to be created.
> > Right? What about options that need to be passed to the
> publisher?
> >
> >
> > Yes, if we look at the fields of the publisher (link below), there
> are only two fields: 'created' and
> > 'publisher'. Since 'created' is set on the server automatically,
> the user would specify only the href to the
> > publisher in the POST body. For the MVP, we don't accept one-time
> options, and all other options are
> > configured on the publisher which is a different url call from both
> the publish controller and the publication
> > resource. So for the MVP this approach would work well. The future
> case also is better with this approach (I
> > think). When we do introduce one-time options, where will we store
> them?
>
> Options are not stored. That's the point of them being one-time
> options vs. publisher attributes. My
> concern
> is that making this choice means that we will never support one-time
> publishing options. What do
> others think
> of that?
>
> We will probably be store them on the
> > publication too, and that makes sense, because we can't store N,
> one-time publish options on 1 publisher
> > instance, but we can store N, one-time publish options on N
> publications.
> >
> >
> https://github.com/pulp/pulp/blob/15857fb0831c0998219a32e8d6ba52abdba20888/platform/pulpcore/app/models/publication.py#L6
>
> <https://github.com/pulp/pulp/blob/15857fb0831c0998219a32e8d6ba52abdba20888/platform/pulpcore/app/models/publication.py#L6>
> >
> >
> >
> > On 10/31/2017 03:13 PM, Brian Bouterse wrote:
> > > @dkliban, I'm +1 on that.
> > >
> > > @all, Please jump in if this is not the best direction for us
> to go.
> > >
> > > On Tue, Oct 31, 2017 at 3:55 PM, Dennis Kliban
> <[email protected] <mailto:[email protected]> <mailto:[email protected]
> <mailto:[email protected]>>
> > <mailto:[email protected] <mailto:[email protected]>
> <mailto:[email protected]
> <mailto:[email protected]>>>> wrote:
> > >
> > > On Tue, Oct 31, 2017 at 3:52 PM, Brian Bouterse
> <[email protected]
> <mailto:[email protected]> <mailto:[email protected]
> <mailto:[email protected]>>
> <mailto:[email protected] <mailto:[email protected]>
> > <mailto:[email protected] <mailto:[email protected]>>>>
> wrote:
> > >
> > > Would that return the 202 w/ a link to the task
> because the publication hasn't been created yet? Then
> > > using the created_resources they can see what was
> created, and in the event of failure the task fails
> > > and there are no created_resources.
> > >
> > > @dkliban is ^ the idea?
> > >
> > >
> > > Yes, the response would the same as it for the /publish
> URL right now. This is just a change in the URL
> > > that is used to make the request.
> > >
> > >
> > >
> > > On Tue, Oct 31, 2017 at 3:48 PM, Dennis Kliban
> <[email protected] <mailto:[email protected]> <mailto:[email protected]
> <mailto:[email protected]>>
> > <mailto:[email protected] <mailto:[email protected]>
> <mailto:[email protected]
> <mailto:[email protected]>>>> wrote:
> > >
> > >
> > >
> > > On Tue, Oct 31, 2017 at 3:40 PM, Brian Bouterse
> <[email protected] <mailto:[email protected]>
> > <mailto:[email protected] <mailto:[email protected]>>
> <mailto:[email protected]
> <mailto:[email protected]> <mailto:[email protected]
> <mailto:[email protected]>>>>
> > > wrote:
> > >
> > > +1 to updating #3033 to have a
> created_resources attribute which would be a list of
> > > GenericForeignKeys. It also needs docs, but
> I'm not entirely sure where.
> > >
> > > If we're going to introduce the above
> attribute, I think having the controller endpoint as-is
> > > would be the most usable. @dkliban do you see
> value in changing the URL structure if the
> > > created_resources attribute is introduced?
> > >
> > >
> > > This API call creates a publication resource. A
> POST to publishers/<id>/publications/ seems most
> > > appropriate for creating new publication
> resources.
> > >
> > > I can help review/groom these if that is
> helpful.
> > >
> > > -Brian
> > >
> > >
> > > On Tue, Oct 31, 2017 at 1:39 PM, David Davis
> <[email protected] <mailto:[email protected]>
> <mailto:[email protected]
> <mailto:[email protected]>>
> > > <mailto:[email protected]
> <mailto:[email protected]> <mailto:[email protected]
> <mailto:[email protected]>>>> wrote:
> > >
> > > Personally I am not opposed to the url
> endpoint you suggest.
> > >
> > > It also seems like there is some
> consensus around adding a ‘created resources’
> > > relationship to Task or at least
> prototyping that out to see what it would look like.
> > >
> > > If no one disagrees, should I update
> issue #3033 with those two items?
> > >
> > >
> > > David
> > >
> > > On Wed, Oct 25, 2017 at 1:23 PM, Dennis
> Kliban <[email protected] <mailto:[email protected]>
> <mailto:[email protected] <mailto:[email protected]>>
> > > <mailto:[email protected]
> <mailto:[email protected]> <mailto:[email protected]
> <mailto:[email protected]>>>> wrote:
> > >
> > > On Wed, Oct 25, 2017 at 11:24 AM,
> David Davis <[email protected] <mailto:[email protected]>
> <mailto:[email protected]
> <mailto:[email protected]>>
> > > <mailto:[email protected]
> <mailto:[email protected]> <mailto:[email protected]
> <mailto:[email protected]>>>> wrote:
> > >
> > > I don’t know that the ambiguity
> around whether a task has a publication or not is
> > > a big deal. If I call the
> publication endpoint, I’d expect a publication task
> > > which either has 1 publication or
> 0 (if the publication failed) attached to it.
> > >
> > > In terms of ambiguity, I see a
> worse problem around adding a task_id field to
> > > publications. As a user, I don’t
> know if a publication failed or not when I get
> > > back a publication object.
> Instead, I have to look up the task to see if it is a
> > > real (or successful) publication.
> Moreover, since we allow users to remove/clean
> > > up tasks, that task may not even
> exist anymore.
> > >
> > >
> > > I agree that the ephemeral nature of
> tasks makes the originally proposed solution
> > > non-deterministic. I am open to
> associating 'resources created' with a task instead.
> > >
> > > However, I still think there is value
> in changing the rest API endpoint for starting a
> > > publish task to POST
> > >
> /api/v3/repositories/<repo-id>/publishers/<type>/<name>/publications/.
> However, I will
> > > start a separate thread for that
> discussion.
> > >
> > > - Dennis
> > >
> > >
> > >
> > > David
> > >
> > > On Wed, Oct 25, 2017 at 11:03 AM,
> Brian Bouterse <[email protected] <mailto:[email protected]>
> <mailto:[email protected] <mailto:[email protected]>>
> > > <mailto:[email protected]
> <mailto:[email protected]> <mailto:[email protected]
> <mailto:[email protected]>>>> wrote:
> > >
> > >
> > >
> > > On Tue, Oct 24, 2017 at 10:00
> PM, Michael Hrivnak <[email protected] <mailto:[email protected]>
> <mailto:[email protected] <mailto:[email protected]>>
> > > <mailto:[email protected]
> <mailto:[email protected]> <mailto:[email protected]
> <mailto:[email protected]>>>> wrote:
> > >
> > >
> > >
> > > On Tue, Oct 24, 2017 at
> 2:11 PM, Brian Bouterse <[email protected] <mailto:[email protected]>
> <mailto:[email protected] <mailto:[email protected]>>
> > >
> <mailto:[email protected] <mailto:[email protected]>
> <mailto:[email protected] <mailto:[email protected]>>>> wrote:
> > >
> > > Thanks everyone for
> all the discussion! I'll try to
> recap the
> > problem
> > > and some of the
> solutions I've heard. I'll also share
> some of my
> > > perspective on them
> too.
> > >
> > > What problem are we
> solving?
> > > When a user calls
> "publish" (the action API endpoint)
> they get a 202
> > > w/ a link to the
> task. That task will produce a
> publication. How can
> > > the user find the
> publication that was produced by the
> task? How can
> > > the user be sure the
> publication is fully complete?
> > >
> > >
> > > What are our options?
> > > 1) Start linking to
> created objects from task status.
> I believe its
> > > been clearly stated
> about why we can't do this. If
> it's not
> > clear, or
> > > if there are other
> things we should consider, let's
> talk about it.
> > > Acknowledging or
> establishing agreement on this is
> crucial because a
> > > change like this
> would bring back a lot of the user
> pain from
> > pulp2. I
> > > believe the HAL
> suggestion falls into this area.
> > >
> > >
> > > I may have missed
> something, but I do not think this is
> clear. I
> > know that
> > > Pulp 2's API included a
> lot of unstructured data, but that
> is not at all
> > > what I'm suggesting here.
> > >
> > > It is standard and
> recommended practice for REST API
> responses to
> > include
> > > links to resources along
> with information about what type of
> > resource each
> > > link references. We could
> include a reference to the created
> > resource and
> > > an identifier for what
> type of resource it is, and that
> would be well
> > > within the bounds of good
> REST API design. HAL is just one of
> > several ways
> > > to accomplish that, and
> I'm not pitching any particular
> solution
> > there. In
> > > any case, I'm not sure
> what the problem would be with this
> approach.
> > >
> > >
> > > I agree it is a standard
> practice for a resource to include
> links to other
> > > resources, but the proposal
> is to include "generic" links is
> different and
> > > creates a different user
> experience. I believe referencing the
> task from the
> > > publication will be easier
> for users and clients. When a user
> looks up a
> > > publication, they will always
> know they'll get between 0 and 1
> links to a
> > > task. You can use that to
> check the state of the publication.
> If we link to
> > > "generic" resources (like a
> publication) from a task, then if
> I ask a
> > user "do
> > > you expect task
> ede3af3e-d5cf-4e18-8c57-69ac4d4e4de6 to
> contain a link to a
> > > publication or not?" you
> can't know until you query it. I
> think that
> > ambiguity
> > > was a pain point in Pulp2. I
> don't totally reject this
> solution, but this is
> > > an undesirable property (I
> think).
> > >
> > >
> > >
> > >
> > > 2) Have the user find
> the publication via query that
> sorts on
> > time and
> > > filters only for a
> specific publisher. This could be
> fragile because
> > > with a multi-user
> system and no hard references
> between publications
> > > and tasks, answering
> the question "which is the
> publication for
> > me" is
> > > hard because another
> user could have submitted a
> publish too. While
> > > not totally perfect,
> this could work.
> > >
> > >
> > > In theory if a user
> queried for a publication from a
> specific publisher
> > > that was created between
> the start and end times of the
> task, that
> > should
> > > unambiguously identify
> the correct publication. But
> depending on
> > > timestamps is not a
> particularly robust nor
> confidence-inspiring way to
> > > reference a resource.
> > >
> > > Agreed and Agreed
> > >
> > >
> > >
> > >
> > > 3) Have the user
> create a publication directly like
> any other REST
> > > resource, and help
> the user understand the state of that
> > resource over
> > > time. I believe the
> proposal at the start of this
> thread is
> > > recommending this
> solution. I'm also +1 on this solution.
> > >
> > >
> > > I think the problem with
> this is that a user cannot create a
> > publication.
> > > A user can only ask a
> plugin to create a publication.
> Until the plugin
> > > creates the publication,
> there is no publication.
> > >
> > >
> > > Note a publication is an
> object, but really we mean a
> publication and it's
> > > related PublishedArtifact,
> PublishedMetadat, etc objects. It
> would be
> > > straightforward for a user to
> create a publication using the
> viewset and
> > have
> > > the task associated with it
> call the publisher to build out
> the associated
> > > PublishedArtifact,
> PublishedContent, PublishedMetadata, etc.
> We should
> > explore
> > > if this is good or not, but
> it is possible.
> > >
> > > As an aside, this is related
> to a problem everyone should be
> aware of: the
> > > existence of a publication
> does not guarantee that publication
> is finished
> > > publishing. Even with option
> 1, where the task creates the
> publisher and
> > links
> > > to it in the task status,
> while the publisher is running it
> must save the
> > > Publication so that the
> PublishedArtifact, etc can link to it.
> So for any
> > > given publication, in order
> to know if it's "fully finished
> and consistent"
> > > you must be able to check the
> status of the associated task that
> > produced it.
> > >
> > >
> > >
> > > As an aside, I don't
> think considering versioned repos
> as a possible
> > > solution is helping
> us with this problem. The scope of
> the current
> > > problem is relatively
> small and the scope of planning
> for versioned
> > > repos is large.
> > >
> > >
> > > Versioned repos is a
> potential solution. In that scenario,
> a user would
> > > request publication of a
> specific repo version (perhaps
> defaulting
> > to the
> > > latest), the publication
> would be linked to that version,
> and that is an
> > > easy mechanism for the
> user to find the publication they want.
> > Ultimately
> > > the user is interested in
> working with a specific content set
> > anyway. They
> > > get a repo to a state
> where it has the content they want,
> and then they
> > > publish that content set.
> No matter what we do with
> publications, users
> > > will think of them in
> terms of related content sets. A
> repo version is
> > > that immutable content
> set they can work with confidently.
> > >
> > >
> > > It's neat to me that that
> versions are snapshots of content
> and publications
> > > are snapshots of content.
> Publications already create much of
> the value
> > > propostion of versioned repos
> with publications. They allow
> you to work with
> > > specific content sets like
> you describe. Also they allow for
> rollback.
> > So that
> > > is all great for our users.
> For this thread, I want to bring the
> > conversation
> > > back to where it started,
> solving a small problem about
> linking two
> > resources
> > > that already exist.
> > >
> > >
> > > It helps the rollback
> scenario a lot as well. Versioning
> repos allows a
> > > user to see what the
> differences are between two content
> sets, and thus
> > > two different
> publications, which informs them about when
> and how
> > far back
> > > they should roll back a
> distribution.
> > >
> > >
> > > - user discovers a
> horrible flaw in a piece of content
> > > - user queries for which
> version of the repo introduced
> that piece
> > of content
> > > - user updates the
> distribution to serve the publication
> that came
> > before
> > > the one which introduced
> the piece of content, optionally
> re-publishing
> > > that version in case its
> publication was deleted or had
> never been
> > made in
> > > the first place.
> > >
> > > --
> > >
> > > Michael Hrivnak
> > >
> > > Principal Software
> Engineer, RHCE
> > >
> > > Red Hat
> > >
> > >
> > >
> > >
> _______________________________________________
> > > Pulp-dev mailing list
> > > [email protected]
> <mailto:[email protected]>
> <mailto:[email protected] <mailto:[email protected]>>
> > <mailto:[email protected] <mailto:[email protected]>
> <mailto:[email protected]
> <mailto:[email protected]>>>
> > >
> https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>
> > <https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>>
> > >
> <https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>
> <https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>>>
> > >
> > >
> > >
> > >
> _______________________________________________
> > > Pulp-dev mailing list
> > > [email protected]
> <mailto:[email protected]>
> <mailto:[email protected] <mailto:[email protected]>>
> <mailto:[email protected]
> <mailto:[email protected]>
> > <mailto:[email protected] <mailto:[email protected]>>>
> > >
> https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>
> > <https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>>
> > >
> <https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>
> > <https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>>>
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > Pulp-dev mailing list
> > > [email protected] <mailto:[email protected]>
> <mailto:[email protected]
> <mailto:[email protected]>>
> > > https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>
> <https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>>
> > >
> >
> >
> > _______________________________________________
> > Pulp-dev mailing list
> > [email protected] <mailto:[email protected]>
> <mailto:[email protected]
> <mailto:[email protected]>>
> > https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>
> <https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>>
> >
> >
>
>
> _______________________________________________
> Pulp-dev mailing list
> [email protected] <mailto:[email protected]>
> https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>
>
>
>
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Pulp-dev mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-dev
