Overall +1, we should make a 1-shot uploader. It will need a bit of code from plugin writers to make it work though.
I believe the "manual and generic" feature offerings we have now are pretty good. I don't think we should take them away. To describe yet another related use case; consider the uploading of a tarball with a lot of units in it (maybe a whole repo). For instance bunch of Ansible roles in a tarball is what a user would want to upload to Pulp, so you would want to upload the whole tarball at once. I see three cases: a) The generic method (what we already have). The user supplies all metadata because we can't know the right attributes without them telling it to us. b) The one-shot uploader to create a single content unit from a single Artifact. I think this is what is being proposed here. +1 to this. c) Uploading a bunch of content units in one binary, e.g. a tarball. We should leave this to plugin writers to implement for now. In terms of implementing (b) I believe it would be useful to have a Content unit provide an optional class method that will instantiate a content unit (but not save it) from an Artifact. Then core can offer an API that accepts a single Artifact binary and what content unit type it is and it should create+save the Artifact and then create+save the Content unit from that Artifact. This doesn't address adding it to/from repositories. That would be ok too as an additional option, but you probably want to do it to a whole bunch at once somehow. Also it will need to go through the tasking system for correctness. On Tue, Feb 19, 2019 at 11:52 AM David Davis <davidda...@redhat.com> wrote: > What about the case of multi-artifact content? Don’t they require separate > artifact creation and content creation routes? > > David > > > On Tue, Feb 19, 2019 at 11:40 AM Austin Macdonald <aus...@redhat.com> > wrote: > >> I think the key question to ask is: >> What circumstances would require the creation of Content that would not >> be met by a one-shot upload? >> >> On Tue, Feb 19, 2019 at 11:34 AM Daniel Alley <dal...@redhat.com> wrote: >> >>> @Matthias why would you prefer to keep the normal create? As you >>> mention, the "orphan cleanup" issue applies to both, so there's no >>> advantage given to the former. >>> >>> The normal "create" ( POST .../content/plugin/type/ ...) is unidiomatic >>> and inconsistent, because the fields needed to upload a content are >>> different from the fields on the actual serializer. Most content types >>> keep metadata inside the packages themselves, so you can't let the user >>> specify the content field values, you have to contort everything so that >>> instead of hydrating the serializer from user input, it does so by parsing >>> the content. >>> >>> There's also the issue that the libraries we're using to parse the >>> (Python and RPM) packages do some validation on the filename to see that it >>> has the right extension and so forth, and since artifacts are >>> content-addressed and don't store that information, with normal create you >>> have to pass the filename of the original artifact *back in* at the time >>> you create it, and then copy the file from Pulp content storage into a temp >>> directory under a filename which will validate properly, which is highly >>> unfortunate. >>> >>> With one-shot upload, you avoid both of those problems, because there's >>> no broken expectations as to what fields should be accepted, and because it >>> should be possible (though I haven't tried it) to parse the original file >>> *before* saving it as an artifact, thus avoiding a lot of mess. And you >>> also get the option to add it straight into a repository. In my opinion, >>> it's a straight upgrade. >>> >>> On Tue, Feb 19, 2019 at 10:57 AM Matthias Dellweg <dell...@atix.de> >>> wrote: >>> >>>> I have no objections to having the "one shot upload" or even "one shot >>>> upload into repositoryversion". I think, i would like to keep the >>>> 'traditional' create anyway. >>>> The problem i see with create and one shot upload is, that another >>>> thing could have triggered 'delete orphans' at the wrong time, and you >>>> shiny new content unit disappears, before you can add it to a >>>> repository version. >>>> >>>> On Mon, 18 Feb 2019 14:41:54 -0500 >>>> Austin Macdonald <aus...@redhat.com> wrote: >>>> >>>> > Originally, our upload story was as follows: >>>> > The user will upload a new file to Pulp via POST to /artifacts/ >>>> > (provided by core) >>>> > The user will create a new plugin specific Content via POST to >>>> > /path/to/plugin/content/, referencing whatever artifacts that are >>>> > contained, and whatever fields are expected for the new content. >>>> > The user will add the new content to a repository via POST to >>>> > /repositories/1/versions/ >>>> > >>>> > However, this is somewhat cumbersome to the user with 3 API calls to >>>> > accomplish something that only took one call in Pulp 2. >>>> > >>>> > There are a couple of different paths plugins have taken to improve >>>> > the user experience: >>>> > The Python plugin follows the above workflow, but reads the Artifact >>>> > file to determine the values for the fields. The RPM plugin has gone >>>> > even farther and created a new endpoint for "one shot" upload that >>>> > perform all of this in a single call. I think it is likely that the >>>> > Python plugin will move more in the "one shot" direction, and other >>>> > plugins will probably follow. >>>> > >>>> > That said, I think we should discuss this as a community to encourage >>>> > plugins to behave similarly, and because there may also be a >>>> > possibility for sharing some of code. It is my hope that a "one shot >>>> > upload" could do 2 things: 1) Upload and create Content. 2) >>>> > Optionally add that content to repositories. >>>> _______________________________________________ >>>> Pulp-dev mailing list >>>> Pulp-dev@redhat.com >>>> https://www.redhat.com/mailman/listinfo/pulp-dev >>>> >>> _______________________________________________ >> Pulp-dev mailing list >> Pulp-dev@redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-dev >> > _______________________________________________ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev >
_______________________________________________ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev