> What i do not see is, how it is supposed to work for > multiple-artifact content (does it really exist? Docker?). Can one > upload multiple files in one api-call?
Well, as I mention, I don't think one-shot upload would be a good fit there. I'm *not* suggesting we try to cram everything through a one-shot upload pattern that would completely replace what we do currently. It's the same with no-artifact content such as errata, I think sending a POST to the content viewset is still the best solution there. But for plugins that have a single artifact and metadata stored internally (which will be most of them), normal "create" meets their needs rather poorly, so the question is thus: - Should we have a second semi-official pattern, one-shot upload, and should Pulp do anything to facilitate this workflow if possible - For plugins that implement one-shot upload, should they *also* try to implement the normal "create" endpoints, even though it's un-idiomatic and the implementation is by necessity much more complicated than one-shot while also being less flexible. But doing these things wouldn't prevent any plugin from doing things the way they're done currently. The question is, should *plugins* support both ways of uploading for "consistency" [2], not whether Pulp should do so. [3] 1. unless you count the digest *of* the file, but we don't expose that via the serializer, so still doesn't expose that particular ergonomic issue 2. *endpoint* consistency, perhaps, but because of the issues I mentioned, you may or may not need to pass in the original filename of the artifact, which isn't a field on the serializer at all. This breaks with all the expectations of what a viewset is supposed to do 3. Pulp doesn't actually "support" this to begin with, it's 100% provided by Django REST Framework On Tue, Feb 19, 2019 at 12:07 PM Matthias Dellweg <dell...@atix.de> wrote: > If you parse all the metadata from the file / existing artifact, i see > no real difference (from the users perspective). One call would accept > a file upload, the other a reference to an existing artifact. The > latter might impose a problem internally if the artifacts are stored > *somewhere*, as you say. In the case of the file plugin, you need to > specify a file / artifact_ref and a relative_path. So no big difference > again. > What i do not see is, how it is supposed to work for > multiple-artifact content (does it really exist? Docker?). Can one > upload multiple files in one api-call? > > Does the internal copying around hurt so much as to not wanting to > implement the create endpoint? > > I know, the orphaning problem is really only circumvented by an atomic > one shot into the repo version endpoint. But i thought the problem is > related to this story. > > On Tue, 19 Feb 2019 11:33:18 -0500 > Daniel Alley <dal...@redhat.com> wrote: > > > @Matthias why would you prefer to keep the normal create? As you > > mention, the "orphan cleanup" issue applies to both, so there's no > > advantage given to the former. > > > > The normal "create" ( POST .../content/plugin/type/ ...) is > > unidiomatic and inconsistent, because the fields needed to upload a > > content are different from the fields on the actual serializer. Most > > content types keep metadata inside the packages themselves, so you > > can't let the user specify the content field values, you have to > > contort everything so that instead of hydrating the serializer from > > user input, it does so by parsing the content. > > > > There's also the issue that the libraries we're using to parse the > > (Python and RPM) packages do some validation on the filename to see > > that it has the right extension and so forth, and since artifacts are > > content-addressed and don't store that information, with normal > > create you have to pass the filename of the original artifact *back > > in* at the time you create it, and then copy the file from Pulp > > content storage into a temp directory under a filename which will > > validate properly, which is highly unfortunate. > > > > With one-shot upload, you avoid both of those problems, because > > there's no broken expectations as to what fields should be accepted, > > and because it should be possible (though I haven't tried it) to > > parse the original file *before* saving it as an artifact, thus > > avoiding a lot of mess. And you also get the option to add it > > straight into a repository. In my opinion, it's a straight upgrade. > > > > On Tue, Feb 19, 2019 at 10:57 AM Matthias Dellweg <dell...@atix.de> > > wrote: > > > > > I have no objections to having the "one shot upload" or even "one > > > shot upload into repositoryversion". I think, i would like to keep > > > the 'traditional' create anyway. > > > The problem i see with create and one shot upload is, that another > > > thing could have triggered 'delete orphans' at the wrong time, and > > > you shiny new content unit disappears, before you can add it to a > > > repository version. > > > > > > On Mon, 18 Feb 2019 14:41:54 -0500 > > > Austin Macdonald <aus...@redhat.com> wrote: > > > > > > > Originally, our upload story was as follows: > > > > The user will upload a new file to Pulp via POST to /artifacts/ > > > > (provided by core) > > > > The user will create a new plugin specific Content via POST to > > > > /path/to/plugin/content/, referencing whatever artifacts that are > > > > contained, and whatever fields are expected for the new content. > > > > The user will add the new content to a repository via POST to > > > > /repositories/1/versions/ > > > > > > > > However, this is somewhat cumbersome to the user with 3 API calls > > > > to accomplish something that only took one call in Pulp 2. > > > > > > > > There are a couple of different paths plugins have taken to > > > > improve the user experience: > > > > The Python plugin follows the above workflow, but reads the > > > > Artifact file to determine the values for the fields. The RPM > > > > plugin has gone even farther and created a new endpoint for "one > > > > shot" upload that perform all of this in a single call. I think > > > > it is likely that the Python plugin will move more in the "one > > > > shot" direction, and other plugins will probably follow. > > > > > > > > That said, I think we should discuss this as a community to > > > > encourage plugins to behave similarly, and because there may also > > > > be a possibility for sharing some of code. It is my hope that a > > > > "one shot upload" could do 2 things: 1) Upload and create > > > > Content. 2) Optionally add that content to repositories. > > > _______________________________________________ > > > Pulp-dev mailing list > > > Pulp-dev@redhat.com > > > https://www.redhat.com/mailman/listinfo/pulp-dev > > > > > > > > Herzliche Grüße aus München > > Matthias Dellweg > ______________________________________________________ > Dr. Matthias M. Dellweg > > (Open Source Software Engineer) > > Tel: +49 (0)89 452 35 38-12 > Fax: +49 (0)89 452 35 38-290 > E-Mail: dell...@atix.de > > ATIX - The Linux & Open Source Company > > ATIX Informationstechnologie und Consulting AG > Parkring 15 > 85748 Garching bei München > www.atix.de > > > Registergericht: Amtsgericht München, Registernummer: HRB 168930 > USt.-Id.: DE209485962 > Vorstand: Thomas Merz (Vors.), Mark Hlawatschek > Vorsitzender des Aufsichtsrats: Dr. Martin Buss >
_______________________________________________ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev