On Fri, Feb 22, 2019 at 9:36 AM Justin Sherrill <jsher...@redhat.com> wrote:
> > On 2/18/19 2:41 PM, Austin Macdonald wrote: > > Originally, our upload story was as follows: > The user will upload a new file to Pulp via POST to /artifacts/ (provided > by core) > The user will create a new plugin specific Content via POST to > /path/to/plugin/content/, referencing whatever artifacts that are > contained, and whatever fields are expected for the new content. > The user will add the new content to a repository via POST to > /repositories/1/versions/ > > However, this is somewhat cumbersome to the user with 3 API calls to > accomplish something that only took one call in Pulp 2. > > How would you do this with one call in pulp2? > https://docs.pulpproject.org/dev-guide/integration/rest-api/content/upload.html > seems to suggest 3-4 calls. > Some plugins implemented the pulp2 equivalent of a one-shot uploader. Those docs are for pulp2's core which don't include the plugin's docs. > > There are a couple of different paths plugins have taken to improve the > user experience: > The Python plugin follows the above workflow, but reads the Artifact file > to determine the values for the fields. The RPM plugin has gone even > farther and created a new endpoint for "one shot" upload that perform all > of this in a single call. I think it is likely that the Python plugin will > move more in the "one shot" direction, and other plugins will probably > follow. > > How does the RPM one shot api work? Will it be compatible with whatever > solution https://pulp.plan.io/issues/4196 arrives at? > You would upload the Artifact as binary data along with what content type it is and what relative path it uses and Pulp creates the Artifact, Content unit, ContentArtifact. It should be compatible with issue 4196 because django's binary form data should allow for parallel uploading before calling the view handler. It may take 2 calls though. The issue to me isn't about the number of calls as it is the client data payload complexity. > I would hate for all our plugins to move to One shot methods which users > can't even rely on. > I don't think we're taking the "generic" uploading away. You can always rely on that. The issue w/ one-shot is that it's not possible (literally) for many content types, e.g. Artifact-less content. It's also hard for multi-artifact Content so that would probably still be something plugin writers would provide as a custom thing for their content type. Regardless it's just not possible to have consistency in this area. > My problem with single api calls to upload files is that we cannot > reliably use them due to limitation in request sizes. We have to be > prepared to use multiple calls to upload files regardless. Maybe if a user > is using some plugin that never has super large files (ansible?) you could > be confident you would never hit a request size limitation. But file, > docker, and yum all would require multiple calls to get the physical data > to the server. > I believe arbitrarily large files can be uploaded either through multi-part form data or through the django-chunked interface. We'll see what happens with 4196, but I expect arbitrary payload size to be a requirement for Pulp users. > I care more about having a consistent method for uploading files than > having fewer api calls. If we need a some content specific api, that's > fine, but please make it a consistent part of the process. > It sounds like the 4-call interface is the only choice then if consistency is a must. There isn't a way to offer consistency for one-shot uploaders. Is it ok that Katello will have to fill out all of the field data when you post the content type? What could be better? > I feel like we may be chasing the wrong goal here (fewer calls vs a more > consistent experience). > > > That said, I think we should discuss this as a community to encourage > plugins to behave similarly, and because there may also be a possibility > for sharing some of code. It is my hope that a "one shot upload" could do 2 > things: 1) Upload and create Content. 2) Optionally add that content to > repositories. > > _______________________________________________ > Pulp-dev mailing > listPulp-dev@redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev > > _______________________________________________ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev >
_______________________________________________ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
