On 2/19/19 12:30 PM, Brian Bouterse wrote:
Overall +1, we should make a 1-shot uploader. It will need a bit of
code from plugin writers to make it work though.
I believe the "manual and generic" feature offerings we have now are
pretty good. I don't think we should take them away.
To describe yet another related use case; consider the uploading of a
tarball with a lot of units in it (maybe a whole repo). For instance
bunch of Ansible roles in a tarball is what a user would want to
upload to Pulp, so you would want to upload the whole tarball at once.
I see three cases:
a) The generic method (what we already have). The user supplies all
metadata because we can't know the right attributes without them
telling it to us.
b) The one-shot uploader to create a single content unit from a single
Artifact. I think this is what is being proposed here. +1 to this.
c) Uploading a bunch of content units in one binary, e.g. a tarball.
We should leave this to plugin writers to implement for now.
Could b & c be the same api? The api could (depending on what the
plugin wants to implement) accept either one unit file or a tar file
with a bunch of different units.
In terms of implementing (b) I believe it would be useful to have a
Content unit provide an optional class method that will instantiate a
content unit (but not save it) from an Artifact. Then core can offer
an API that accepts a single Artifact binary and what content unit
type it is and it should create+save the Artifact and then create+save
the Content unit from that Artifact.
This doesn't address adding it to/from repositories. That would be ok
too as an additional option, but you probably want to do it to a whole
bunch at once somehow. Also it will need to go through the tasking
system for correctness.
On Tue, Feb 19, 2019 at 11:52 AM David Davis <davidda...@redhat.com
<mailto:davidda...@redhat.com>> wrote:
What about the case of multi-artifact content? Don’t they require
separate artifact creation and content creation routes?
David
On Tue, Feb 19, 2019 at 11:40 AM Austin Macdonald
<aus...@redhat.com <mailto:aus...@redhat.com>> wrote:
I think the key question to ask is:
What circumstances would require the creation of Content that
would not be met by a one-shot upload?
On Tue, Feb 19, 2019 at 11:34 AM Daniel Alley
<dal...@redhat.com <mailto:dal...@redhat.com>> wrote:
@Matthias why would you prefer to keep the normal create?
As you mention, the "orphan cleanup" issue applies to
both, so there's no advantage given to the former.
The normal "create" ( POST .../content/plugin/type/ ...)
is unidiomatic and inconsistent, because the fields needed
to upload a content are different from the fields on the
actual serializer. Most content types keep metadata
inside the packages themselves, so you can't let the user
specify the content field values, you have to contort
everything so that instead of hydrating the serializer
from user input, it does so by parsing the content.
There's also the issue that the libraries we're using to
parse the (Python and RPM) packages do some validation on
the filename to see that it has the right extension and so
forth, and since artifacts are content-addressed and don't
store that information, with normal create you have to
pass the filename of the original artifact *back in* at
the time you create it, and then copy the file from Pulp
content storage into a temp directory under a filename
which will validate properly, which is highly unfortunate.
With one-shot upload, you avoid both of those problems,
because there's no broken expectations as to what fields
should be accepted, and because it should be possible
(though I haven't tried it) to parse the original file
*before* saving it as an artifact, thus avoiding a lot of
mess. And you also get the option to add it straight into
a repository. In my opinion, it's a straight upgrade.
On Tue, Feb 19, 2019 at 10:57 AM Matthias Dellweg
<dell...@atix.de <mailto:dell...@atix.de>> wrote:
I have no objections to having the "one shot upload"
or even "one shot
upload into repositoryversion". I think, i would like
to keep the
'traditional' create anyway.
The problem i see with create and one shot upload is,
that another
thing could have triggered 'delete orphans' at the
wrong time, and you
shiny new content unit disappears, before you can add
it to a
repository version.
On Mon, 18 Feb 2019 14:41:54 -0500
Austin Macdonald <aus...@redhat.com
<mailto:aus...@redhat.com>> wrote:
> Originally, our upload story was as follows:
> The user will upload a new file to Pulp via POST to
/artifacts/
> (provided by core)
> The user will create a new plugin specific Content
via POST to
> /path/to/plugin/content/, referencing whatever
artifacts that are
> contained, and whatever fields are expected for the
new content.
> The user will add the new content to a repository
via POST to
> /repositories/1/versions/
>
> However, this is somewhat cumbersome to the user
with 3 API calls to
> accomplish something that only took one call in Pulp 2.
>
> There are a couple of different paths plugins have
taken to improve
> the user experience:
> The Python plugin follows the above workflow, but
reads the Artifact
> file to determine the values for the fields. The RPM
plugin has gone
> even farther and created a new endpoint for "one
shot" upload that
> perform all of this in a single call. I think it is
likely that the
> Python plugin will move more in the "one shot"
direction, and other
> plugins will probably follow.
>
> That said, I think we should discuss this as a
community to encourage
> plugins to behave similarly, and because there may
also be a
> possibility for sharing some of code. It is my hope
that a "one shot
> upload" could do 2 things: 1) Upload and create
Content. 2)
> Optionally add that content to repositories.
_______________________________________________
Pulp-dev mailing list
Pulp-dev@redhat.com <mailto:Pulp-dev@redhat.com>
https://www.redhat.com/mailman/listinfo/pulp-dev
_______________________________________________
Pulp-dev mailing list
Pulp-dev@redhat.com <mailto:Pulp-dev@redhat.com>
https://www.redhat.com/mailman/listinfo/pulp-dev
_______________________________________________
Pulp-dev mailing list
Pulp-dev@redhat.com <mailto:Pulp-dev@redhat.com>
https://www.redhat.com/mailman/listinfo/pulp-dev
_______________________________________________
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev
_______________________________________________
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev
