Re: [Pulp-dev] the "relative path" problem

[Daniel Alley]

There is a video call scheduled to discuss this issue tomorrow (Tuesday
April 28th) at 13:30 UTC (please convert to your local time).
https://meet.google.com/scy-csbx-qiu

On Sat, Apr 25, 2020 at 7:02 AM David Davis  wrote:

> I had a chance to think about this some more yesterday and wanted to email
> out my thoughts. I also think that this change sounds scary and will have a
> big impact on plugin writers so I thought of a couple alternatives:
>
> First, we could add a relative_path field to RepositoryContent instead of
> moving it there. This would be an optional field. It would be up to plugins
> to manage this field and they would still need to populate the
> relative_path field on ContentArtifact. But plugins could use this optional
> field to store relative paths per repository and then use this field when
> generating publications.
>
> The second alternative is one that is already laid out in the original
> email but to call it out again: it would be to not solve this in pulpcore.
> RPM would create its own object that would map content in a repository to
> relative_paths.
>
> David
>
>
> On Tue, Apr 21, 2020 at 9:22 AM Quirin Pamp  wrote:
>
>> Hi,
>>
>>
>> I am not currently very well versed in the classes involved, but moving
>> relative_path around sounds slightly scary with the potential to break
>> things.
>>
>>
>> As such, I would be interested to be kept in the loop as this moves
>> forward. (Mailing list once there is some movement is entirely sufficient
>> )
>>
>>
>> Thanks,
>>
>> Quirin Pamp
>> --
>> *From:* pulp-dev-boun...@redhat.com  on
>> behalf of Ina Panova 
>> *Sent:* 21 April 2020 14:07:13
>> *To:* Daniel Alley 
>> *Cc:* Pulp-dev 
>> *Subject:* Re: [Pulp-dev] the "relative path" problem
>>
>> Daniel,
>>
>> how about setting up a meeting and brainstorm the alternatives, pros/cons
>> there?
>>
>>
>> 
>> Regards,
>>
>> Ina Panova
>> Senior Software Engineer| Pulp| Red Hat Inc.
>>
>> "Do not go where the path may lead,
>>  go instead where there is no path and leave a trail."
>>
>>
>> On Fri, Apr 17, 2020 at 5:57 PM Daniel Alley  wrote:
>>
>> Bump, this item needs to move forwards soon.  Does anyone have any
>> thoughts?
>>
>> On Wed, Apr 1, 2020 at 9:40 AM Pavel Picka  wrote:
>>
>> Hi,
>> I'd like to add one more question to this topic. Do you think it is a
>> blocker for PRs [0] & [1] as by testing [2] this features I haven't run
>> into real world example where two really same name packages appears.
>> I think this is a 'must have' feature but until we solve/decide it we can
>> have two features working may with warning in docs for users that can
>> happen in some 'special' repositories.
>>
>> To follow topic directly I like proposed move to 'RepositoryContent' and
>> add it to its uniqueness constraint (if I understand well).
>>
>> [0] https://github.com/pulp/pulp_rpm/pull/1657
>> [1] https://github.com/pulp/pulp_rpm/pull/1642
>> [2] tested with centos 7, 8, opensuse and SLE repositories
>>
>> On Wed, Apr 1, 2020 at 3:22 PM Daniel Alley  wrote:
>>
>> We'd like to start a discussion on the "relative path problem" identified
>> recently.
>> Problem:
>>
>> Currently, a relative_path is tied to content in Pulp. This means that if
>> a content unit exists in two places within a repository or across
>> repositories, it has to be stored as two separate content units. This
>> creates redundant data and potential confusion for users.
>>
>> As a specific example, we need to support mirroring content in pulp_rpm
>> . Currently, for each location at
>> which a single package is stored, we’ll need to create a content unit. We
>> could end up with several records representing a single package. Users may
>> be confused about why they see multiple records for a package and they may
>> have trouble for example deciding which content unit to copy.
>> Proposed Solution:
>>
>> Move “relative_path” from its current location on ContentArtifact, to
>> RepositoryContent. This will require a sizable data migration. It is
>> possibly the case that in rare cases, repository versions may change
>> slightly due to deduplication.
>>
>> A repository-version-wide uniqueness constraint will be present on
>> “relative_path”, independently of any other repository uniquness
>> constraints (repo_key_fields) defined by the plugin writer.
>>
>> Modify the Stages API so that the relative_path can be processed in the
>> correct location – instead of “DeclarativeArtifact” it will likely need to
>> go on “DeclarativeContent”
>>
>> Remove “location_href” from the RPM Package content model – it was never
>> a true part of the RPM (file) metadata, it is derived from the repository
>> metadata. So storing it as a part of the Content unit doesn’t entirely make
>> sense.
>> Alternatives
>>
>> In most cases, a content unit will have a single relative path for a
>> content unit. Creating a general solution to solve a one-off problem is
>> usually not a good 

Re: [Pulp-dev] redmine process for katello-integration-related issues

[Grant Gainey]

On Mon, Apr 27, 2020 at 3:08 PM Brian Bouterse  wrote:

> On Mon, Apr 27, 2020 at 2:52 PM Grant Gainey  wrote:
>
>>
>> Sooo, how about the following for YARP (Yet Another Redmine Process):
>>
>>- Anything with a katello-PX tag, gets a *Katello* tag
>>- Open issues with a Katello-P1 tag, get set to *High* priority
>>- Open issues with a Katello-P1 or Katello-P2 tag, acquire a "
>>*Katello-Release-3.16*"[1] tag
>>- Open issues with a Katello-P3 tag acquire a "*Katello-Release-3.17*"[1]
>>tag
>>- Katello-PX tags are removed
>>- Milestone is used in pulpcore and plugins to describe the version *of
>>that project* when an issue is expected to be delivered
>>- *Katello devs *that open issues as they test are *responsible* for
>>setting *Priority* and adding the *Katello tag*
>>- *Pulp/Katello triage meetings* are *responsible* for adding 
>> *Katello-3.XX
>>tags* and *validating Priority*
>>- *Pulp/plugin devs* *prioritize* work for katello *by
>>Katello-version and priority*.
>>
>> On the downside, we end up with Yet More Tags doing things this way.
>>
> My goal is to have fewer tags and a simpler process. Here's my
> counterproposal. Can we replace the 'Katello - P1", "Katello - P2",
> "Katello - P3" with a single tag called ''Katello"? I think the weekly
> check-in meeting will allow us to handle everything else in terms of
> prioritization.
>

I actually think that will be *less simple *after about the second
triage-meeting.

The first five bullets up there are a one-time process to get us away from
Katello-PX tags (and is prob going to be me pushing the buttons, so I'm ok
with it :) ) So let's leave them aside.

Milestone use is up to the plugin/project teams, to use or not. It's just
listed to make clear it is *not* useful in the katello-PX context.

With "just use a single tag 'Katello'", triage team meets weekly-ish,
decides when things need to be done and in what priority. How is that
recorded? As someone looking for "what I should work on next", where do I
get that info? Do I have to find meeting minutes, or a spreadsheet, or
worst track someone down and interrupt them? When priorities change, how do
we make sure nobody's working from last-week's minutes?

The last three bullets give us three responsible parties, each with one
job. Katello-devs remember to mark issues 'Katello' when they open them.
Triage meeting writes "needed by" and "priority" down *in the issues
themselves*, rather than in a separate document. Developers let redmine
tell them what order to work on things.

I think my proposal makes things *simpler* - because if we follow it, there
should be way less "asking around" to figure out what priorities are. Tools
like redmine/bugzilla should be able to give a developer (or a manager, or
an Interested Party) a view of what's being worked on, when, and in what
order, without requiring extra reporting/documentation/meetings.

Thoughts?

G


>
>> On the upside, issues no longer require updating every release, devs can
>> prioritize using an issue-query, and everyone can see what everyone else's
>> expectations are without needing a meeting for it.
>>
>> I am absolutely certain I have missed something - it was all the way back
>> on Friday morning when I had the last conversation about this, which may as
>> well have been in the Pleistocene as far as my neurons are concerned.
>> Please comment with suggestions/corrections.
>>
>> G
>>
>> [1] I prob am off on my katello-versions - but you get the idea...
>>
>>
>> How's that sound?
>>>
>>> G
>>>
>>>
>>> On Thu, Apr 23, 2020 at 6:29 AM David Davis 
>>> wrote:
>>>
 pulpcore is not the only project with milestones. Each project has its
 own set of milestones to represent different versions. The issue is your
 proposal is trying to assign all Katello-PX issues to a pulpcore milestone.
 That won't work if you have say a pulp_container or pulp_ansible issue with
 a Katello-PX tag.

 You could go through all the Katello-PX issues and try to figure out
 which milestone in which project to assign it to but with so many
 milestones and projects, it's going to be a huge pain. My suggestion was to
 just worry about the open issues and not try to retroactively assign these
 issues to milestones.

 David


 On Wed, Apr 22, 2020 at 7:12 PM Grant Gainey 
 wrote:

>
>
> On Wed, Apr 22, 2020 at 6:17 PM David Davis 
> wrote:
>
>> A couple observations: the 3.3[0] and 3.4[1] milestones already exist
>> in redmine. Also, you won't be able to assign any of the MODIFIED issues 
>> to
>> 3.3 because they're all plugin issues and the 3.3 milestone is exclusive 
>> to
>> pulpcore. IMHO, I probably wouldn't assign any issues to any milestones. 
>> I
>> think it would be worse having an issue on the wrong milestone than it
>> being unassigned.
>>
>
> If pulpcore is the only 

[Pulp-dev] Export-by-repositoryversion : design thinking needed!

[Grant Gainey]

Discussions with Katello uncovered a use-case that we thought about briefly
in the initial design of import/export, and then put aside to get to
first-light. That issue is "must be able to export a specific set of
RepositoryVersions for the specified Repositories" (as opposed to "export
current-version always"). This is a large use case for katello's
lifecycle-workflows.

In the design doc this is mentioned under 'Exporter' - the implication
being that if one wants this, one creates a new Exporter every time. This
is...not a great approach.

Exporter and Importer work at the Repository level. This is so that the
user can map upstream and downstream repositories once, at
Importer-create-time, and then reuse that Importer repeatedly. If an
Exporter is a single-use entity, as this approach would imply, it requires
the user to do that startup work every time.

A better (imho) approach is to allow a set of RepositoryVersions to be
specified at Export-creation time (by, say, providing a
repositories=[___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] redmine process for katello-integration-related issues

[Brian Bouterse]

On Mon, Apr 27, 2020 at 2:52 PM Grant Gainey  wrote:

> Once more unto the breach, dear friends, once more...
>
> When last we saw our process, we were at something like this:
>
> On Thu, Apr 23, 2020 at 8:12 AM Grant Gainey  wrote:
>
>> OK, so let's take a step back.  The thing that the initial proposal
>> is/was attempting to address is that with the current P1/2/3 system, issues
>> that are P2 one day become P1 the next, basically tied to katello's and
>> pulp's release cycles. The goal was to be able to a) mark all issues that
>> are important to katelllo, b) mark them for "when katello will need them"
>> to meet its promises, and then c) be able to prioritize work inside a
>> release cycle. Note that we were/are trying to get away from priority
>> meaning "which release" - there are things that have been discussed, that
>> are nice-to-haves (and therefore not High priority), that would need to be
>> available by a given release for katello to take advantage of them (ie
>> slated for a milestone). We also need to separate blockers from "work as
>> usual". The initial proposal would solve a) by tagging something as
>> 'Katello', b) by use of the milestone field to show what release *of
>> pulp* the thing needed to be available by, and let us use Priority to
>> address all the aspects of c), with blockers getting the Urgent priority.
>>
>> Having done a bunch more digging, you're right, 'milestone' as it stands
>> can't be used this way for anything other than pulpcore.
>>
>> I think what we can do at this point is something like this:
>>
>>- set milestone for pulpcore issues, P2 = 3.4, P3 = 3.5
>>- mark *all* the Katello-PX tagged issues with an
>>additional 'Katello' tag
>>- any open Katello-P1 issues get set to *High* priority
>>- remove the Katello-PX tags from *pulpcore issues only*
>>- for the plugins, leave the Katello-PX tags - plugins who want to
>>follow this process can set version-milestones as appropriate and remove
>>when they're done
>>
>>
> A(nother) requirement that needs to be addressed is "marking issues that
> are opened *by the katello team*" - not "need this feature for
> release-X", but "found this bug/issue" kind of things. This is a(nother)
> cross-plugin need.
>
> So, back to "what are we trying to accomplish".
>
> The current process has problems because it:
>
>- conflates "needed by" and "priority"
>- conflates "opened *by* katello" and "opened *for* katello"
>- requires a lot of manual work as part of the release cycle to update
>as the P2s become 1s and the 3s become 2s etc
>
> Relying on 'milestone', as we initially proposed, is a nonstarter.
> Milestones are project-specific, aimed at a *project's* release-cycle,
> and Care Not for pulpcore or katello. The cross-plugin Thing we have, is
> 'Tag'. We'd like to use that for both "katello found this" and "katello
> needs this by X". In this context, 'X' is *katello** release*, because
> that's what drives "katello needs this by..." discussions.
>
> Sooo, how about the following for YARP (Yet Another Redmine Process):
>
>- Anything with a katello-PX tag, gets a *Katello* tag
>- Open issues with a Katello-P1 tag, get set to *High* priority
>- Open issues with a Katello-P1 or Katello-P2 tag, acquire a "
>*Katello-Release-3.16*"[1] tag
>- Open issues with a Katello-P3 tag acquire a "*Katello-Release-3.17*"[1]
>tag
>- Katello-PX tags are removed
>- Milestone is used in pulpcore and plugins to describe the version *of
>that project* when an issue is expected to be delivered
>- *Katello devs *that open issues as they test are *responsible* for
>setting *Priority* and adding the *Katello tag*
>- *Pulp/Katello triage meetings* are *responsible* for adding *Katello-3.XX
>tags* and *validating Priority*
>- *Pulp/plugin devs* *prioritize* work for katello *by Katello-version
>and priority*.
>
> On the downside, we end up with Yet More Tags doing things this way.
>
My goal is to have fewer tags and a simpler process. Here's my
counterproposal. Can we replace the 'Katello - P1", "Katello - P2",
"Katello - P3" with a single tag called ''Katello"? I think the weekly
check-in meeting will allow us to handle everything else in terms of
prioritization.


> On the upside, issues no longer require updating every release, devs can
> prioritize using an issue-query, and everyone can see what everyone else's
> expectations are without needing a meeting for it.
>
> I am absolutely certain I have missed something - it was all the way back
> on Friday morning when I had the last conversation about this, which may as
> well have been in the Pleistocene as far as my neurons are concerned.
> Please comment with suggestions/corrections.
>
> G
>
> [1] I prob am off on my katello-versions - but you get the idea...
>
>
> How's that sound?
>>
>> G
>>
>>
>> On Thu, Apr 23, 2020 at 6:29 AM David Davis 
>> wrote:
>>
>>> pulpcore is not the only project with 

Re: [Pulp-dev] redmine process for katello-integration-related issues

[Grant Gainey]

Once more unto the breach, dear friends, once more...

When last we saw our process, we were at something like this:

On Thu, Apr 23, 2020 at 8:12 AM Grant Gainey  wrote:

> OK, so let's take a step back.  The thing that the initial proposal is/was
> attempting to address is that with the current P1/2/3 system, issues that
> are P2 one day become P1 the next, basically tied to katello's and pulp's
> release cycles. The goal was to be able to a) mark all issues that are
> important to katelllo, b) mark them for "when katello will need them" to
> meet its promises, and then c) be able to prioritize work inside a release
> cycle. Note that we were/are trying to get away from priority meaning
> "which release" - there are things that have been discussed, that are
> nice-to-haves (and therefore not High priority), that would need to be
> available by a given release for katello to take advantage of them (ie
> slated for a milestone). We also need to separate blockers from "work as
> usual". The initial proposal would solve a) by tagging something as
> 'Katello', b) by use of the milestone field to show what release *of pulp*
> the thing needed to be available by, and let us use Priority to address all
> the aspects of c), with blockers getting the Urgent priority.
>
> Having done a bunch more digging, you're right, 'milestone' as it stands
> can't be used this way for anything other than pulpcore.
>
> I think what we can do at this point is something like this:
>
>- set milestone for pulpcore issues, P2 = 3.4, P3 = 3.5
>- mark *all* the Katello-PX tagged issues with an additional 'Katello'
>tag
>- any open Katello-P1 issues get set to *High* priority
>- remove the Katello-PX tags from *pulpcore issues only*
>- for the plugins, leave the Katello-PX tags - plugins who want to
>follow this process can set version-milestones as appropriate and remove
>when they're done
>
>
A(nother) requirement that needs to be addressed is "marking issues that
are opened *by the katello team*" - not "need this feature for release-X",
but "found this bug/issue" kind of things. This is a(nother) cross-plugin
need.

So, back to "what are we trying to accomplish".

The current process has problems because it:

   - conflates "needed by" and "priority"
   - conflates "opened *by* katello" and "opened *for* katello"
   - requires a lot of manual work as part of the release cycle to update
   as the P2s become 1s and the 3s become 2s etc

Relying on 'milestone', as we initially proposed, is a nonstarter.
Milestones are project-specific, aimed at a *project's* release-cycle, and
Care Not for pulpcore or katello. The cross-plugin Thing we have, is 'Tag'.
We'd like to use that for both "katello found this" and "katello needs this
by X". In this context, 'X' is *katello** release*, because that's what
drives "katello needs this by..." discussions.

Sooo, how about the following for YARP (Yet Another Redmine Process):

   - Anything with a katello-PX tag, gets a *Katello* tag
   - Open issues with a Katello-P1 tag, get set to *High* priority
   - Open issues with a Katello-P1 or Katello-P2 tag, acquire a "
   *Katello-Release-3.16*"[1] tag
   - Open issues with a Katello-P3 tag acquire a "*Katello-Release-3.17*"[1]
   tag
   - Katello-PX tags are removed
   - Milestone is used in pulpcore and plugins to describe the version *of
   that project* when an issue is expected to be delivered
   - *Katello devs *that open issues as they test are *responsible* for
   setting *Priority* and adding the *Katello tag*
   - *Pulp/Katello triage meetings* are *responsible* for adding *Katello-3.XX
   tags* and *validating Priority*
   - *Pulp/plugin devs* *prioritize* work for katello *by Katello-version
   and priority*.

On the downside, we end up with Yet More Tags doing things this way.

On the upside, issues no longer require updating every release, devs can
prioritize using an issue-query, and everyone can see what everyone else's
expectations are without needing a meeting for it.

I am absolutely certain I have missed something - it was all the way back
on Friday morning when I had the last conversation about this, which may as
well have been in the Pleistocene as far as my neurons are concerned.
Please comment with suggestions/corrections.

G

[1] I prob am off on my katello-versions - but you get the idea...


How's that sound?
>
> G
>
>
> On Thu, Apr 23, 2020 at 6:29 AM David Davis  wrote:
>
>> pulpcore is not the only project with milestones. Each project has its
>> own set of milestones to represent different versions. The issue is your
>> proposal is trying to assign all Katello-PX issues to a pulpcore milestone.
>> That won't work if you have say a pulp_container or pulp_ansible issue with
>> a Katello-PX tag.
>>
>> You could go through all the Katello-PX issues and try to figure out
>> which milestone in which project to assign it to but with so many
>> milestones and projects, it's going to be a huge pain. 

[Pulp-dev] unit test runners disabled for Pulp 2 repositories

[Dennis Kliban]

Due to challenges with maintaining infrastructure used to run unit tests
for Pulp 2 Pull Requests on GitHub, I've disabled the requirement for these
checks to pass before merging. I do not anticipate any challenges with
manually testing PRs due to the low volume of changes in those repos.
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] Certguard compatibility with Centos/RHEL7

[Brian Bouterse]

Here's a recap of a plan between @jsherrill @sajha @ehelms and myself
around adding a feature to certguard that will provide compatibility with
Apache 2.6.4. Currently certguard is compatible only with Apache 2.6.10+.
Below is a summary of the plan.

* pulp_certguard to introduce compatibility with non-urlencoded
certificates. The goal is by May 8th at the latest, but sooner is better.
Ideally May 4th - 6th.

* This work and technical solution (not yet written) will be tracked with
this issue: https://pulp.plan.io/issues/6574

* This functionality will be released as certguard 0.1.0rc5. Katello will
test certguard from a PR or master and then switch to 0.1.0rc5 when it's on
pypi

Any questions, comments, or concerns are welcome.

Thanks!
Brian
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] pulp_file 1.0

[Daniel Alley]

+1 to 1.0

On Mon, Apr 27, 2020 at 5:57 AM Ina Panova  wrote:

> Based on the extended reply from David referring to semver, I am in favour
> or releasing pulp_file 1.0.
>
> Also, comments inline.
> 
> Regards,
>
> Ina Panova
> Senior Software Engineer| Pulp| Red Hat Inc.
>
> "Do not go where the path may lead,
>  go instead where there is no path and leave a trail."
>
>
> On Wed, Apr 22, 2020 at 7:02 PM Brian Bouterse 
> wrote:
>
>> tl;dr we follow semver.org and I agree with your reasoning, so I'm
>> convinced 1.0 would be fine. While I'm not in favor of the change, I'm
>> ready to disagree and commit.
>>
>> In the interests of sharing perspectives, here's mine. The issue with
>> semver.org is that it's exclusively focused on change management, and it
>> ignores what I perceive as a cultural association with > 1.0 software to
>> mean "broadly tested and low risk". Is pulp_file at a point where backwards
>> compatibility is a primary concern and prohibited yes. Do the developers of
>> pulp_file recommend it to be run in production, yes. As a user, is it a low
>> risk software due to many folks having already deployed it in production,
>> no. In fact pulp_file is maybe in the high to medium risk category based on
>> the number of folks who are actually running it in production.
>>
>
> Brian, this is a kind of chicken and the egg problem. Let's be fair and
> answer - how many folks will deploy something that is 0.y.z and not
> production ready?
> Not a lot of folks will deploy it in the production unless we release and
> say - this is stable enough for production use. Only after that we will
> have enough numbers to fairly say if it is low/high/medium risk software.
>
>
>> Having said all that, I'm ready to support your proposal on the semver
>> basis. Your reasoning is sound. Thank you for writing your thoughts here
>> and your effort to make it great.
>>
>> On Wed, Apr 22, 2020 at 11:32 AM David Davis 
>> wrote:
>>
>>> I want to expound on my own reasoning behind why pulp_file should be
>>> bumped to 1.0 because I realize my original email was probably too brief
>>> and I apologize for that.
>>>
>>> The thing that I would refer to is semver.org which we've used as a
>>> guide for versioning. semver.org defines a 0.Y release as:
>>>
>>>Major version zero (0.y.z) is for initial development. Anything MAY
>>> change at any time. The public API SHOULD NOT be considered stable.
>>>
>>> Moreover, semver.org has this question/answer:
>>>
>>> How do I know when to release 1.0.0?
>>>
>>> If your software is being used in production, it should probably
>>> already be 1.0.0. If you have a stable API on which users have come to
>>> depend, you should be 1.0.0. If you’re worrying a lot about backwards
>>> compatibility, you should probably already be 1.0.0.
>>>
>>> I think we meet both of these criteria. I expect that Pulp users are
>>> probably using pulp_file in production already. In terms of its API, we've
>>> had only two small features in the last couple releases of pulp_file since
>>> 0.1.0[0] and no major changes to the public API (there was the rename of
>>> one field). I don't foresee any major changes to the public api anytime
>>> soon. There's not a roadmap for new features either and certainly nothing I
>>> see that could cause major changes to pulp_file's API.
>>>
>>> I think that in this context it makes sense to bump it to 1.0 to
>>> communicate to our users that the pulp_file API is stable enough to use in
>>> production.
>>>
>>> Thoughts?
>>>
>>> [0] https://github.com/pulp/pulp_file/blob/master/CHANGES.rst
>>>
>>> David
>>>
>>>
>>> On Wed, Apr 22, 2020 at 10:59 AM David Davis 
>>> wrote:
>>>
 I feel differently especially when considering that most other Pulp
 plugins are at > 1.0. Can you explain why you think pulp_file shouldn't be
 at 1.0?

 David


 On Wed, Apr 22, 2020 at 10:57 AM Brian Bouterse 
 wrote:

> I've seen software live in the < 1.0 area for a long time and graduate
> when it's in broad, production use. That's a difficult thing to assess
> accurately, but to me, pulp_file hasn't reached that point.
>
>
> On Tue, Apr 21, 2020 at 2:20 PM David Davis 
> wrote:
>
>> With the next release of pulp_file, I'd propose we bump the version
>> to 1.0. The pulp_file plugin has reached a level of maturity and 
>> stability
>> that I think it could be considered production-ready. I've opened a PR to
>> bump the version to 1.0.0:
>>
>> https://github.com/pulp/pulp_file/pull/380
>>
>> Feedback welcome. I'll set a deadline of April 27, 2020.
>>
>> David
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> 

Re: [Pulp-dev] serializers in pulp 3 can't use write_only fields

[Daniel Alley]

I just want to point out that we have a somewhat similar kind of feature
implemented, "minimal serializers", which uses the 2-serializer approach.
It is a very tiny amount of extra code/work.

e.g.
https://github.com/pulp/pulpcore/blob/master/pulpcore/app/serializers/task.py#L119
https://github.com/pulp/pulp_rpm/blob/master/pulp_rpm/app/serializers.py#L290

the implementation
https://github.com/pulp/pulpcore/blob/master/pulpcore/app/viewsets/base.py#L114-L139

I lean towards option 2 as well for consistency.

On Mon, Apr 27, 2020 at 5:20 AM Ina Panova  wrote:

> I like option number 2 more, even if there is more work behind the scene
> it is more explicit and does not leave opened questions like ' why certain
> fields are null'
> 
> Regards,
>
> Ina Panova
> Senior Software Engineer| Pulp| Red Hat Inc.
>
> "Do not go where the path may lead,
>  go instead where there is no path and leave a trail."
>
>
> On Fri, Apr 24, 2020 at 5:24 PM Dennis Kliban  wrote:
>
>> Today during open floor we concluded that write_only fields cannot be
>> used in serializers because any automated handling of them during OpenAPI
>> schema generation will cause mutations to the schema which will result in
>> backwards incompatible changes to our bindings users.
>>
>> This change should only affect the Content serializers that have a 'file'
>> and 'repository' write_only fields that can be used for uploading content
>> into a repository in a single request. Two solutions were proposed:
>>
>> 1) Use a single serializer for GET and POST and leave 'file' and
>> 'repository' null for GET requests. This solution is the simplest for
>> plugin writers because they don't have to do anything extra. However, it
>> does cause users to wonder why those fields are present and null on GET
>> requests.
>>
>> 2) Require plugin writers to provide 2 serializers for Content - one for
>> GET and another for POST. This is more work for plugin writers, but will
>> make it clear to users what fields are expected in each case.
>>
>> Even though I initially proposed 1, I am in favor of adopting 2. What
>> does everyone else think?
>>
>> We will need to provide validation at startup that will check serializers
>> for presence of write_only fields. The application should fail to start if
>> any are present. This way plugin writers will know not to use the fields.
>> We will also need to provide plugin writer docs that clearly state the
>> guidelines around write_only fields and suggestions for getting by without
>> them.
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] pulp_file 1.0

[Ina Panova]

Based on the extended reply from David referring to semver, I am in favour
or releasing pulp_file 1.0.

Also, comments inline.

Regards,

Ina Panova
Senior Software Engineer| Pulp| Red Hat Inc.

"Do not go where the path may lead,
 go instead where there is no path and leave a trail."


On Wed, Apr 22, 2020 at 7:02 PM Brian Bouterse  wrote:

> tl;dr we follow semver.org and I agree with your reasoning, so I'm
> convinced 1.0 would be fine. While I'm not in favor of the change, I'm
> ready to disagree and commit.
>
> In the interests of sharing perspectives, here's mine. The issue with
> semver.org is that it's exclusively focused on change management, and it
> ignores what I perceive as a cultural association with > 1.0 software to
> mean "broadly tested and low risk". Is pulp_file at a point where backwards
> compatibility is a primary concern and prohibited yes. Do the developers of
> pulp_file recommend it to be run in production, yes. As a user, is it a low
> risk software due to many folks having already deployed it in production,
> no. In fact pulp_file is maybe in the high to medium risk category based on
> the number of folks who are actually running it in production.
>

Brian, this is a kind of chicken and the egg problem. Let's be fair and
answer - how many folks will deploy something that is 0.y.z and not
production ready?
Not a lot of folks will deploy it in the production unless we release and
say - this is stable enough for production use. Only after that we will
have enough numbers to fairly say if it is low/high/medium risk software.


> Having said all that, I'm ready to support your proposal on the semver
> basis. Your reasoning is sound. Thank you for writing your thoughts here
> and your effort to make it great.
>
> On Wed, Apr 22, 2020 at 11:32 AM David Davis 
> wrote:
>
>> I want to expound on my own reasoning behind why pulp_file should be
>> bumped to 1.0 because I realize my original email was probably too brief
>> and I apologize for that.
>>
>> The thing that I would refer to is semver.org which we've used as a
>> guide for versioning. semver.org defines a 0.Y release as:
>>
>>Major version zero (0.y.z) is for initial development. Anything MAY
>> change at any time. The public API SHOULD NOT be considered stable.
>>
>> Moreover, semver.org has this question/answer:
>>
>> How do I know when to release 1.0.0?
>>
>> If your software is being used in production, it should probably
>> already be 1.0.0. If you have a stable API on which users have come to
>> depend, you should be 1.0.0. If you’re worrying a lot about backwards
>> compatibility, you should probably already be 1.0.0.
>>
>> I think we meet both of these criteria. I expect that Pulp users are
>> probably using pulp_file in production already. In terms of its API, we've
>> had only two small features in the last couple releases of pulp_file since
>> 0.1.0[0] and no major changes to the public API (there was the rename of
>> one field). I don't foresee any major changes to the public api anytime
>> soon. There's not a roadmap for new features either and certainly nothing I
>> see that could cause major changes to pulp_file's API.
>>
>> I think that in this context it makes sense to bump it to 1.0 to
>> communicate to our users that the pulp_file API is stable enough to use in
>> production.
>>
>> Thoughts?
>>
>> [0] https://github.com/pulp/pulp_file/blob/master/CHANGES.rst
>>
>> David
>>
>>
>> On Wed, Apr 22, 2020 at 10:59 AM David Davis 
>> wrote:
>>
>>> I feel differently especially when considering that most other Pulp
>>> plugins are at > 1.0. Can you explain why you think pulp_file shouldn't be
>>> at 1.0?
>>>
>>> David
>>>
>>>
>>> On Wed, Apr 22, 2020 at 10:57 AM Brian Bouterse 
>>> wrote:
>>>
 I've seen software live in the < 1.0 area for a long time and graduate
 when it's in broad, production use. That's a difficult thing to assess
 accurately, but to me, pulp_file hasn't reached that point.


 On Tue, Apr 21, 2020 at 2:20 PM David Davis 
 wrote:

> With the next release of pulp_file, I'd propose we bump the version to
> 1.0. The pulp_file plugin has reached a level of maturity and stability
> that I think it could be considered production-ready. I've opened a PR to
> bump the version to 1.0.0:
>
> https://github.com/pulp/pulp_file/pull/380
>
> Feedback welcome. I'll set a deadline of April 27, 2020.
>
> David
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
 ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] serializers in pulp 3 can't use write_only fields

[Ina Panova]

I like option number 2 more, even if there is more work behind the scene it
is more explicit and does not leave opened questions like ' why certain
fields are null'

Regards,

Ina Panova
Senior Software Engineer| Pulp| Red Hat Inc.

"Do not go where the path may lead,
 go instead where there is no path and leave a trail."


On Fri, Apr 24, 2020 at 5:24 PM Dennis Kliban  wrote:

> Today during open floor we concluded that write_only fields cannot be used
> in serializers because any automated handling of them during OpenAPI schema
> generation will cause mutations to the schema which will result in
> backwards incompatible changes to our bindings users.
>
> This change should only affect the Content serializers that have a 'file'
> and 'repository' write_only fields that can be used for uploading content
> into a repository in a single request. Two solutions were proposed:
>
> 1) Use a single serializer for GET and POST and leave 'file' and
> 'repository' null for GET requests. This solution is the simplest for
> plugin writers because they don't have to do anything extra. However, it
> does cause users to wonder why those fields are present and null on GET
> requests.
>
> 2) Require plugin writers to provide 2 serializers for Content - one for
> GET and another for POST. This is more work for plugin writers, but will
> make it clear to users what fields are expected in each case.
>
> Even though I initially proposed 1, I am in favor of adopting 2. What does
> everyone else think?
>
> We will need to provide validation at startup that will check serializers
> for presence of write_only fields. The application should fail to start if
> any are present. This way plugin writers will know not to use the fields.
> We will also need to provide plugin writer docs that clearly state the
> guidelines around write_only fields and suggestions for getting by without
> them.
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev