+1 from me. One of the problems I foresee though is that this could make cherry picking difficult if we have tests that depend on other tests. For example, suppose you have change A that adds some tests and then change B adds some tests on top of A's tests. It'll make cherry picking B without A tricky.
We'll need to be careful to avoid such situations. David On Tue, Aug 18, 2020 at 4:48 AM Matthias Dellweg <mdell...@redhat.com> wrote: > +1 > > On Tue, Aug 18, 2020 at 10:39 AM Pavel Picka <ppi...@redhat.com> wrote: > > > > Big +1 to require (at least) one basic/positive functional test if > possible. > > Maybe we can set up a review checklist (contains 'check for basic test'). > > > > We already have some docs which we can update a bit [0] to be yet more > plugin writer friendly and point back to it from plugins' docs. > > Like extending it with the way the test can be run when pulp is > installed e.g. by the rpm package. > > > > [0] https://docs.pulpproject.org/contributing/tests.html > > > > On Mon, Aug 17, 2020 at 10:07 PM Brian Bouterse <bmbou...@redhat.com> > wrote: > >> > >> I have two goals with this: > >> > >> 1. Improve the stability of pulpcore and it's plugins > >> > >> 2. Enable all downstreams and packagers with tests and information on > how to run those tests, so they can run the automated tests even as the > downstream is curated with a series of backports. This closely follows the > model of a downstream maintainer who backports a feature/fix from upstream > into a RPM and runs the upstream tests to make sure their patch didn't > break functionality. > >> > >> # Proposal > >> Upstream could require a basic functional test for each feature or a > test for each bug fix. This test would be in the same commit as the feature > or fix causing it to "follow the code" so when it's cherry picked > downstream or into a package, the test goes with it. I believe we don't > change the test during cherry pick time because although the code > implementation may change what the feature or fix provides will not. > >> > >> Additionally, upstream should document how to run these tests easily > and have the goal of making that easy. > >> > >> This policy change would ultimately be decided by each plugin and > pulpcore separately. I don't want to make it a requirement to be part of > the github.com/pulp/ organization in any way, so if a plugin can't, or > won't, make this change that's ok, they can still host at github.com/pulp/ > . > >> > >> As a pulpcore and pulp_file maintainer, I'm proposing this policy for > both of those repos specifically. > >> > >> In terms of enforcement, I'm pitching manual enforcement by review as a > start. We can automate it later, but I hope that's a separate discussion to > focus on the policy change and keep it simple. > >> > >> # Brian's take > >> This is a non-trivial contribution requirement so we need to really > think about it. My personal take is that at this point in the 3.y release > line it's time to trade feature/fix velocity for stability. Also > backporting is about to become a regular activity, so I think we have a > practical motivation to adopt this, even though it will slow down the > future features and bug fix velocity. > >> > >> What's your perspective? > >> > >> _______________________________________________ > >> Pulp-dev mailing list > >> Pulp-dev@redhat.com > >> https://www.redhat.com/mailman/listinfo/pulp-dev > > > > > > > > -- > > Pavel Picka > > Red Hat > > _______________________________________________ > > Pulp-dev mailing list > > Pulp-dev@redhat.com > > https://www.redhat.com/mailman/listinfo/pulp-dev > > > _______________________________________________ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev > >
_______________________________________________ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
