@dkliban, I think that is a good solution. Since it's currently using the latest GA version of core, all we need to do is extend the current job as you describe. I rewrite the issue [0] to reflect that last part of the work. This is on the current sprint at NEW so I imagine it will get picked up soon. If we should do something different feel free to rewrite.
[0]: https://pulp.plan.io/issues/2751 -Brian On Mon, Jun 5, 2017 at 10:45 AM, Dennis Kliban <dkli...@redhat.com> wrote: > What if we ran our plugin unit tests against both the latest GA build and > nightly build of core? > > If the tests pass with the GA version, the job is marked as successful. If > not, core packages are upgraded to the latest nightly and unit tests are > run again. If the unit tests fail again, the job is marked as failed. If > the unit tests pass with the latest nightly, the job is marked as > successful. > > On Sun, Jun 4, 2017 at 12:03 PM, Brian Bouterse <bbout...@redhat.com> > wrote: > >> After thinking about this more, I realized that for the remainder of >> Pulp2 at least, we need to have the plugin unittest runner test against the >> nightly version of core and not the latest GA. Using the GAs won't work >> because not only is the 'I need unreleased code from platform' a problem >> with the PR that needs it but also a problem for all subsequent PRs after >> its merged. That second part makes using GA core as the basis for plugin >> testing probably a non-starter. Assuming that, the next step for 2751 is to >> update the GA urls to be the "stable nightly" URLs. >> >> We also need to look into the nightlies to check on their reliability. In >> theory, each night an "unstable nightly" of core gets built nightly in >> Jenkins, tested with pulp-smash, and if all tests pass it gets "promoted" >> to a separate URL for "stable nightlies". Let me know if we should move >> this to another thread, but I've got these questions about nightlies. >> >> 1. Who investigates when the "unstable nightly" fails to build? >> 2. Who investigates when a "unstable nightly" fails to be promoted to a >> "stable nightly" due to pulp-smash failures? >> 3. Who is in charge of maintaining these Jenkins jobs over time and are >> they currently maintained? >> 4. Who is in charge of managing the directory structure on >> repos.fedorapeople.org? >> 5. Where are the docs on ^? >> >> With Pulp3 I think we can switch to using the latest GA as the basis for >> plugin testing which would be better in several ways. >> >> -Brian >> >> On Fri, Jun 2, 2017 at 5:06 PM, Brian Bouterse <bbout...@redhat.com> >> wrote: >> >>> That is a good point, and one we are giving some thought to through >>> convo on #pulp-dev and the issue [0]. The case of a plugin needing an >>> unreleased change from core would fail with this change. It's a tradeoff >>> though because if we go with nightlies as the version of core that is used, >>> whenever the nightlies break, the unittest PR runners also will, which has >>> been a reliability issue with the plugin unittest runner for a while. >>> >>> I wrote some on the issue about it, but I see the 'plugin needs >>> unreleased code from core' as a special case, not a normal case. It used to >>> be common, but it's getting less common, which is good, because >>> contributing to a plugin should not involve changes to the core as the >>> norm. It will happen from time to time, so we can handle the special case, >>> specially by running the unittests locally with the necessary unreleased >>> version of platform and posting the results as evidence that its safe to >>> merge. >>> >>> [0]: https://pulp.plan.io/issues/2751 >>> >>> On Fri, Jun 2, 2017 at 4:43 PM, Michael Hrivnak <mhriv...@redhat.com> >>> wrote: >>> >>>> What about cases where a plugin wants to use something that's new in >>>> the unreleased core? The master branch of a plugin will usually be released >>>> with the master branch of the core in the next 2.y release for example. >>>> That seems like a normal scenario; is it facilitated somehow with this >>>> testing change? >>>> >>>> On Fri, Jun 2, 2017 at 4:33 PM, Dennis Kliban <dkli...@redhat.com> >>>> wrote: >>>> >>>>> In an effort to resolve issue 2751[0], I updated the PR builder job >>>>> for plugins. Each PR for a plugin will now be tested against the latest >>>>> stable release of the core found here[1]. This will ensure that the plugin >>>>> is maintaining compatibility with the latest stable core and that we are >>>>> only testing one change at a time. >>>>> >>>>> >>>>> [0] https://pulp.plan.io/issues/2751 >>>>> [1] https://repos.fedorapeople.org/pulp/pulp/stable/latest/ >>>>> >>>>> >>>>> -Dennis >>>>> >>>>> _______________________________________________ >>>>> Pulp-dev mailing list >>>>> Pulp-dev@redhat.com >>>>> https://www.redhat.com/mailman/listinfo/pulp-dev >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> Michael Hrivnak >>>> >>>> Principal Software Engineer, RHCE >>>> >>>> Red Hat >>>> >>>> _______________________________________________ >>>> Pulp-dev mailing list >>>> Pulp-dev@redhat.com >>>> https://www.redhat.com/mailman/listinfo/pulp-dev >>>> >>>> >>> >> >
_______________________________________________ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev