On Thu, Sep 1, 2016 at 11:48 AM, Emilien Macchi <emil...@redhat.com> wrote: > On Thu, Aug 25, 2016 at 9:16 AM, Steven Hardy <sha...@redhat.com> wrote: >> On Wed, Aug 17, 2016 at 07:20:59AM -0400, James Slagle wrote: >>> On Wed, Aug 17, 2016 at 4:04 AM, Dmitry Tantsur <dtant...@redhat.com> wrote: >>> > However, the current gate system allows to run jobs based on files >>> > affected. >>> > So we can also run a scenario covering ironic on THT check/gate if >>> > puppet/services/*ironic* is affected, but not in the other cases. This >>> > won't >>> > cover all potential failures, but it would be of great help anyway. It >>> > should also run in experimental pipeline, so that it can be triggered on >>> > any >>> > patch. >>> > >>> > This is in addition to periodic jobs you're proposing, not replacing them. >>> > WDYT? >>> >>> Using the files affected to trigger a scenario test that uses the >>> affected composable service sounds like a really good idea to me. >> >> +1 I think this sounds like a really good idea. >> >> Now that we're doing almost all per-service configuration in the respective >> templates and puppet profiles, this should be much easier to implement I >> think so definitely +1 on giving it a go. >> > > So I would like to give an update about this. > The work has been done to have the structure in place. > I first used experimental pipeline to test the new jobs but they are > now in check pipeline, as non-voting. > > I created a multinode jobs CI matrix here: > https://github.com/openstack-infra/tripleo-ci#service-testing-matrix > I encourage everyone to have a quick look at it. > > What is happening now? > > - If you submit a patch in THT (in puppet/services/ceilometer*) it > will trigger scenario001 job (example with Telemetry). Same thing for > Puppet profiles in puppet-tripleo. Note: with Zuul v3 we'll be able to > make things more granular and specific to projects and files. We're > looking for having it! > - Since multinode-nonha initially created by slagle is now a bit > redundant with scenarios, I'll make it lighter to test basic compute > services with the less services as possible. > - I'll continue to extend the scenarios complexity and start testing > different backends (ie, ceph/rbd on scenario001 with Telemetry, etc), > like we're doing in Puppet CI: > https://github.com/openstack/puppet-openstack-integration#description > - For now, we're using pingtest to test that services actually work. I > guess it's good for now, but we also might want to consider Tempest > sometimes. I know Tempest already runs on periodic jobs, but should we > also consider it in those multinode jobs? The feedback would be > valuable though but we would have to make tempest configuration > composable, depending on the services we actually run (maybe using > discovery, etc). > > Any help and feedback on this topic is highly welcome, > -- > Emilien Macchi
I wrote a blog post so people can understand how it works: http://my1.fr/blog/scaling-up-tripleo-ci-coverage-with-scenarios/ Let me know if I need to go more in details, any feedback is welcome as usual. -- Emilien Macchi __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev