On Tue, Aug 08, 2017 at 09:57:24AM -0500, Monty Taylor wrote: > Heya, > > I started working on the Zuul Migration script - largely because we're > starting to have more mappings of old job to new job than I can keep in my > head and I wanted to be able to write them down as we make them. > > https://review.openstack.org/#/q/topic:zuul-v3-migration > > I doesn't do much yet - but I did add a zuul-v3 job that runs the migration > on project-config and collects the results so we should be able to examine > that as we work on it. > > I've run in to a conceptual issue already that I'd like feedback on. > > In dealing with project-templates, we have a bit of an issue. Take this > snippet from layout.yaml: > > project-templates: > - name: loci-jobs > check: > - 'gate-{name}-ubuntu-xenial' > gate: > - 'gate-{name}-ubuntu-xenial' > > We have two choices of how we can do transforms of things we don't > explicitly map. We can expand them all and make completely equivilent jobs, > so that we'd generate the following jobs: > > - job: > name: gate-loci-cinder > > - job: > name: gate-loci-glance > > If we do that, then we can't really migrate the templates and would have to > put expanded templates into the resulting project pipeline config, so we'd > have: > > - project: > name: openstack/loci-cinder > check: > - gate-loci-cinder > gate: > - gate-loci-cinder > > this is the safest - we KNOW that we can do this migration today and the > jobs will all work as today - but the resulting config will be uglier. This > is also a data loss as we'll lose ALL of our current templates. > > We could also depend on the mapping construct and define a mapping for these > loci jobs: > > - old: gate-{name}-ubuntu-xenial > name: test-loci > > in which case we could make: > > - job: > name: test-loci > > - project-template: > name: loci-jobs > check: > - test-loci > gate: > - test-loci > > - project: > name: openstack/loci-cinder > templates: > - loci-jobs > > But doing this will require a much more careful audit of our output - we > cannot be sure that cases we haven't looked at will work and we essentially > need to hand-verify every project-template to make sure the jobs it contains > don't have edge-conditions we need to map. > > As a third option - we could auto-expand {name} in project-templates to the > name of the project template and generate jobs that way: > > - job: > name: gate-loci-jobs > > - project-template: > name: loci-jobs > check: > - gate-loci-jobs > gate: > - gate-loci-jobs > > - project: > name: openstack/loci-cinder > templates: > - loci-jobs > > This is automatic and doesn't have data loss - but might be a little > confusing. It also means for some of our templates we'll have duplicate jobs > we don't need. {name}-tarball, for instance, shows up in many template. Now > - we'll have a mapping for {name}-tarball - but for jobs that we don't have > explicit mappings for and that do show up in multiple templates it might be > weird.
I am fine with option 3, I don't expect it to be 100% perfect but should give us a good base to start refactoring things. I see this as a 1 time cost, that once import for openstack-infra, we can start work on reducing duplicate data. > > Thoughts? > > _______________________________________________ > OpenStack-Infra mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra _______________________________________________ OpenStack-Infra mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
