Hey everybody!
The time has come at long last, we're actually aiming at a concrete date
for rolling Zuul v3 live. \o/
tl;dr - Assuming things go well we want to do it at the PTG
We couldn't be more excited - there are a giant pile of feature requests
from folks that we've been responding with "yes, as soon as Zuul v3 is
here you can do that" - and that's not any more fun for us as it is for you.
If you have no idea what Zuul v3 is - there is a section at the end of
the email just for you.
The Plan
========
Zuul v3 itself works great - we're using it to run tests on Zuul itself.
The biggest bulk of work at the moment is ... job content.
We know that we can do a one-time migration of job content - because we
do it every day for our current jobs (they are translated from
jenkins-job-builder to ansible on the fly in Zuul 2.5 today) However,
that translated content is ugly - and Zuul v3 allows us to make a bunch
of things nicer. We'd like to have as many of our jobs translated to
'nice' versions for the cutover as possible so that as people copy-pasta
jobs they've got good content to work from.
We're currently working on writing v3-native versions of the most common
jobs. This includes the tox-based unittest jobs and common things like
tarball creation, pypi and docs publication. Next up is getting
devstack-gate jobs done. Between tox and devstack-gate - we should have
the majority of our jobs covered, so any outliers we couldn't
auto-translate into something nice and new will have good examples to
work from for followup cleaning.
devstack-gate question
----------------------
There is an open question for devstack-gate. Namely - given the
mechanism of how it works with defining a bunch of environment variables
and then defining a shell function - it's currently unknown if we'll be
able to fully auto-translate all of the existing project-specific
special devstack-gate jobs, or whether we'll need to just migrate them
all to things that look pretty similar to how they look now and have a
few clear examples people can follow as they feel like updating things.
We should know more about that soon as we're working devstack-gate jobs
this week.
Go Live
-------
On the Saturday and Sunday immediately preceeding the PTG, we will run a
few trial cutovers. That is, we'll do a short zuul downtime, run job
migration scripts, turn zuul v3 on, run for a little bit to see how
things go, then turn it off and turn v2 back on. This will let us
double-check the migration during a time that's likely slower than
normal for any last minute dead-in-the-water issues.
First thing Monday morning of the PTG, we'll throw the switch for real.
We'll be around all week in a sort of "open office hours" form to help
folks if there are any questions, if there are any job execution edge
cases that need to be worked through - or if people are now excited by
all the new features and have questions about how to make use of them.
We'll essentially be planning on our PTG to be all about making sure any
questions any of you have can be fielded with plenty of bandwidth.
It's possible we won't be ready. We're pretty optimistic, but this is a
big move, and something unforseen could come up between now and then.
We're not going to throw the switch without being confident that things
will go well just to meet a date. If we decide to cancel the cutover,
we'll send an announcement to the list
Between Now and Then
====================
By and large the leadup to this should have very little impact on folks.
There are some jobs that use scripts that are pre-baked into images that
come from the jenkins/scripts directory in project-config. Those will no
longer be used in Zuul v3 and will be migrated into Zuul v3 job
definitions. Those scripts are currently under a soft-freeze, meaning
that we'd prefer to not make any changes to them - but if a change is
needed we need to ensure it gets forward-ported to the zuulv3 version.
Closer to the PTG we'll put those scripts under a hard-freeze. They're
all tricky to deal with anyway because they get baked in to images, so
we're not expecting a ton of impact from that.
We may need to freeze all of project-config in the days leading up to
the cutover, but since those are the days leading up to the PTG it's
unlikely that should be a super active time for that repo.
Third Party CI
==============
Any Third Party CI that is currently using Zuul should continue using
Zuul v2 as you are today. Once we go live for OpenStack we'll be in a
position to start discussing upgrade plans for 3rd Party operators. The
goal is to migrate all of you - but we don't want folks to start
tackling that until we're happy we've got automation and documentation
worked out. After the PTG will be a great time to start discussing what
that plan wants to look like.
Wait - what the heck is Zuul v3 and why are we doing this?
==========================================================
We've started a documentation page on this topic in the Infra Manual.
It is short right now, but as we get closer to the cutover, we will
continue to update this page with information about the migration:
https://docs.openstack.org/infra/manual/zuulv3.html
Zuul v3 is the new version of Zuul that incorporates a ton of learning
about pain points from the last several years. It adds in-repo config,
native multi-node testing, ansible-based job definitions, full DAG job
dependencies and support for integration with systems OpenStack doesn't
otherwise use such as GitHub.
Zuul v3 is designed to be run by not-Infra. We're happy that Zuul has
been useful for other people over the past few years, especially our
Third-Party CI community - but up until v3 the OpenStack project was
always consideration #1 and other users were best-effort.
With v3 we formally consider users who are not the OpenStack Project to
be first-class users. We have recently added 3 core reviewers to Zuul
who are not involved with running OpenStack's Zuul. (one runs a Zuul v3
at BMW and two run a Zuul v3 for the IBM BonnyCI project)
We have a bunch of work teed up for post PTG rollout to add more support
for a wider array of usecases. The OpNFV Infra team has an important
feature request that Tristan from the RH Software Factory team has
started implementing. We have planned support for container execution,
both on a single-container and a COE (kubernetes/mesos) level, as well
as additional triggers/reporters such as FedMsg as used by Fedora and
Debian projects.
More information on the background of Zuul v3 can be found at:
https://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html
and
http://lists.openstack.org/pipermail/openstack-dev/2017-February/113065.html
Docs for Zuul v3 itself can currently be found at:
https://docs.openstack.org/infra/zuul/feature/zuulv3/
As usual, you can find us in #openstack-infra and in #zuul.
Thanks!
Monty
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
