Hi Jim,
A couple of questions for you:
1) You mentioned how to coordinate changes between glance and nova but what
about devstack. Does that same process apply to devstack as well? For example
if there were a configuration file change (api-paste changes often and can
cause failures) would I push the required change to devstack first and then
nova? Or would we need to make devstack handle multiple versions of the
configuration files?
2) Where are the devstack instances running (public cloud, private Openstack
cloud, etc.). If the public cloud is down for maintenance does that mean code
can't land? Are there any plans to run this on a private or OpenStack backed
cloud? Regardless of what we are doing is there a backup plan in place so that
code can land?
3) Are there any plans on making this run on branches in merge prop (before we
approve them)? I would love to know that devstack passes ahead of time before I
approve a branch.
Thanks,
Dan
-----Original Message-----
From: "James E. Blair" <cor...@inaugust.com>
Sent: Thursday, December 29, 2011 5:51pm
To: "OpenStack Mailing List" <openstack@lists.launchpad.net>
Subject: [Openstack] Integration test gating on trunk
Hi,
A few weeks ago, I wrote about turning on an integration test gating job
for the stable/diablo branch. That's been running for a while now, and
during that time with help from Anthony Young and Jesse Andrews, we've
been able to address the issues we saw and make the job fairly reliable.
At the last design summit, we agreed that we should gate trunk
development of at least nova and its immediate dependencies on some kind
of integration test. The biggest change this introduces to developer
workflow is how to handle changes that affect more than one project. At
the design summit, it was decided that such changes should be authored
so that the system continues to function as each is merged in order. In
other words, if you need to modify nova and glance, you might make a
change to nova that accepts old and new behaviors from glance, then
change glance.
The job we've been developing uses devstack to set up nova, glance, and
keystone, and then runs the relevant exercise.sh tests. Obviously
that's not a lot of testing, but it does at least ensure that nova can
perform its basic functions, which, again, was an important milestone
identified at the summit. Once tempest is ready for this, we'll start
using it.
At this point, I believe the testing infrastructure is stable enough for
us to turn on gating for all branches of nova, glance, and keystone
(also python-novaclient, devstack, and openstack-ci, which are involved
in the setup and running of the tests).
I would not be surprised if we run into some problems. We might see
transient network errors in the test setup, in which case you can just
re-trigger the job (you can vote "Approved" again), and we can see if
there's some caching or local mirroring we can do to reduce that risk.
We might encounter non-deterministic behavior in the setup and running
of OpenStack, in which case it would be best to treat that as a bug in
devstack or the affected component and improve the software. I think
that kind of problem is the sort of thing that our CI system should be
uncovering, so even though it's annoying if it affects landing a patch
you're working on, I think it's a net positive to the effort overall.
Also, we just might catch real bugs.
Having said that, the Jenkins job has been running in silent mode on
master for several days with few false errors. My feeling from the
design summit was that it was generally understood there would be a
shakedown period, and people are willing to accept some risk and some
extra work for the benefits an integration test gating job will brink.
I think we're at that point, so I'd like to turn this job on Tuesday,
January 3rd.
-Jim
_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
