No disagreement with anything you say below, Matt. More testing of all kinds, including unit tests, should be a priority.
Cheers, jay On Wed, Feb 16, 2011 at 11:37 PM, Matt Dietz <[email protected]> wrote: > These are all great points Jay. > > I'd like to re-echo the comment about unit tests. Obviously they aren't > the panacea, but they can protect against some of the dumber errors that > have made their way into trunk. One particular bug stopped one developer > on my team dead in his tracks, and it ended up being a semi-colon in place > of a colon. There's a lot of utility in simply "exercising" code... > > I think we may want to consider everyone's favorite topic of code coverage > as well in all of this. Specifically, we may want to take note of code > coverage on any given merge, and if subsequent merges reduce that number, > we throw a fit/reject the patch. I know that won't be a popular solution, > but it would definitely put a stop to the lack of unit tests. > > On 2/16/11 4:27 PM, "Jay Pipes" <[email protected]> wrote: > >>Hey all, >> >>It's come to my attention that a number of folks are not happy that >>Nova's trunk branch (lp:nova) is, shall we say, "less than stable". :) >> >>First, before going into some suggestions on keeping trunk more >>stable, I'd like to point out that trunk is, by nature, an actively >>developed source tree. Nobody should have an expectation that they can >>simply bzr branch lp:nova and everything will magically work with a) >>their existing installations of software packages, b) whatever code >>commits they have made locally, or c) whatever specific >>hypervisor/volume/network environment that they test their local code >>with. The trunk branch is, after all, in active development. >> >>That said, there's *no* reason we can't *improve* the relative >>stability of the trunk branch to make life less stressful for >>contributors. Here are a few suggestions on how to keep trunk a bit >>more stable for those developers who actively develop from trunk. >> >>1) Participate fully in code reviews. If you suspect a proposed branch >>merge will "mess everything up for you", then you should notify >>reviewers and developers about your concerns. Be proactive. >> >>2) If you pull trunk and something breaks, don't just complain about >>it. Log a bug immediately and talk to the reviewers/approvers of the >>patch that broke your environment. Be constructive in your criticism, >>and be clear about why the patch should have been more thoroughly or >>carefully reviewed. If you don't, we're bound to repeat mistakes. >> >>3) Help us to write functional and integration tests. It's become >>increasingly clear from the frequency of breakages in trunk (and other >>branches) that our unit tests are nowhere near sufficient to catch a >>large portion of bugs. This is to be expected. Our unit tests use >>mocks and stubs for virtually everything, and they only really test >>code interfaces, and they don't even test that very well. We're >>working on adding functional tests to Hudson that will run, as the >>unit test do, before any merge into trunk, with any failure resulting >>in a failed merge. However, we need your help to create functional >>tests and integration tests (tests that various *real* components work >>together properly). We also need help writing test cases that ensure >>software library dependencies and other packaging issues are handled >>properly and don't break with minor patches. >> >>4) If you have a specific environment/setup that you use (Rackers and >>Anso guys, here...), then we need your assistance to set up test >>clusters that will pull trunk into a wiped test environment and ensure >>that a series of realistic calls to the Nova API are properly handled. >>I know some of you are working on getting hardware ready. We need help >>from the software teams to ensure that these environments are >>initialized with the exact setups you use. >> >>The more testing we fire off against each potential merge into trunk, >>and the more those tests are hitting real-life deployment >>environments, the more stable trunk will become and the easier your >>life as a contributor will be. >> >>Thanks in advance for your assistance, and please don't hesitate to >>expand on any more suggestions you might have to stabilize trunk. >> >>-jay >> >>_______________________________________________ >>Mailing list: https://launchpad.net/~openstack >>Post to : [email protected] >>Unsubscribe : https://launchpad.net/~openstack >>More help : https://help.launchpad.net/ListHelp > > > > Confidentiality Notice: This e-mail message (including any attached or > embedded documents) is intended for the exclusive and confidential use of the > individual or entity to which this message is addressed, and unless otherwise > expressly indicated, is confidential and privileged information of Rackspace. > Any dissemination, distribution or copying of the enclosed material is > prohibited. > If you receive this transmission in error, please notify us immediately by > e-mail > at [email protected], and delete the original message. > Your cooperation is appreciated. > > _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : [email protected] Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
