I agree. I propose we always keep lp:nova (or lp:<project>) stable, and instead create "trunks" like lp:nova/testing that all the test/regression systems can be run against. This is pretty similar to how we did things with Drizzle, a commit would bounce down the line, finlly landing in lp:<project> when it was verified.
-Eric On Thu, Feb 24, 2011 at 10:41:02AM -0800, Justin Santa Barbara wrote: > Sounds like we're actually in agreement, just disagreeing about the > topology of the branches! > My preferred topology is actually a series of increasingly stable trunks, > each of which has passed each level of the smoke tests (one platform, > multi platform, torture tests). *The nice thing about this is that then a > release can simply be seen as the next level up in the stability > hierarchy, with a manual pull. *I think Google Chrome has a particularly > sophisticated process which we could learn from: > http://techcrunch.com/2011/01/11/google-chrome-release-cycle-slideshow/ > I still think we should start simple with two trunks - stable and > unstable. *But I'll be happy if with per-branch testing also - that just > seems a lot harder to me! > Justin > > On Thu, Feb 24, 2011 at 10:24 AM, Trey Morris <trey.mor...@rackspace.com> > wrote: > > Instead of an unstable trunk, i think code should just be better vetted > before it lands in the trunk. If the difference between trunk and your > proposed unstable trunk is a set of automated tests, then those tests > can just as easily be run on a LP branch before it gets into current > trunk. We just need the tests. No matter how many stability levels of > trunk we have, without better tests we can't guarantee stability. > > On Thu, Feb 24, 2011 at 11:15 AM, Justin Santa Barbara > <jus...@fathomdb.com> wrote: > > Hi Jay, > I couldn't agree more. *I had another bug come up yesterday on another > of my patches (I know - not a good day for me!) where I again broke > the OpenStack API by requiring the metadata attribute. > In this case, it was missed by the unit tests. *I believe I was always > passing metadata, so simply missed the real world case. *Here's the > bug report: > https://bugs.launchpad.net/nova/+bug/724143 > This brings up a number of points though: > On testing > 1. The bug reporter apparently knows how to program, but instead of > us getting a test case which we could immediately use, we got a > Ruby test case. *I think we should do whatever we can so that > people that are moderately comfortable with code also feel > comfortable submitting a failing test case that we can use. *I > think this means having some version of an API client, or even a > Reference-Implementation API client, in the source tree, and using > it. > 2. It's not clear to me how we deal with unit test cases that are > failing - cases that represent a found bug that is not yet fixed. > *Maybe it should be submitted on a bugNNN branch, with failing > tests, and then whoever works on the bug can branch from it? *(Of > course, we'd be lucky if all bugs went like this, but it does > happen. *I often find myself fixing tangential issues, and I'd > like to know the 'right thing to do' is) > 3. On this particular metadata bug, it was my fault. *I therefore > submitted a very rapid hotfix which just fixes the issue. *I then > coded up unit tests that use the OpenStack API, and did in fact > hit the issue naturally, and then included my fix which resolved > it. *I submitted the 'full patch' as a separate branch. > 4. This 'full patch' branch includes unit tests that bring up the > OpenStack API and various services in-process, and runs tests just > like a user. *It would have caught the metadata issue.* > 5. The hope is that we can reuse the same tests as smoke tests, by > simply tweaking them to work against real services instead of > bringing up in-process stub-services. *These could be (some of) > the smoke tests in Hudson. > 6. I'd hope that we could have two levels of these smoke tests, one > that runs on a single configuration (e.g. KVM, OpenISCSI, Glance); > another that runs a matrix of configurations (and might take an > hour or more to run) > 7. Ideally we'd have a torture test that would run overnight and be > randomized and try to find obscure bugs, even if issues found are > not necessarily repeatable in the way that non-randomized tests > are. > We need an unstable trunk: > 1. In general, it seems that our end-users are using trunk for > unreleased functionality and treating it as if it were released. > *I don't think we should be encouraging that, because I know I'll > make more mistakes in future and some of them will make it past > the reviewers' defensive line into trunk; it's also simply not > realistic to require reviewers to review every combination - e.g. > how can a reviewer really review my HP SAN patch without an HP > SAN? *There will be issues in trunk, and if we have to revert them > rather than just fixing them it will slow us down. *The current > situation is bad for our users and bad for developers.* > 2. One way we could keep everyone happy is by using our test suite to > auto-merge from an 'unstable trunk' into 'stable trunk', only once > code passes tests. *Commits would initially merge into 'unstable > trunk', and we would try to keep that branch moving forwards > rather than reverting things that go wrong. *Of course, > maintaining a good 'stable trunk' relies on having good tests, but > I think we're getting there. *It's also great incentive to write > good smoke tests. > 3. Jay: I believe you've done this to great success on the Drizzle > project? > Justin > > On Thu, Feb 24, 2011 at 6:13 AM, Jay Pipes <jaypi...@gmail.com> wrote: > > Hi all, > > I'd like to bring up an alternate reason why it was approved and > subsequently reverted. > > The test cases for the OpenStack API (and much of the EC2 API) > assume > way too many things and mock out too many things. In addition, since > there are zero smoketests for the OpenStack API, there were no > functional tests that would have *immediately* highlighted this > problem (and many other recent EC2 vs OS API problems). > > In other words, sure, we should revert the patch to "fix things", > however the priority should *not* be on refactoring the auth API or > the way the auth layer in Nova is handled. The priority should be on > writing a smoketest for the OpenStack API so that we can link it > into > Hudson and these types of issues can be automatically caught. > -jay > On Wed, Feb 23, 2011 at 10:03 PM, Paul Voccio > <paul.voc...@rackspace.com> wrote: > > Justin, > > I think you hit upon the reason of why I think it was approved and > reverted. > > Because it hadn't been talked about in a blueprint or a mail sent > to the > > list (I think I'm up to date on the threads) and a patch landed > means other > > alternatives weren't considered before pushing it through to begin > with. I > > think we're all open to talking about how to better the auth > system and make > > improvements. Dragon has already discussed some alternatives and > suggestions > > on the BP page below. I think this is the right way to continue > the dialog > > and we all can agree on a good way forward. > > I'm confident we can figure it out. > > If I missed a conversation, my apologies. > > pvo > > From: Vishvananda Ishaya <vishvana...@gmail.com> > > Date: Wed, 23 Feb 2011 18:19:41 -0800 > > To: Justin Santa Barbara <jus...@fathomdb.com> > > Cc: <openstack@lists.launchpad.net> > > Subject: Re: [Openstack] Should the OpenStack API re-use the EC2 > > credentials? > > > > Hey Justin, > > Does it make any difference that the way the auth is > (theoretically) > > supposed to work with the os api is that the user gets an auth > token from an > > external auth server and then uses username / authtoken to > actually contact > > the api? *I think it is just faked out right now to use the > access_key > > instead of doing external auth, but I think the reason it works > like it does > > is because the plan was to switch to external auth eventually. > > Vish > > On Feb 23, 2011, at 5:56 PM, Justin Santa Barbara wrote: > > > > I previously fixed OpenStack authentication so it would use the > same > > credentials as EC2. *This bugfix was just reverted, because it > caused > > OpenStack API users to have to enter in different credentials > (sorry!), but > > primarily because it hadn't been*discussed*on the mailing list. > *So here > > goes! > > Here's a > > > > blueprint:*https://blueprints.launchpad.net/nova/+spec/authentication-consistency > > Here's an overview of the problem: > > EC2 uses an (api_key, api_secret) pair. *Post-revert, OpenStack > uses the > > api_key(!) as the password, but a different value entirely as the > username: > > (username, api_key). *The bugfix made it so that both APIs used > the EC2 > > credentials (api_key, api_secret) . *This did mean that anyone > that had > > saved the 'bad' OpenStack credentials was unable to continue to > use those > > credentials. *I also overlooked exporting the updated credentials > in novarc > > (though a merge request was pending). > > I actually thought originally that this was a straight-up bug, > rather than a > > design 'decision', so I should definitely have flagged it better. > *Again, > > sorry to those I impacted. > > As things stand now, post-revert, this is probably a security > flaw, because > > the EC2 API does not treat the api_key as a secret. *The EC2 API > can > > (relatively) safely be run over non-SSL, because it uses > signatures instead > > of passing the shared secret directly. > > This is also not very user-friendly. *Post-revert, an end-user > must know > > whether any particular cloud tool uses the EC2 API or the > OpenStack API, so > > that they can enter in the correct pair of credentials. *That > doesn't seem > > like a good idea; I think there should be one set of credentials. > > > > There is some discussion about the idea of having the api_key be > > user-friendly. *I don't think it buys us anything, because the > api_secret is > > still going to be un-friendly, but I have no objection as long as > it is does > > in a way that does not break existing users of the EC2 API. > > I propose that: > > *(1) the OpenStack API and EC2 credentials should be the same as > each other > > (whatever they are) for the sake of our collective sanity and > > *(2) we have to change the current configuration anyway for > security > > reasons. > > *(3) We should not change the EC2 credentials, because we've > shipped the EC2 > > API and our users have an expectation that we won't break them > without good > > reason, so > > *(4) we must change the credentials for users of the (non-shipped) > OpenStack > > API. > > Estimated user impact: I believe there are two people that will be > affected, > > and it will take them ~1 minute each, so total impact ~2 minutes. > > The longer we delay fixing this, the more people we break and the > bigger the > > impact. *It seems that we have no choice but to do a > > non-backwards-compatible authentication change, but I believe this > is OK at > > the moment because the OpenStack API is not yet stable/released - > i.e. we > > can still make fixes without worrying about backwards > compatibility shims. > > We're not in "The Old New Thing" land yet :-) > > > > > > As an aside, I am very unhappy about the way this revert was > pushed through > > by Rackspace team-members, seemingly without much consideration of > > alternatives. *Perhaps we should consider changing from needing > two > > core-approves, to needing one Rackspace core-approve and one > non-Rackspace > > core-approve. > > > > Justin > > > > > > _______________________________________________ > > Mailing list: https://launchpad.net/~openstack > > Post to ****: openstack@lists.launchpad.net > > Unsubscribe : https://launchpad.net/~openstack > > More help **: https://help.launchpad.net/ListHelp > > > > _______________________________________________ Mailing list: > > https://launchpad.net/~openstack Post to : > openstack@lists.launchpad.net > > Unsubscribe : https://launchpad.net/~openstack More help : > > https://help.launchpad.net/ListHelp > > > > Confidentiality Notice: This e-mail message (including any > attached or > > embedded documents) is intended for the exclusive and confidential > use of > > the > > individual or entity to which this message is addressed, and > unless > > otherwise > > expressly indicated, is confidential and privileged information of > > Rackspace. > > Any dissemination, distribution or copying of the enclosed > material is > > prohibited. > > If you receive this transmission in error, please notify us > immediately by > > e-mail > > at ab...@rackspace.com, and delete the original message. > > Your cooperation is appreciated. > > > > _______________________________________________ > > Mailing list: https://launchpad.net/~openstack > > Post to * * : openstack@lists.launchpad.net > > Unsubscribe : https://launchpad.net/~openstack > > More help * : https://help.launchpad.net/ListHelp > > > > > > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to * * : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help * : https://help.launchpad.net/ListHelp > > Confidentiality Notice: This e-mail message (including any attached or > embedded documents) is intended for the exclusive and confidential use of the > individual or entity to which this message is addressed, and unless otherwise > expressly indicated, is confidential and privileged information of Rackspace. > Any dissemination, distribution or copying of the enclosed material is > prohibited. > If you receive this transmission in error, please notify us immediately by > e-mail > at ab...@rackspace.com, and delete the original message. > Your cooperation is appreciated. > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
