Apologies for top posting. Just wanted to say +1 This all makes sense to me.
Vish On May 3, 2012, at 4:08 AM, Mark McLoughlin wrote: > Hey, > > We discussed this during the "baking area for features" design summit > session. I found that discussion fairly frustrating because there were > so many of us involved and we all were either wanting to discuss > slightly different things or had a slightly different understanding of > what we were discussing. So, here's my attempt to put some more > structure on the discussion. > > tl;dr - subsystem branches are managed by trusted domain experts and > feature branches are just temporary rebasing branches on personal github > forks. We've got a tonne of work to do figuring out how this would all > work. We should probably pick a single subsystem and start with that. > > ... > > Firstly, problem definition: > > - Nova is big, complex and has a fairly massive rate of churn. While > the nova-core team is big, there isn't enough careful review going > on by experts in particular areas and there's a consistently large > backlog of reviews. > > - Developers working on features are very keen to have their work > land somewhere and this leads to half-finished features being > merged onto master rather than developers collaborating to get a > feature to a level of completeness and polish before merging into > master. > > Some assumptions about the solution: > > - There should be a small number of domain experts who can approve > changes to each of major subsystems. This will encourage > specialization and give more clear lines of responsibility. > > - There should be a small number of project dictators who have final > approval on merge proposals, but who are not expected to review > every patch in great detail. This is good because we need someone > with an overall view of the project who can make sure efforts in > the various subsystems are coordinated, without that someone being > massively overloaded. > > - New features should be developed on a branch and brought to a level > of completeness before being merged into master. This is good > because we don't want half-baked stuff in master but also because > it encourages developers to break their features into stages where > each stage of the work can be brought to completion and merged > before moving on to the next stage. > > - In essence, we're assuming some variation of the kernel distributed > development model. > > (FWIW, my instinct is to avoid the kernel model on projects. Mostly > because it's extremely complex and massive overkill for most > projects. Looking at the kernel history with gitk is enough to send > anyone screaming for the hills. However, Nova seems to be big > enough that we're experiencing the same pressures that drove the > kernel to adopt their model) > > Ok, what are "subsystem branches" and how would they work? > > - Subsystem branches would have a small number of maintainers who can > approve a change. These would be domain experts providing strong > oversight over a particular area. > > (In gerrit, this is a branch with a small team or single person who > can +1 approve a review) > > - Project dictators don't need to do detailed reviews of merge > proposals from subsystem maintainers. The dictator's role is mostly > just to sign off on the merge proposal. However, the dictator can > comment in the proposal on things which could have been done better > and the subsystem maintainer should take note of these comments and > perhaps retroactively fix them up. Ultimately, though, the dictator > can have exercise a veto if the merge proposal is unacceptable or > if the subsystem maintainer is consistently making the same > mistakes. > > - It would be up to the project dictators to help drive patches > through the right subsystem branches - e.g. they might object if > one subsystem maintainer merged a patch that inappropriately cut > into another subsystem or they might refuse to merge a given patch > into the main branch unless it went through the appropriate > subsystem branch. > > (In gerrit, this would mean a small team or single person who can > +1 approve merge proposals on master. They would -1 proposals > submitted against master which should have been submitted against a > subsystem branch.) > > - Subsystem branches might not necessarily be blessed centrally. It > might be a case that anyone can create such a branch and, over > time, establish trust with the project dictators. Subsystem > branches would come and go. This is the mechanism by which > subsystem maintainership is transferred between people over time. > > (In gerrit, this means people need to easily be able to create > their own branches) > > (What's more difficult to imagine in gerrit is how a new, potential > subsystem maintainer comes along, starts hoovering up patches into > her branch and submitting them in batches. Where does she hoover > them up from and how does she say "I've merged this into my branch, > don't merge it via another branch") > > - Bisectability remains important. Subsystem maintainers don't merge > broken commits into their subsystem branch and the project > dictators can enforce this using their veto. It is not good enough > for subsystem maintainers to consistently merge broken commits into > their branch, fix it up with a later commit and include both > commits their merge proposals. > > (I don't think we'd use Jenkins to enforce this, but subsystem > maintainers might use it as a tool to help them catch issues. So, > the full set of gating tests would only gate merges into master but > subsystem branches might choose to gate merges into their branch on > the unit tests. Subsystem maintainers might also use Smokestack to > pre-gate merge proposals to the subsystem branch) > > - Subsystem branches would not rebase unless the project dictator > outright rejects a merge request from the subsystem branch (i.e. > "I'm not merging commit abcdef0! Fix it and rebase!"). This means > the subsystem maintainer will need to regularly (probably only when > there are conflicts to be dealt with) merge master back into the > subsystem branch. > > - Plausible subsystem branches are e.g.: > > - OpenStack APIs > - EC2 API > - virt > - libvirt driver > - xenapi driver > - vmware driver > - networking > - volumes > - scheduler > - RPC > > Deciding which areas make sense as a subsystem branch is > non-trivial. > > Should there be a "DB" subsystem? Probably not, because that would > mean every new feature needs to come through this branch or, > alternatively, the DB maintainer would need to accept DB schema > additions without the context of how it's being used higher up the > stack. > > Ok, so why does it make sense to have an "OpenStack APIs" > subsystem? Don't all features affect that branch too? Well, maybe, > but the APIs really do need strong oversight. Perhaps we can be > confident that we can add e.g. a new scheduler feature through the > scheduler branch and then later merge any API additions through the > APIs branch. > > And how about feature branches? > > - Feature branches are relatively short-lived (i.e. weeks or months > rather than years) branches for a specific feature. They are a > mechanism for developers to work on a patch series in the open until > the feature is complete enough to be merged into a subsystem branch > or master. > > (I'm not sure gerrit is right for this. Why not just do it in > folk's github forks? I think all people are looking for is for > people to be more aware of feature branches. How about if you put > details of your feature branch in the blueprint for the feature?) > > (If not using gerrit, can developers configure Jenkins to CI their > branch? Or is Smokestack the right tool?) > > - Feature branches rebase, do not contain merge commits and each > commit on the branch is functional, bisectable and self-contained. > > - When a feature branch is ready to be merged into a subsystem > branch, the patch series is submitted for review. The subsystem > maintainer will likely require changes to individual patches and > these changes would be made on the feature branch and squashed back > into the appropriate individual patch. > > (Ideally gerrit's "topic review" feature will get upstream and > we'll use that. This would mean that a patch series could be > proposed for review as a single logical unit while still keeping > individual patches as separate commits) > > - Because feature branches rebase, active day-to-day collaboration > with others is difficult. You certainly can't have multiple people > rebasing the same branch, that way lies madness. > > There are ways to have multiple people work actively on the same > rebasing branch e.g. > > http://blogs.gnome.org/markmc/2011/02/26/git-rebasing-cont/ > > but, ultimately, feature branches are going to be owned by a single > person who might incorporate patches from others. > > (Incorporating the work of others, rebasing and squashing means a > patch might have multiple contributors but only one author listed in > git. That makes CLA enforcement impossible, but we should drop the > CLA in favour of the kernel-like Signed-off-by: tag. See this > discussion: https://lists.launchpad.net/openstack/msg06544.html ) > > - One option for longer-lived, active collaboration is for a subsystem > maintainer to create a feature branch and review the work as it is > ongoing. The idea being that the subsystem maintainer commits to > not requiring the feature branch to be rebased before it is merged > into the subsystem branch. > > Cheers, > Mark. > _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp