On Thu, Aug 7, 2014 at 1:26 PM, Joe Gordon <joe.gord...@gmail.com> wrote: > > > > On Tue, Aug 5, 2014 at 9:03 AM, Thierry Carrez <thie...@openstack.org> > wrote: >> >> Hi everyone, >> >> With the incredible growth of OpenStack, our development community is >> facing complex challenges. How we handle those might determine the >> ultimate success or failure of OpenStack. >> >> With this cycle we hit new limits in our processes, tools and cultural >> setup. This resulted in new limiting factors on our overall velocity, >> which is frustrating for developers. This resulted in the burnout of key >> firefighting resources. This resulted in tension between people who try >> to get specific work done and people who try to keep a handle on the big >> picture. >> >> It all boils down to an imbalance between strategic and tactical >> contributions. At the beginning of this project, we had a strong inner >> group of people dedicated to fixing all loose ends. Then a lot of >> companies got interested in OpenStack and there was a surge in tactical, >> short-term contributions. We put on a call for more resources to be >> dedicated to strategic contributions like critical bugfixing, >> vulnerability management, QA, infrastructure... and that call was >> answered by a lot of companies that are now key members of the OpenStack >> Foundation, and all was fine again. But OpenStack contributors kept on >> growing, and we grew the narrowly-focused population way faster than the >> cross-project population. >> >> >> At the same time, we kept on adding new projects to incubation and to >> the integrated release, which is great... but the new developers you get >> on board with this are much more likely to be tactical than strategic >> contributors. This also contributed to the imbalance. The penalty for >> that imbalance is twofold: we don't have enough resources available to >> solve old, known OpenStack-wide issues; but we also don't have enough >> resources to identify and fix new issues. >> >> We have several efforts under way, like calling for new strategic >> contributors, driving towards in-project functional testing, making >> solving rare issues a more attractive endeavor, or hiring resources >> directly at the Foundation level to help address those. But there is a >> topic we haven't raised yet: should we concentrate on fixing what is >> currently in the integrated release rather than adding new projects ? > > > TL;DR: Our development model is having growing pains. until we sort out the > growing pains adding more projects spreads us too thin. > +100
> In addition to the issues mentioned above, with the scale of OpenStack today > we have many major cross project issues to address and no good place to > discuss them. > We do have the ML, as well as the cross-project meeting every Tuesday [1], but we as a project need to do a better job of actually bringing up relevant issues here. [1] https://wiki.openstack.org/wiki/Meetings/ProjectMeeting >> >> >> We seem to be unable to address some key issues in the software we >> produce, and part of it is due to strategic contributors (and core >> reviewers) being overwhelmed just trying to stay afloat of what's >> happening. For such projects, is it time for a pause ? Is it time to >> define key cycle goals and defer everything else ? > > > > I really like this idea, as Michael and others alluded to in above, we are > attempting to set cycle goals for Kilo in Nova. but I think it is worth > doing for all of OpenStack. We would like to make a list of key goals before > the summit so that we can plan our summit sessions around the goals. On a > really high level one way to look at this is, in Kilo we need to pay down > our technical debt. > > The slots/runway idea is somewhat separate from defining key cycle goals; we > can be approve blueprints based on key cycle goals without doing slots. But > with so many concurrent blueprints up for review at any given time, the > review teams are doing a lot of multitasking and humans are not very good at > multitasking. Hopefully slots can help address this issue, and hopefully > allow us to actually merge more blueprints in a given cycle. > I'm not 100% sold on what the slots idea buys us. What I've seen this cycle in Neutron is that we have a LOT of BPs proposed. We approve them after review. And then we hit one of two issues: Slow review cycles, and slow code turnaround issues. I don't think slots would help this, and in fact may cause more issues. If we approve a BP and give it a slot for which the eventual result is slow review and/or code review turnaround, we're right back where we started. Even worse, we may have not picked a BP for which the code submitter would have turned around reviews faster. So we've now doubly hurt ourselves. I have no idea how to solve this issue, but by over subscribing the slots (e.g. over approving), we allow for the submissions with faster turnaround a chance to merge quicker. With slots, we've removed this capability by limiting what is even allowed to be considered for review. Thanks, Kyle >> >> >> On the integrated release side, "more projects" means stretching our >> limited strategic resources more. Is it time for the Technical Committee >> to more aggressively define what is "in" and what is "out" ? If we go >> through such a redefinition, shall we push currently-integrated projects >> that fail to match that definition out of the "integrated release" inner >> circle ? >> >> The TC discussion on what the integrated release should or should not >> include has always been informally going on. Some people would like to >> strictly limit to end-user-facing projects. Some others suggest that >> "OpenStack" should just be about integrating/exposing/scaling smart >> functionality that lives in specialized external projects, rather than >> trying to outsmart those by writing our own implementation. Some others >> are advocates of carefully moving up the stack, and to resist from >> further addressing IaaS+ services until we "complete" the pure IaaS >> space in a satisfactory manner. Some others would like to build a >> roadmap based on AWS services. Some others would just add anything that >> fits the incubation/integration requirements. >> >> >> On one side this is a long-term discussion, but on the other we also >> need to make quick decisions. With 4 incubated projects, and 2 new ones >> currently being proposed, there are a lot of people knocking at the door. > > > > I have a slightly different short term opinion on what 'OpenStack' should > be: it should work really well. > > While we need to figure out howto increase our strategic resources, > realistically I think that will still be the limiting factor in Kilo. So in > order to better allocate our cross project strategic resources, I think we > should do a project level triage ('identify projects that are likely to die, > regardless of what care they receive' to paraphrase the medical definition > of the term), and tighten our focus until we get our development process > into a better state. > > >> >> >> Thanks for reading this braindump this far. I hope this will trigger the >> open discussions we need to have, as an open source project, to reach >> the next level. > > > Thank you for bringing this topic up. > >> >> >> Cheers, >> >> -- >> Thierry Carrez (ttx) >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev