Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
> -Original Message- > From: Jay Pipes [mailto:jaypi...@gmail.com] > Sent: 26 September 2014 22:51 > To: openstack-dev@lists.openstack.org > Subject: Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent > model > > Heh, I just got off the phone with Monty talking about this :) Comments > inline... > > On 09/22/2014 03:11 PM, Tim Bell wrote: > > The quality designation is really important for the operator community > > who are trying to work out what we can give to our end users. > > So, I think it's important to point out here that there are three different > kinds of > operators/deployers: > > * Ones who use a distribution of OpenStack (RDO, UCA, MOS, Nebula, Piston, > etc) > * Ones who use Triple-O > * Ones who go it alone and install (via source, a mixture of source and > packages, via config management like Chef or Puppet, etc) > > In reality, you are referring to the last group, since operators in the first > group > are saying "we are relying on a distribution to make informed choices about > what is ready for prime time because we tested these things together". > Operators in the second group are really only HP right now, AFAICT, and > Triple- > O's "opinion" on the production readiness of the things it deploys in the > undercloud are roughly equal to "all of the integrated release that the TC > defines". > > I personally think that if an operator is choosing to be in the third group, > then > they are taking on the responsibility of testing what they deploy in a > staging/test > environment in order to validate that it meets their own requirements. In > other > words, the onus of determining whether something is production ready is on > their own shoulders. If the operator chose to deploy OpenStack on top of a > vanilla/custom Linux distribution instead of Red Hat, Ubuntu, OpenSUSE, or > whatever, we would not complain to the Linux kernel community that they have > not made quality designations available for all the pieces we decided to put > in > our custom Linux distribution. Likewise, I think that is the role of OpenStack > distributions: the choices and options that the distributor makes are a > result of > testing certain things together and the distributor's opinion of quality. If a > deployer of OpenStack chooses to go it alone and not use an OpenStack > distribution, that's totally cool, but I don't believe it should be the > OpenStack > developer community's responsibility to vouch for the production readiness of > each component of OpenStack. > > Now, Monty has a big problem with this idea. I know, because he just told me > he > does :) Monty thinks this attitude of relying on OpenStack distributions to > make > choices about production quality of OpenStack components leads inevitably to > "open core" opportunities, with some companies choosing to label a few > upstream components as production quality and others (the ones the company > developed internally) as their production quality choices. > > I happen to doubt that relying on OpenStack distributions to make these > choices > will lead to open core stuff. > I think there is a hybrid model in addition to your 3 listed of those who take a distribution for the core functions and are then cherry picking the other components according to the cloud they wish to offer. Thus, a distribution would define a base code set complying with the standard APIs and then deployers can select from a stackforge like repository for additional function not in the distribution. An example would be if you used RDO to deploy your cloud and then the Murano package from stackforge to give additional function not in the distribution. Puppetlabs recently announced 'approved' modules (http://www.infoq.com/news/2014/09/puppet-approved-modules) which are intended to help select modules which are not part of the core set but have had a good track record in production. Clearly, there is a difference in governance between the projects which would need a different mechanism for gathering the overall state such as some crowd-sourcing approach like Drupal does. Tim > Best, > -jay > > > Offering early helps to establish the real-life experience and give > > good feedback on the designs. However, the operator then risks > > leaving their users orphaned if the project does not get a sustainable > > following or significant disruption if the APIs change. > > > > The packaging teams are key here as well. When do Ubuntu and Red Hat > > work out the chain of pre-reqs etc. to produce installable packages, > > packstack/juju tool support ? > > > > We
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Robert Collins on Friday, September 26, 2014 3:33 PM wrote: > On 27 September 2014 09:43, Jay Pipes wrote: > > Hi James, thanks for the corrections/explanations. A comment inline > (and a > > further question) :) > > > Oh, good to know. Sorry, my information about Triple-O's undercloud > setup is > > clearly outdated. I thought that the undercloud was build from source > > repositories devstack-style. Thanks for hitting me with a cluestick. > :) > > Thats one installation strategy :). > > >> Even when not installing via a distribution, and either directly > from > >> trunk or the integrated release tarballs, I don't know that any > >> TripleO opinion enters into it. TripleO uses the integrated projects > >> of OpenStack to deploy an overcloud. In an overcloud, you may see > >> support for some incubated projects, depending on if there's > interest > >> from the community for that support. > > > > > > OK, interesting. So, in summary, Triple-O really doesn't offer much > of a > > "this is production-ready" stamp to anything based on whether it > deploys a > > project or not. So, operators who deploy with Triple-O would be in > the > > "you're on your own" camp from the bulleted list above. Would that be > a fair > > statement? > > TripleO upstream doesn't offer a production-ready stamp for the > workload clouds; for deploy clouds we do - simply because you can't > use non-production-ready services to do your deploy... some of our > stamps have substantial caveats today (e.g. Heat) - but they are being > worked on. > > But then Nova upstream doesn't offer production-ready stamps either. > Are cells production ready? Instance groups? Or generally any new > feature? > > *distributions* of TripleO offer production ready stamps. > > RDO offers one > HP Helion offers one. > > In exactly the same way that distributions offer production stamps > about Nova, distributions that use TripleO offer production stamps > about Nova :). > > And I think this is James's point. Your category 2 above saying that > TripleO is different is just confused: TripleO is a deployment > architecture [evolving into a set of such], *not* a distribution > channel. 1 and 3 are distribution channels. [Rocky] I'd like to make a couple of points; first is how many "commercial deployers/operators" would consider any of OpenStack "production ready" and if they do, what would that subset actually be? ( a little snarky, but not really) Second, I'd like to point out that Defcore is attempting to provide guidance along these lines, but may be considered a bit more "strict" than a "Production Ready" label. Then again, it may be less strict, depending on test coverage;-) Check out the scoring criteria here: https://wiki.openstack.org/wiki/Governance/CoreCriteria In principle, OpenStack functionality has to have been "production tested" with a fairly large distribution base, along with a number of other criteria. As defined, it would definitely be a subset of "production ready" but it would be a reasonably safe subset that could then be built upon. Beyond the DefCore criteria, RefStack is heading towards the point where any operator could run all of Tempest or any selection of Tempest tests against his/her installed cloud and view all the results. They could even save them to do trend analysis. But, this still begs the question of what is "production ready" especially for Open Source code. Perhaps we need the release notes to be very specific on which APIs, features and/or projects are new/radically altered in a specific release. This could allow operators to install "Juno release/Icehouse functionality" which would theoretically be much better than "Icehouse Release" which is what a lot of shops would do, reasoning that Icehouse has been out six months, so the gotchas are known and patches have been released to fix the worst issues. Whereas, I think that most people in QA and/or deep in the various projects would say that the functionality that was released in Icehouse that also is in Juno would be more performant and less buggy than the Icehouse release. So really, the question might really be: how do you get deployers who want greater stability to actually get it by deploying the current release with past release's functionality subset? And how do you communicate that some of the past release's functionality is still not production ready? Hard problem. > > -Rob > > -- > Robert Collins > Distinguished Technologist > HP Converged Cloud > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Excerpts from Jay Pipes's message of 2014-09-26 14:43:40 -0700: > Hi James, thanks for the corrections/explanations. A comment inline (and > a further question) :) > > On 09/26/2014 05:35 PM, James Slagle wrote: > > On Fri, Sep 26, 2014 at 4:50 PM, Jay Pipes wrote: > >> Heh, I just got off the phone with Monty talking about this :) Comments > >> inline... > >> > >> On 09/22/2014 03:11 PM, Tim Bell wrote: > >>> > >>> The quality designation is really important for the operator > >>> community who are trying to work out what we can give to our end > >>> users. > >> > >> > >> So, I think it's important to point out here that there are three different > >> kinds of operators/deployers: > >> > >> * Ones who use a distribution of OpenStack (RDO, UCA, MOS, Nebula, > >> Piston, > >> etc) > >> * Ones who use Triple-O > >> * Ones who go it alone and install (via source, a mixture of source and > >> packages, via config management like Chef or Puppet, etc) > > > > I'm not sure TripleO fits in this list. It is not just a collection of > > prescriptive OpenStack bits used to do a deployment. TripleO is > > tooling to build OpenStack to deploy OpenStack. You can use whatever > > "source" (packages, distribution, released tarballs, straight from > > git) you want to build that OpenStack. TripleO could deploy your first > > or third bullet item. > > OK, fair point, thanks for that added bit of description. > > >> In reality, you are referring to the last group, since operators in the > >> first group are saying "we are relying on a distribution to make informed > >> choices about what is ready for prime time because we tested these things > >> together". Operators in the second group are really only HP right now, > >> AFAICT, and Triple-O's "opinion" on the production readiness of the things > >> it deploys in the undercloud are roughly equal to "all of the integrated > >> release that the TC defines". > > > > FWIW, TripleO offers deploying using distributions, by installing from > > packages from the RDO repositories. There's nothing RDO specific about > > it though, any packaged OpenStack distribution could be installed with > > the TripleO tooling. RDO is just likely the most well tested. > > Oh, good to know. Sorry, my information about Triple-O's undercloud > setup is clearly outdated. I thought that the undercloud was build from > source repositories devstack-style. Thanks for hitting me with a > cluestick. :) > > > Even when not installing via a distribution, and either directly from > > trunk or the integrated release tarballs, I don't know that any > > TripleO opinion enters into it. TripleO uses the integrated projects > > of OpenStack to deploy an overcloud. In an overcloud, you may see > > support for some incubated projects, depending on if there's interest > > from the community for that support. > > OK, interesting. So, in summary, Triple-O really doesn't offer much of a > "this is production-ready" stamp to anything based on whether it deploys > a project or not. So, operators who deploy with Triple-O would be in the > "you're on your own" camp from the bulleted list above. Would that be a > fair statement? The one thing that builds an overcloud in a prescriptive manner is called 'tripleo-incubator' precisely because we don't believe it is ready to be called a release. It does focus on testing the integrated release only though, and so is closer to your original #2. It's just important to note that this is just our own distribution, and does not fully represent the whole output of the program. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 27 September 2014 09:43, Jay Pipes wrote: > Hi James, thanks for the corrections/explanations. A comment inline (and a > further question) :) > Oh, good to know. Sorry, my information about Triple-O's undercloud setup is > clearly outdated. I thought that the undercloud was build from source > repositories devstack-style. Thanks for hitting me with a cluestick. :) Thats one installation strategy :). >> Even when not installing via a distribution, and either directly from >> trunk or the integrated release tarballs, I don't know that any >> TripleO opinion enters into it. TripleO uses the integrated projects >> of OpenStack to deploy an overcloud. In an overcloud, you may see >> support for some incubated projects, depending on if there's interest >> from the community for that support. > > > OK, interesting. So, in summary, Triple-O really doesn't offer much of a > "this is production-ready" stamp to anything based on whether it deploys a > project or not. So, operators who deploy with Triple-O would be in the > "you're on your own" camp from the bulleted list above. Would that be a fair > statement? TripleO upstream doesn't offer a production-ready stamp for the workload clouds; for deploy clouds we do - simply because you can't use non-production-ready services to do your deploy... some of our stamps have substantial caveats today (e.g. Heat) - but they are being worked on. But then Nova upstream doesn't offer production-ready stamps either. Are cells production ready? Instance groups? Or generally any new feature? *distributions* of TripleO offer production ready stamps. RDO offers one HP Helion offers one. In exactly the same way that distributions offer production stamps about Nova, distributions that use TripleO offer production stamps about Nova :). And I think this is James's point. Your category 2 above saying that TripleO is different is just confused: TripleO is a deployment architecture [evolving into a set of such], *not* a distribution channel. 1 and 3 are distribution channels. -Rob -- Robert Collins Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Hi James, thanks for the corrections/explanations. A comment inline (and a further question) :) On 09/26/2014 05:35 PM, James Slagle wrote: On Fri, Sep 26, 2014 at 4:50 PM, Jay Pipes wrote: Heh, I just got off the phone with Monty talking about this :) Comments inline... On 09/22/2014 03:11 PM, Tim Bell wrote: The quality designation is really important for the operator community who are trying to work out what we can give to our end users. So, I think it's important to point out here that there are three different kinds of operators/deployers: * Ones who use a distribution of OpenStack (RDO, UCA, MOS, Nebula, Piston, etc) * Ones who use Triple-O * Ones who go it alone and install (via source, a mixture of source and packages, via config management like Chef or Puppet, etc) I'm not sure TripleO fits in this list. It is not just a collection of prescriptive OpenStack bits used to do a deployment. TripleO is tooling to build OpenStack to deploy OpenStack. You can use whatever "source" (packages, distribution, released tarballs, straight from git) you want to build that OpenStack. TripleO could deploy your first or third bullet item. OK, fair point, thanks for that added bit of description. In reality, you are referring to the last group, since operators in the first group are saying "we are relying on a distribution to make informed choices about what is ready for prime time because we tested these things together". Operators in the second group are really only HP right now, AFAICT, and Triple-O's "opinion" on the production readiness of the things it deploys in the undercloud are roughly equal to "all of the integrated release that the TC defines". FWIW, TripleO offers deploying using distributions, by installing from packages from the RDO repositories. There's nothing RDO specific about it though, any packaged OpenStack distribution could be installed with the TripleO tooling. RDO is just likely the most well tested. Oh, good to know. Sorry, my information about Triple-O's undercloud setup is clearly outdated. I thought that the undercloud was build from source repositories devstack-style. Thanks for hitting me with a cluestick. :) Even when not installing via a distribution, and either directly from trunk or the integrated release tarballs, I don't know that any TripleO opinion enters into it. TripleO uses the integrated projects of OpenStack to deploy an overcloud. In an overcloud, you may see support for some incubated projects, depending on if there's interest from the community for that support. OK, interesting. So, in summary, Triple-O really doesn't offer much of a "this is production-ready" stamp to anything based on whether it deploys a project or not. So, operators who deploy with Triple-O would be in the "you're on your own" camp from the bulleted list above. Would that be a fair statement? Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Fri, Sep 26, 2014 at 4:50 PM, Jay Pipes wrote: > Heh, I just got off the phone with Monty talking about this :) Comments > inline... > > On 09/22/2014 03:11 PM, Tim Bell wrote: >> >> The quality designation is really important for the operator >> community who are trying to work out what we can give to our end >> users. > > > So, I think it's important to point out here that there are three different > kinds of operators/deployers: > > * Ones who use a distribution of OpenStack (RDO, UCA, MOS, Nebula, Piston, > etc) > * Ones who use Triple-O > * Ones who go it alone and install (via source, a mixture of source and > packages, via config management like Chef or Puppet, etc) I'm not sure TripleO fits in this list. It is not just a collection of prescriptive OpenStack bits used to do a deployment. TripleO is tooling to build OpenStack to deploy OpenStack. You can use whatever "source" (packages, distribution, released tarballs, straight from git) you want to build that OpenStack. TripleO could deploy your first or third bullet item. > > In reality, you are referring to the last group, since operators in the > first group are saying "we are relying on a distribution to make informed > choices about what is ready for prime time because we tested these things > together". Operators in the second group are really only HP right now, > AFAICT, and Triple-O's "opinion" on the production readiness of the things > it deploys in the undercloud are roughly equal to "all of the integrated > release that the TC defines". FWIW, TripleO offers deploying using distributions, by installing from packages from the RDO repositories. There's nothing RDO specific about it though, any packaged OpenStack distribution could be installed with the TripleO tooling. RDO is just likely the most well tested. Even when not installing via a distribution, and either directly from trunk or the integrated release tarballs, I don't know that any TripleO opinion enters into it. TripleO uses the integrated projects of OpenStack to deploy an overcloud. In an overcloud, you may see support for some incubated projects, depending on if there's interest from the community for that support. -- -- James Slagle -- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Heh, I just got off the phone with Monty talking about this :) Comments inline... On 09/22/2014 03:11 PM, Tim Bell wrote: The quality designation is really important for the operator community who are trying to work out what we can give to our end users. So, I think it's important to point out here that there are three different kinds of operators/deployers: * Ones who use a distribution of OpenStack (RDO, UCA, MOS, Nebula, Piston, etc) * Ones who use Triple-O * Ones who go it alone and install (via source, a mixture of source and packages, via config management like Chef or Puppet, etc) In reality, you are referring to the last group, since operators in the first group are saying "we are relying on a distribution to make informed choices about what is ready for prime time because we tested these things together". Operators in the second group are really only HP right now, AFAICT, and Triple-O's "opinion" on the production readiness of the things it deploys in the undercloud are roughly equal to "all of the integrated release that the TC defines". I personally think that if an operator is choosing to be in the third group, then they are taking on the responsibility of testing what they deploy in a staging/test environment in order to validate that it meets their own requirements. In other words, the onus of determining whether something is production ready is on their own shoulders. If the operator chose to deploy OpenStack on top of a vanilla/custom Linux distribution instead of Red Hat, Ubuntu, OpenSUSE, or whatever, we would not complain to the Linux kernel community that they have not made quality designations available for all the pieces we decided to put in our custom Linux distribution. Likewise, I think that is the role of OpenStack distributions: the choices and options that the distributor makes are a result of testing certain things together and the distributor's opinion of quality. If a deployer of OpenStack chooses to go it alone and not use an OpenStack distribution, that's totally cool, but I don't believe it should be the OpenStack developer community's responsibility to vouch for the production readiness of each component of OpenStack. Now, Monty has a big problem with this idea. I know, because he just told me he does :) Monty thinks this attitude of relying on OpenStack distributions to make choices about production quality of OpenStack components leads inevitably to "open core" opportunities, with some companies choosing to label a few upstream components as production quality and others (the ones the company developed internally) as their production quality choices. I happen to doubt that relying on OpenStack distributions to make these choices will lead to open core stuff. Best, -jay Offering early helps to establish the real-life experience and give good feedback on the designs. However, the operator then risks leaving their users orphaned if the project does not get a sustainable following or significant disruption if the APIs change. The packaging teams are key here as well. When do Ubuntu and Red Hat work out the chain of pre-reqs etc. to produce installable packages, packstack/juju tool support ? We do need to have some way to show that an layer #2 package is ready for prime time production and associated criteria (packages available, docs available, >1 company communities, models for HA and scale, …) Tim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 09/18/2014 02:53 PM, Monty Taylor wrote: Hey all, I've recently been thinking a lot about Sean's Layers stuff. So I wrote a blog post which Jim Blair and Devananda were kind enough to help me edit. http://inaugust.com/post/108 Enjoy. I enjoyed your post (though I don't agree with everything, of course) :) I typed up some of my thoughts here about these questions we're struggling with and some proposals on possible ways forward. http://www.joinfu.com/2014/09/answering-the-existential-question-in-openstack/ Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Sep 26, 2014, at 1:25 AM, Thierry Carrez wrote: > That said, singling out the test infrastructure (3) and the release > management (2) is a bit unfair to other horizontal efforts, like > Documentation, Translations, or general QA, which also suffer from a > scale issue. The Docs team, in particular, couldn't really scale either > and already implemented two tiers within the integrated release -- the > part they directly support, and the part they help / provide tools for. You are correct I left out some very important cross project teams (Sorry Anne!). We call the projects that tap these shared resources today “integrated”. It seems like if we keep going down this path, we need to either: a) make “integrated” == layer1 This would be a tricky proposition because of the way it might look in the community, but it may be necessary if we are already too overloaded to handle the cross-project concerns at our current scale. b) clearly dilineate “integrated” vs layer1 in some other way This would likely involve changing the meaning of integrated to mean a bit less than it does today: just best effort for projects outside of layer1. > All the "other" projects would be able to get help and tools from those > horizontal teams, but would just not be directly taken care of. This is > how Docs currently work (the two tiers within integrated release), this > is how Release management currently works (integrated vs. incubated), > this is how the test infra would like to be able to work... Making sure > we at least support the layer #1 is just a matter of setting the same > basic expectations for the same basic set of projects. It sounds like you are suggesting that b) has already occurred to some degree so that we should just continue along those lines? Vish signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/25/2014 08:42 PM, Vishvananda Ishaya wrote: > It seems that this discussion has actually illustrated shortcomings in our > answers to 3 separate questions, and people have been throwing out ideas > that attempt to solve all 3. Perhaps we need to address each one individually. > > The three questions are: > > 1. Which projects are “part of openstack”? > 2. Which projects are released as a single unit? > 3. Which projects are tested together > > The current answers are: > 1. Three levels incubation, integration, core > 2. Things that reach the integration level > 3. Things that reach the integration level. There is a fourth question, although it's somewhat related to the first: do we continue the concept of 'incubated' and 'integrated'? Do these processes and designations improve OpenStack, or do they impede progress? Does designating a development effort as "in the club" discourage other, potentially better solutions? Or would keeping things wide open simply dilute the efforts of people working on these solutions? Assuming that we can achieve consensus as to what constitutes Ring0/Layer1/core, the question then becomes how does everything else get handled? IOW, are they all treated as independent additions? Or do some receive a special designation (e.g., "certified good enough for the Cern LHC"), and others are simply "your results may vary"? The issues with gating everything together aren't the cause for making changes to how OpenStack is organized; they are the symptom that we can't continue along the current path indefinitely. Breaking out the testing of the core from everything else is a good start, and will help alleviate many of the gating issues, but I think it's also important to realize that the growing pains are not limited to testing. - -- Ed Leafe -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.14 (GNU/Linux) iQIcBAEBAgAGBQJUJY6MAAoJEKMgtcocwZqLuxwP/1QSoHhhegZETtxVyrxBisOs sjJfh+wzLqjfEqkwUOZ0SX2aEFHuimNE5lWmPPB7hzffc6Z4aNxITjP/fCry77Ax IQsGTbTGG0gsXiyO/+/x6c6Yfkcmu6fui8L/hde0u7klN/8tATkp/5xwW4qvGXY7 1WJwBsHDx8MniqoJ2echP5uUytFMxGgK9tHA0bnWUEBNVCVt8tmczNTrRJzHY49P ZbdSDvwypvClfSLgu1/z8yb0Wv54dy0eafi6Gf+kYL0e3i5//Pv3YkSKrHTqm3Ww R1V6Acr/BOT6MC8Ln1TF2gNgSo8iCAOfIakkwmCLJCCuvOmKA3hbch16Ggqhm1Ua mDyuUcOICKs+NX07cIKnVkI5WpIGqCQ0M/HS1jiix3tOPHe3sie85Jqser9PGnis Hwtva100HqnBARp3vips7inXughfiiQ3UHW41l4aLuSf0faba1v2CiFrQfE6Nvm3 ownAMdp3qCtBgXkl7KcgFyyWeIZVUEDbxdMw+A5GR3PIsUDD07a2YB/mY03pVHc2 lacZFlw2aFoeVjnRhU988h+LnQvwM4+4mqpcnJvDsVGalV5CALI/I0CzJMjKLfZV t02jtLZz7CcUDPxGwmqF82rn+BYzJX/glpMYYHVQVzKL7MJr/H5eg8M1oi0huNoA lSfQ2U3/c4A1eURsLybm =qAHC -END PGP SIGNATURE- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Vishvananda Ishaya wrote: > The three questions are: > > 1. Which projects are “part of openstack”? > 2. Which projects are released as a single unit? > 3. Which projects are tested together That's a good summary, yes. Currently we have a number of horizontal teams, which must support equally an always-increasing number of "integrated" projects. That means 1=2=3. But in the case of (3) it just might not make sense, and in the case of (2) it doesn't scale infinitely. That said, singling out the test infrastructure (3) and the release management (2) is a bit unfair to other horizontal efforts, like Documentation, Translations, or general QA, which also suffer from a scale issue. The Docs team, in particular, couldn't really scale either and already implemented two tiers within the integrated release -- the part they directly support, and the part they help / provide tools for. > The current answers are: > 1. Three levels incubation, integration, core > 2. Things that reach the integration level > 3. Things that reach the integration level. > > Some proposed answers: > 1. Lightweight incubation a la apache > 2. Monty’s layer1 > 3. Direct dependencies and close collaborators > > Discussing the propased answers(in reverse order): > I think we have rough consensus around 3: that we should move > towards functional testing for direct dependencies and let the > projects decide when they want to co-gate. The functional > co-gating should ideally be based on important use-cases. > > 2 is a bit murkier. In the interest of staying true to our roots > the best we can probably do is to allow projects to opt-out of > the coordinated release and for thierry to specifically select > which projects he is willing to coordinate. Any other project > could co-release with the integrated release but wouldn’t be > centrally managed by thierry. There is also a decision about > what the TCs role is in these projects. Like I said above, that seems to single out release management, while other horizontal efforts are facing the same challenge. I fear if we let each horizontal program choose which set of projects it supports, it will result in a bit of a mess, where no expectation is clearly set (project A is on the common release but Anne doesn't touch its docs directly, project B releases when it wants to and has Anne's team working on it directly...) If the "horizontal teams" all agree to directly support the same non-expanding set (for the sake of the exercise, let's call it layer #1), why not have it ? It's definitely a set that test infra, QA, Release Management and Docs would feel comfortable supporting, AFAIK. All the "other" projects would be able to get help and tools from those horizontal teams, but would just not be directly taken care of. This is how Docs currently work (the two tiers within integrated release), this is how Release management currently works (integrated vs. incubated), this is how the test infra would like to be able to work... Making sure we at least support the layer #1 is just a matter of setting the same basic expectations for the same basic set of projects. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 09/26/2014 03:42 AM, Vishvananda Ishaya wrote: > > On Sep 25, 2014, at 4:01 PM, Robert Collins wrote: > >> So I guess I'm saying: >> >>Lets decouple 'what is openstack' from 'what we test together on >> every commit'. > > It seems that this discussion has actually illustrated shortcomings in our > answers to 3 separate questions, and people have been throwing out ideas > that attempt to solve all 3. Perhaps we need to address each one individually. > > The three questions are: > > 1. Which projects are “part of openstack”? > 2. Which projects are released as a single unit? > 3. Which projects are tested together > > The current answers are: > 1. Three levels incubation, integration, core > 2. Things that reach the integration level > 3. Things that reach the integration level. > > Some proposed answers: > 1. Lightweight incubation a la apache > 2. Monty’s layer1 > 3. Direct dependencies and close collaborators > > Discussing the propased answers(in reverse order): > I think we have rough consensus around 3: that we should move > towards functional testing for direct dependencies and let the > projects decide when they want to co-gate. The functional > co-gating should ideally be based on important use-cases. > > 2 is a bit murkier. In the interest of staying true to our roots > the best we can probably do is to allow projects to opt-out of > the coordinated release and for thierry to specifically select > which projects he is willing to coordinate. Any other project > could co-release with the integrated release but wouldn’t be > centrally managed by thierry. There is also a decision about > what the TCs role is in these projects. Assuming there would be enough volunteers, would it make sense to have a release manager per layer? And have those release managers working together on an integrated release? or even have someone in each project working on the release with the release manager *but* still have an integrated release? I'm afraid that not having an integrated release will end up in a whole mess. When should packages be released? How will stable maintenance work? When will distros build the packages? > 1 Has some unanswerd questions, like is there another > level “graduation” where the tc has some kind of technical > oversight? What is the criteria for it? etc. As mentioned in other emails[0], I think we should reduce this levels to the minimum and work on a guided grow path for projects that will take them to the point where they'll be considered production-ready/stable/you-name-it. Moreover, this process should encourage collaboration (certainly not as much as we've pushed it so far) over competition but still welcome the later. [0] http://lists.openstack.org/pipermail/openstack-dev/2014-September/047114.html Flavio -- @flaper87 Flavio Percoco ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Sep 25, 2014, at 4:01 PM, Robert Collins wrote: > So I guess I'm saying: > >Lets decouple 'what is openstack' from 'what we test together on > every commit'. It seems that this discussion has actually illustrated shortcomings in our answers to 3 separate questions, and people have been throwing out ideas that attempt to solve all 3. Perhaps we need to address each one individually. The three questions are: 1. Which projects are “part of openstack”? 2. Which projects are released as a single unit? 3. Which projects are tested together The current answers are: 1. Three levels incubation, integration, core 2. Things that reach the integration level 3. Things that reach the integration level. Some proposed answers: 1. Lightweight incubation a la apache 2. Monty’s layer1 3. Direct dependencies and close collaborators Discussing the propased answers(in reverse order): I think we have rough consensus around 3: that we should move towards functional testing for direct dependencies and let the projects decide when they want to co-gate. The functional co-gating should ideally be based on important use-cases. 2 is a bit murkier. In the interest of staying true to our roots the best we can probably do is to allow projects to opt-out of the coordinated release and for thierry to specifically select which projects he is willing to coordinate. Any other project could co-release with the integrated release but wouldn’t be centrally managed by thierry. There is also a decision about what the TCs role is in these projects. 1 Has some unanswerd questions, like is there another level “graduation” where the tc has some kind of technical oversight? What is the criteria for it? etc. Maybe addressing these things separately will allow us to make progress. Vish signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 26 September 2014 10:28, Zane Bitter wrote:
> So it goes without saying that I support the latter part ("functionally test
> against their real dependencies"). I'm not convinced by the idea of not
> having an integrated release though. Time-based releases seem to be pretty
> popular lately - the Linux kernel, most distributions and the two major
> open-source web browsers use them, for example - and as a developer I don't
> feel especially qualified to second-guess what the release schedule for a
> particular component should actually be. The current cycle works decently
> well, is not obviously worse than any particular alternative, and aligns
> semi-nicely with the design summits (FWIW I think I would actually prefer
> the design summit take place _before_ the release, but that is a whole other
> discussion).
>
> We actually discussed with Monty at this week's Heat meeting his proposal to
> move the UI projects to a continuous release schedule. (For the moment Heat
> is actually blocked on severe limitations in the usefulness of standalone
> mode, but we expect those to shake out over the near term anyway.) I think
> there was general agreement that there would be some big upsides - it really
> sucks telling the 15th user that we already redesigned that thing to solve
> your issue like 4 months ago, but since you're stuck on Icehouse we can't
> help. On the other hand, there's big downsides too. We're still making major
> changes, and it's really nice to be able to let them bed in as part of a
> release cycle.
I think bedding them in is great, but sometimes thats more than a
cycle. We benefit if we can decouple 'make big change' from 'big
change is bedded in and ready for users to use'.
> (Since I started this discussion talking about bias, it's worth calling out
> a *huge* one here: my team has to figure out a way to package and distribute
> this stuff.)
:).
> That said, if we can get to a situation where we *choose* to do a
> co-ordinated release for the convenience of developers, distributors and
> (hopefully) users - rather than being forced into it through sheer terror
> that everything will fall over in a flaming heap if we don't - that would
> obviously be a win :)
+1.
> Right, if we _have_ to have it let's not name it something aspirational like
> "Layer 1" or "ring 0". Let's call it "Cluster Foxtrot" and make sure that
> projects are queueing up to get _out_. We can start with Designate, which
> afaik has done nothing to bring upon themselves such a fate ;)
>
> I'm not convinced that it has to be a formal, named thing though. In terms
> of the gating thing, that would be an organisational solution to a purely
> technical problem: up to now there's been no middle ground between a cluster
> of projects that all gate against each other and just not gating against
> each other at all. I'm totally confident that the QA and Infra teams can fix
> that. Those folks are superb at what they do.
>
> And on the release side, I think we're running before we can walk. It's not
> clear that most, or even many, projects would want to abandon a co-ordinated
> release anyway. (I'd actually have no problem with letting projects opt-out
> if they have some reason to - TripleO already effectively did.) External
> stakeholders (including the board) would _inevitably_ treat a shrinking of
> the co-ordinated release as an endorsement of which projects we actually
> care about, and by extension which projects they should care about.
Hmm, this isn't really representative of TripleO's position. We
*want*, *desperately* to be in the integrated gate, and we're nearly
there in terms of donated capacity - we're focusing on on reliability
at the moment. We have no integrated API servers [yet], but Tuskar's
got a very clear plan taking it into the integrated release. Projects
!= Programs :). The non-API-server components of OpenStack as a whole
are mostly not part of the integrated release. The plan with Tuskar
was to get it stable enough to meet the incubation requirements and
then apply. Of course if incubation goes away, thats different :).
> If we ever reach a point where interfaces are stable enough that we don't
> need a co-ordinated release, let's consider _then_ whether to tear the whole
> thing down - in one fell swoop. Taking a half-measure sends exactly the
> wrong signal.
So as soon as you said that the Board would care about what is in the
integrated release, that re-instates the winners-and-losers thing that
a lot of this discussion is about. And Swift is already on a separate
schedule, but its one of our most popular projects... I think there's
something fundamentally mixed up here :0.
> Finally, I would add that unco-ordinated releases, where distributions have
> to select a set of components that synced to the global requirements at
> different times, are not going to be truly feasible until we have a
> container-based deployment system. I expect it to not be an issue in the
> future, but that w
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 25/09/14 15:12, Vishvananda Ishaya wrote:
On Sep 24, 2014, at 10:55 AM, Zane Bitter wrote:
On 18/09/14 14:53, Monty Taylor wrote:
Hey all,
I've recently been thinking a lot about Sean's Layers stuff. So I wrote
a blog post which Jim Blair and Devananda were kind enough to help me edit.
http://inaugust.com/post/108
I think there are a number of unjustified assumptions behind this arrangement
of things. I'm going to list some here, but I don't want anyone to interpret
this as a personal criticism of Monty. The point is that we all suffer from
biases - not for any questionable reasons but purely as a result of our own
experiences, who we spend our time talking to and what we spend our time
thinking about - and therefore we should all be extremely circumspect about
trying to bake our own mental models of what OpenStack should be into the
organisational structure of the project itself.
I think there were some assumptions that lead to the Layer1 model. Perhaps a
little insight into the in-person debate[1] at OpenStack-SV might help explain
where monty was coming from.
Thanks Vish, that is indeed useful background. Apparently I need to get
out more ;)
The initial thought was a radical idea (pioneered by Jay) to completely
dismantle the integrated release and have all projects release independently
and functionally test against their real dependencies. This gained support from
various people and I still think it is a great long-term goal.
So it goes without saying that I support the latter part ("functionally
test against their real dependencies"). I'm not convinced by the idea of
not having an integrated release though. Time-based releases seem to be
pretty popular lately - the Linux kernel, most distributions and the two
major open-source web browsers use them, for example - and as a
developer I don't feel especially qualified to second-guess what the
release schedule for a particular component should actually be. The
current cycle works decently well, is not obviously worse than any
particular alternative, and aligns semi-nicely with the design summits
(FWIW I think I would actually prefer the design summit take place
_before_ the release, but that is a whole other discussion).
We actually discussed with Monty at this week's Heat meeting his
proposal to move the UI projects to a continuous release schedule. (For
the moment Heat is actually blocked on severe limitations in the
usefulness of standalone mode, but we expect those to shake out over the
near term anyway.) I think there was general agreement that there would
be some big upsides - it really sucks telling the 15th user that we
already redesigned that thing to solve your issue like 4 months ago, but
since you're stuck on Icehouse we can't help. On the other hand, there's
big downsides too. We're still making major changes, and it's really
nice to be able to let them bed in as part of a release cycle.
(Since I started this discussion talking about bias, it's worth calling
out a *huge* one here: my team has to figure out a way to package and
distribute this stuff.)
That said, if we can get to a situation where we *choose* to do a
co-ordinated release for the convenience of developers, distributors and
(hopefully) users - rather than being forced into it through sheer
terror that everything will fall over in a flaming heap if we don't -
that would obviously be a win :)
The worry that Monty (and others) had are two-fold:
1. When we had no co-gating in the past, we ended up with a lot of
cross-project breakage. If we jump right into this we could end up in the wild
west were different projects expect different keystone versions and there is no
way to deploy a functional cloud.
2. We have set expectations in our community (and especially with
distributions), that we release a set of things that all work together. It is
not acceptable for us to just pull the rug out from under them.
These concerns show that we must (in the short term) provide some kind of
integrated testing and release. I see the layer1 model as a stepping stone
towards the long term goal of having the projects release independently and
depend on stable interfaces. We aren’t going to get there immediately, so
having a smaller, integrated set of services representing our most common use
case seems like a good first step. As our interfaces get more stable and our
testing gets better it could move to a (once every X months) release that just
packages the current version of the layer1 projects or even be completely
managed by distributions.
We need a way to move forward, but I’m hoping we can do it without a concept of
“specialness” around layer1 projects. I actually see it as a limitation of
these projects that we have to take this stepping stone and cannot disaggregate
completely. Instead it should be seen as a necessary evil so that we don’t
break our users.
Right, if we _have_ to have it let's not name i
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Sep 24, 2014, at 10:55 AM, Zane Bitter wrote: > On 18/09/14 14:53, Monty Taylor wrote: >> Hey all, >> >> I've recently been thinking a lot about Sean's Layers stuff. So I wrote >> a blog post which Jim Blair and Devananda were kind enough to help me edit. >> >> http://inaugust.com/post/108 > > I think there are a number of unjustified assumptions behind this arrangement > of things. I'm going to list some here, but I don't want anyone to interpret > this as a personal criticism of Monty. The point is that we all suffer from > biases - not for any questionable reasons but purely as a result of our own > experiences, who we spend our time talking to and what we spend our time > thinking about - and therefore we should all be extremely circumspect about > trying to bake our own mental models of what OpenStack should be into the > organisational structure of the project itself. I think there were some assumptions that lead to the Layer1 model. Perhaps a little insight into the in-person debate[1] at OpenStack-SV might help explain where monty was coming from. The initial thought was a radical idea (pioneered by Jay) to completely dismantle the integrated release and have all projects release independently and functionally test against their real dependencies. This gained support from various people and I still think it is a great long-term goal. The worry that Monty (and others) had are two-fold: 1. When we had no co-gating in the past, we ended up with a lot of cross-project breakage. If we jump right into this we could end up in the wild west were different projects expect different keystone versions and there is no way to deploy a functional cloud. 2. We have set expectations in our community (and especially with distributions), that we release a set of things that all work together. It is not acceptable for us to just pull the rug out from under them. These concerns show that we must (in the short term) provide some kind of integrated testing and release. I see the layer1 model as a stepping stone towards the long term goal of having the projects release independently and depend on stable interfaces. We aren’t going to get there immediately, so having a smaller, integrated set of services representing our most common use case seems like a good first step. As our interfaces get more stable and our testing gets better it could move to a (once every X months) release that just packages the current version of the layer1 projects or even be completely managed by distributions. We need a way to move forward, but I’m hoping we can do it without a concept of “specialness” around layer1 projects. I actually see it as a limitation of these projects that we have to take this stepping stone and cannot disaggregate completely. Instead it should be seen as a necessary evil so that we don’t break our users. In addition, we should encourage other shared use cases in openstack both for testing (functional tests against groups of services) and for releases (shared releases of related projects). [1] Note this wasn’t a planned debate, but a spontaneous discussion that included (at various points) Monty Taylor, Jay Pipes, Joe Gordon, John Dickenson, Myself, and (undoubtedly) one or two people I”m forgetting. signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 09/24/2014 07:55 PM, Zane Bitter wrote: > On 18/09/14 14:53, Monty Taylor wrote: >> Hey all, >> >> I've recently been thinking a lot about Sean's Layers stuff. So I wrote >> a blog post which Jim Blair and Devananda were kind enough to help me >> edit. >> >> http://inaugust.com/post/108 > > Thanks Monty, I think there are some very interesting ideas in here. > > I'm particularly glad to see the 'big tent' camp reasserting itself, > because I have no sympathy with anyone who wants to join the OpenStack > community and then bolt the door behind them. Anyone who contributes to > a project that is related to OpenStack's goals, is willing to do things > the OpenStack way, and submits itself to the scrutiny of the TC deserves > to be treated as a member of our community with voting rights, entry to > the Design Summit and so on. > > I'm curious how you're suggesting we decide which projects satisfy those > criteria though. Up until now, we've done it through the incubation > process (or technically, the new program approval process... but in > practice we've never added a project that was targeted for eventual > inclusion in the integrated release to a program without incubating it). > Would the TC continue to judge whether a project is doing things the > OpenStack way prior to inclusion, or would we let projects self-certify? > What does it mean for a project to submit itself to TC scrutiny if it > knows that realistically the TC will never have time to actually > scrutinise it? Or are you not suggesting a change to the current > incubation process, just a willingness to incubate multiple projects in > the same problem space? > > I feel like I need to play devil's advocate here, because overall I'm > just not sure I understand the purpose of arbitrarily - and it *is* > arbitrary - declaring "Layer #1" to be anything required to run > Wordpress. To anyone whose goal is not to run Wordpress, how is that > relevant? > > Speaking of arbitrary, I had to laugh a little at this bit: > > Also, please someone notice that the above is too many steps and should > be: > > openstack boot gentoo on-a 2G-VM with-a publicIP with-a 10G-volume > call-it blog.inaugust.com > > That's kinda sorta exactly what Heat does ;) Minus the part about > assuming there is only one kind of application, obviously. > > > I think there are a number of unjustified assumptions behind this > arrangement of things. I'm going to list some here, but I don't want > anyone to interpret this as a personal criticism of Monty. The point is > that we all suffer from biases - not for any questionable reasons but > purely as a result of our own experiences, who we spend our time talking > to and what we spend our time thinking about - and therefore we should > all be extremely circumspect about trying to bake our own mental models > of what OpenStack should be into the organisational structure of the > project itself. > > * Assumption #1: The purpose of OpenStack is to provide a Compute cloud > > This assumption is front-and-centre throughout everything Monty wrote. > Yet this wasn't how the OpenStack project started. In fact there are now > at least three services - Swift, Nova, Zaqar - that could each make > sense as the core of a standalone product. > > Yes, it's true that Nova effectively depends on Glance and Neutron (and > everything depends on Keystone). We should definitely document that > somewhere. But why does it make Nova special? > > * Assumption #2: Yawnoc's Law > > Don't bother Googling that, I just made it up. It's the reverse of > Conway's Law: > > Infra engineers who design governance structures for OpenStack are > constrained to produce designs that are copies of the structure of > Tempest. > > I just don't understand why that needs to be the case. Currently, for > understandable historic reasons, every project gates against every other > project. That makes no sense any more, completely independently of the > project governance structure. We should just change it! There is no > organisational obstacle to changing how gating works. > > Even this proposal doesn't entirely make sense on this front - e.g. > Designate requires only Neutron and Keystone... why should Nova, Glance > and every other project in "Layer 1" gate against it, and vice-versa? > > I suggested in another thread[1] a model where each project would > publish a set of tests, each project would decide which sets of tests to > pull in and gate on, and Tempest would just be a shell for setting up > the environment and running the selected tests. Maybe that idea is crazy > or at least needs more work (it certainly met with only crickets and > tumbleweeds on the mailing list), but implementing it wouldn't require > TC intervention and certainly not by-laws changes. It just requires... > implementing it. > > Perhaps the idea here is that by designating "Layer 1" the TC is > indicating to projects which other projects they should accept gate test > jo
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Thu, Sep 25, 2014 at 3:55 AM, Zane Bitter wrote: > On 18/09/14 14:53, Monty Taylor wrote: > >> Hey all, >> >> I've recently been thinking a lot about Sean's Layers stuff. So I wrote >> a blog post which Jim Blair and Devananda were kind enough to help me >> edit. >> >> http://inaugust.com/post/108 >> > > Thanks Monty, I think there are some very interesting ideas in here. > > I'm particularly glad to see the 'big tent' camp reasserting itself, > because I have no sympathy with anyone who wants to join the OpenStack > community and then bolt the door behind them. Anyone who contributes to a > project that is related to OpenStack's goals, is willing to do things the > OpenStack way, and submits itself to the scrutiny of the TC deserves to be > treated as a member of our community with voting rights, entry to the > Design Summit and so on. > > I'm curious how you're suggesting we decide which projects satisfy those > criteria though. Up until now, we've done it through the incubation process > (or technically, the new program approval process... but in practice we've > never added a project that was targeted for eventual inclusion in the > integrated release to a program without incubating it). Would the TC > continue to judge whether a project is doing things the OpenStack way prior > to inclusion, or would we let projects self-certify? What does it mean for > a project to submit itself to TC scrutiny if it knows that realistically > the TC will never have time to actually scrutinise it? Or are you not > suggesting a change to the current incubation process, just a willingness > to incubate multiple projects in the same problem space? > > I feel like I need to play devil's advocate here, because overall I'm just > not sure I understand the purpose of arbitrarily - and it *is* arbitrary - > declaring "Layer #1" to be anything required to run Wordpress. To anyone > whose goal is not to run Wordpress, how is that relevant? > > Speaking of arbitrary, I had to laugh a little at this bit: > > Also, please someone notice that the above is too many steps and should > be: > > openstack boot gentoo on-a 2G-VM with-a publicIP with-a 10G-volume > call-it blog.inaugust.com > > That's kinda sorta exactly what Heat does ;) Minus the part about assuming > there is only one kind of application, obviously. > > :-) > > I think there are a number of unjustified assumptions behind this > arrangement of things. I'm going to list some here, but I don't want anyone > to interpret this as a personal criticism of Monty. The point is that we > all suffer from biases - not for any questionable reasons but purely as a > result of our own experiences, who we spend our time talking to and what we > spend our time thinking about - and therefore we should all be extremely > circumspect about trying to bake our own mental models of what OpenStack > should be into the organisational structure of the project itself. > > * Assumption #1: The purpose of OpenStack is to provide a Compute cloud > > This assumption is front-and-centre throughout everything Monty wrote. Yet > this wasn't how the OpenStack project started. In fact there are now at > least three services - Swift, Nova, Zaqar - that could each make sense as > the core of a standalone product. > Agree. > > Yes, it's true that Nova effectively depends on Glance and Neutron (and > everything depends on Keystone). We should definitely document that > somewhere. But why does it make Nova special? > > * Assumption #2: Yawnoc's Law > > Don't bother Googling that, I just made it up. It's the reverse of > Conway's Law: > > Infra engineers who design governance structures for OpenStack are > constrained to produce designs that are copies of the structure of > Tempest. > > I just don't understand why that needs to be the case. Currently, for > understandable historic reasons, every project gates against every other > project. That makes no sense any more, completely independently of the > project governance structure. We should just change it! There is no > organisational obstacle to changing how gating works. > > Even this proposal doesn't entirely make sense on this front - e.g. > Designate requires only Neutron and Keystone... why should Nova, Glance and > every other project in "Layer 1" gate against it, and vice-versa? > > I suggested in another thread[1] a model where each project would publish > a set of tests, each project would decide which sets of tests to pull in > and gate on, and Tempest would just be a shell for setting up the > environment and running the selected tests. Maybe that idea is crazy or at > least needs more work (it certainly met with only crickets and tumbleweeds > on the mailing list), but implementing it wouldn't require TC intervention > and certainly not by-laws changes. It just requires... implementing it. > > Perhaps the idea here is that by designating "Layer 1" the TC is > indicating to projects which other projects they should accept gate t
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 2014-09-24 13:55:57 -0400 (-0400), Zane Bitter wrote: [...] > * Assumption #2: Yawnoc's Law > > Don't bother Googling that, I just made it up. It's the reverse of > Conway's Law: > > Infra engineers who design governance structures for OpenStack > are constrained to produce designs that are copies of the > structure of Tempest. > > I just don't understand why that needs to be the case. Currently, > for understandable historic reasons, every project gates against > every other project. That makes no sense any more, completely > independently of the project governance structure. We should just > change it! There is no organisational obstacle to changing how > gating works. [...] Note that to a great extent the current proliferation of integration testing was driven from the developers of many projects. The Infra and QA teams didn't just wake up one morning and decide to chuck all the projects in. Rewind a year or two and remember that we had massive amounts of asymmetry in our testing. Project A would implement some new change and Project B developers would get their jobs insta-broken, then come complaining that clearly this meant we should be gating Project A on whether or not Project B works. For a time we had sufficient (human and server) resources to do that, and so it was comparatively cheap to just keep expanding the list of projects which shared a common set of jobs we hoped exercised enough of everything to act as a canary in the coal mine. We're now running into pretty clear scalability limits on the number of development teams whose changes you can effectively test against each other given the realities of nondeterministic failures, breakdowns in cross-group communication, growth in size of integrated releases, "tragedy of the commons" effects on horizontal efforts/shared resources, external factors like dependency changes, et cetera. As a community we should explore solutions (and clearly are) to these underlying problems, but also need to reconsider some old habits that need changing such as tight coupling to the internal APIs and implementation details of other projects... especially if doing so lets us scale back our integration testing, rather than leaning on it harder so that these undesirable development patterns can continue unabated. -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 09/24/2014 02:48 PM, Clint Byrum wrote: Excerpts from Robert Collins's message of 2014-09-23 21:14:47 -0700: No one helped me edit this :) http://rbtcollins.wordpress.com/2014/09/24/what-poles-for-the-tent/ I hope I haven't zoned out and just channelled someone else here ;) This sounds like "API's are what matters." You did spend some time working with Simon Wardley, didn't you? ;) I think it's a sound argument, but I'd like to banish the term "reference implementation" from any discussions around what OpenStack, as a project, delivers. It has too many negative feelings wrapped up in it. I also want to call attention to how what you describe feels an awful lot like POSIX to me. Basically offering guarantees of API compatibility, but then letting vendors run wild around and behind it. I'm not sure if that is a good thing, or a bad thing. I do, however, think if we can avoid a massive vendor battle that involves multiple vendors pushing multiple implementations, we will save our companies a lot of money, and our users will get what they need sooner. I like what Rob had to say here, and have expressed similar views. Having competition between implementations is good for every one (except for the losers) if that competition takes place in a way that shields users and the ecosystem from the aftermath of such competition. That is what standards, defined apis, whetever we want to call it, is all about. By analogy, competition by electronics companies around who can make the best performing blu-ray player with the most features is a good thing for users and that ecosystem. Competition about whether the ecosystem should use blu-ray or HD DVD, not so much: http://en.wikipedia.org/wiki/High_definition_optical_disc_format_war. This is what I see as the main virtue of the TC blessing things as "the one OpenStack way to do X". There is also the potential of efficiency if more people contribute to the same project that is doing X as compared to multiple projects doing X. But as we have seen, that efficiency is only realized if X turns out to be "the right thing". There is no particular reason to think the TC will be great at picking winners. Blessing apis, though difficult, would have huge benefit and provide more room for leeway and experimentation. Blessing code, before it has been proven in the real world, is the worst of all worlds when it turns out to be wrong. I believe our scale problems can be addressed by thoughtful decentralization and I hope we move in that direction, and in terms of how many pieces of the "run a real cloud" we have in our tent, we may have shot too high. But some of the recent proposals to move to an extreme in the other direction would be a mistake IMO. To be important, and be competitive with non-OpenStack cloud solutions, we need to provide a critical mass so that most other interesting things can glom on and form a larger ecosystem. -David ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 19/09/14 22:37, Monty Taylor wrote: I think we can do what you're saying and generalize a little bit. What if we declared programs, as needed, when we think there is a need to "pick a winner". (I think we can all agree that early winner picking is an unintended but very real side effect of the current system) Ooh, a challenge. I'll bite ;) Here's a question: how many instances are there of two Stackforge projects working on the same problem domain? I can think of one example: Gnocchi and StackTach - though those are (respectively) a branch of and a sort-of competitor to an integrated project (where we supposedly "picked the winner" already). So we have evidence of competitors surviving the picking of a winner, but not a lot of evidence of competition in general. (To be fair, you could make an argument that people are being put off starting competing projects by the prospect of eventually having to submit to winner-picking by the TC.) Don't get me wrong, I think it's clear that our implicit hope for the current model - that picking a winner would mean everyone getting on board with that project and making it better - has failed to materialise. But there's also every reason to think that a model where we rely on competition between projects in the marketplace to determine a winner owes just as much to wishful thinking. I suspect that part of the problem is that there are so many different things any given person could be working on to make users' lives better that when they see a project whose approach they don't agree with, they're more likely to just go work on something else than start a competitor or jump in to try and change the direction. (In fact, the people most qualified to do so are almost by definition the most busy already.) Maybe just throw a few rocks right before the graduation review, that sort of thing. If you were hoping this email would end with some kind of proposed solution, prepare for disappointment. Early winner-picking obviously sucks for the developer who realises that they need to make major changes and has to deal with the extra challenge of maintaining API compatibility and upgradability while doing it. On the other hand, it sucks precisely because that's great for users and operators. A competition model, even assuming that the competition actually arises, just means that the portion of operators who chose the 'wrong' side and their users get hosed when an eventual winner emerges. Potential consequences include delayed interoperability, delayed adoption of new features altogether to avoid the risk, or even permanent lack of interoperability with proprietary solutions being used instead. The only suggestion I can make is one I mentioned in another thread[1]: establish a design principle that the parts of the design which are hard to change (e.g. APIs) must be a simple as possible in order to provide the maximum flexibility of implementation, until such time as both the implementation and the need for more complexity have been validated in the real world. cheers, Zane. [1] http://lists.openstack.org/pipermail/openstack-dev/2014-September/046563.html ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Excerpts from Robert Collins's message of 2014-09-23 21:14:47 -0700: > No one helped me edit this :) > > http://rbtcollins.wordpress.com/2014/09/24/what-poles-for-the-tent/ > > I hope I haven't zoned out and just channelled someone else here ;) > This sounds like "API's are what matters." You did spend some time working with Simon Wardley, didn't you? ;) I think it's a sound argument, but I'd like to banish the term "reference implementation" from any discussions around what OpenStack, as a project, delivers. It has too many negative feelings wrapped up in it. I also want to call attention to how what you describe feels an awful lot like POSIX to me. Basically offering guarantees of API compatibility, but then letting vendors run wild around and behind it. I'm not sure if that is a good thing, or a bad thing. I do, however, think if we can avoid a massive vendor battle that involves multiple vendors pushing multiple implementations, we will save our companies a lot of money, and our users will get what they need sooner. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 18/09/14 14:53, Monty Taylor wrote: Hey all, I've recently been thinking a lot about Sean's Layers stuff. So I wrote a blog post which Jim Blair and Devananda were kind enough to help me edit. http://inaugust.com/post/108 Thanks Monty, I think there are some very interesting ideas in here. I'm particularly glad to see the 'big tent' camp reasserting itself, because I have no sympathy with anyone who wants to join the OpenStack community and then bolt the door behind them. Anyone who contributes to a project that is related to OpenStack's goals, is willing to do things the OpenStack way, and submits itself to the scrutiny of the TC deserves to be treated as a member of our community with voting rights, entry to the Design Summit and so on. I'm curious how you're suggesting we decide which projects satisfy those criteria though. Up until now, we've done it through the incubation process (or technically, the new program approval process... but in practice we've never added a project that was targeted for eventual inclusion in the integrated release to a program without incubating it). Would the TC continue to judge whether a project is doing things the OpenStack way prior to inclusion, or would we let projects self-certify? What does it mean for a project to submit itself to TC scrutiny if it knows that realistically the TC will never have time to actually scrutinise it? Or are you not suggesting a change to the current incubation process, just a willingness to incubate multiple projects in the same problem space? I feel like I need to play devil's advocate here, because overall I'm just not sure I understand the purpose of arbitrarily - and it *is* arbitrary - declaring "Layer #1" to be anything required to run Wordpress. To anyone whose goal is not to run Wordpress, how is that relevant? Speaking of arbitrary, I had to laugh a little at this bit: Also, please someone notice that the above is too many steps and should be: openstack boot gentoo on-a 2G-VM with-a publicIP with-a 10G-volume call-it blog.inaugust.com That's kinda sorta exactly what Heat does ;) Minus the part about assuming there is only one kind of application, obviously. I think there are a number of unjustified assumptions behind this arrangement of things. I'm going to list some here, but I don't want anyone to interpret this as a personal criticism of Monty. The point is that we all suffer from biases - not for any questionable reasons but purely as a result of our own experiences, who we spend our time talking to and what we spend our time thinking about - and therefore we should all be extremely circumspect about trying to bake our own mental models of what OpenStack should be into the organisational structure of the project itself. * Assumption #1: The purpose of OpenStack is to provide a Compute cloud This assumption is front-and-centre throughout everything Monty wrote. Yet this wasn't how the OpenStack project started. In fact there are now at least three services - Swift, Nova, Zaqar - that could each make sense as the core of a standalone product. Yes, it's true that Nova effectively depends on Glance and Neutron (and everything depends on Keystone). We should definitely document that somewhere. But why does it make Nova special? * Assumption #2: Yawnoc's Law Don't bother Googling that, I just made it up. It's the reverse of Conway's Law: Infra engineers who design governance structures for OpenStack are constrained to produce designs that are copies of the structure of Tempest. I just don't understand why that needs to be the case. Currently, for understandable historic reasons, every project gates against every other project. That makes no sense any more, completely independently of the project governance structure. We should just change it! There is no organisational obstacle to changing how gating works. Even this proposal doesn't entirely make sense on this front - e.g. Designate requires only Neutron and Keystone... why should Nova, Glance and every other project in "Layer 1" gate against it, and vice-versa? I suggested in another thread[1] a model where each project would publish a set of tests, each project would decide which sets of tests to pull in and gate on, and Tempest would just be a shell for setting up the environment and running the selected tests. Maybe that idea is crazy or at least needs more work (it certainly met with only crickets and tumbleweeds on the mailing list), but implementing it wouldn't require TC intervention and certainly not by-laws changes. It just requires... implementing it. Perhaps the idea here is that by designating "Layer 1" the TC is indicating to projects which other projects they should accept gate test jobs from (a function previously fulfilled by Incubation). I'd argue that this is a very bad way to do it, because (a) it says nothing to projects outside of "Layer 1" how they should
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Thanks for the summary, Sean. I couldn't follow the thread and this pointed me to the things I needed to read. On 09/24/2014 02:44 PM, Sean Dague wrote: > On 09/18/2014 02:53 PM, Monty Taylor wrote: >> Hey all, >> >> I've recently been thinking a lot about Sean's Layers stuff. So I wrote >> a blog post which Jim Blair and Devananda were kind enough to help me edit. >> >> http://inaugust.com/post/108 > > When I first read Monty's post, my basic reaction was "yes, please". > > I think there are plenty of devils in the details, but all of which we > can work through, we're mostly all reasonable people. > > A couple of follow ups for parts of the thread: > > Concerning the summit: while I understand ttx's concerns at - > http://lists.openstack.org/pipermail/openstack-dev/2014-September/046504.html > - my experience with the summits is the in project alignment isn't being > well served by current format. The absolutely most valuable parts of the > last summit for me were the Operator meetup sessions, and some of the > cross project sessions. > > I think there is an interesting question of what does the TC govern. > Honestly, I'm more in the camp that the TC focus should be on the > Foundational Infrastructure (hey, new words, not sure if they are any > better than layer 1 or ring 0), and have the ecosystem largely > outside TC governance per Joe / Vish's ASF model - > http://lists.openstack.org/pipermail/openstack-dev/2014-September/046877.html. > There are pragmatic reasons for that, which is a TC that's based around > that Foundation will tend to have more shared context about how we make > that better and move forward. > > I'm not sure I see the point of a TC who's main job is ranking 100s of > ecosystem projects on their production readiness... when most of them > don't run production clouds. > > I really like markmc's point about Production Ready being something the > User Committee should probably have more of a hand in - > http://lists.openstack.org/pipermail/openstack-dev/2014-September/046653.html. > I actually think some kind of self certification by projects to make > them easy to evaluate by potential consumers would be really handy. This > template might be a good thing to co-evolve between the User Committee > and the TC. > > I'm completely happy getting rid of incubation, given that we're talking > about a basically static foundation. I think the process of raising TC > expectations on projects this past year exposed an interesting fact that > there were things some of us felt were core values of OpenStack, that a > lot of projects weren't doing. Our approach was "they are doing it > wrong" and to put them on an improvement plan. But I think Monty's > slicing up of things brings out an interesting point. Maybe they were > doing it fine, they just weren't part of the particular shared culture > needed to build foundational infrastructure. Maybe that was ok, because > they weren't actually part of that. > > So, honestly, I'd say full speed ahead on Monty's plan. Is it perfect, > probably not. But I think it's a demonstrable move towards a more > sustainable base, a more inclusive ecosystem, and a better consumption > experience by our users. So how do we put a big stamp on it and make > this the direction we are headed in? I like the way this is shaping out, I believe what has been discussed so far allows for a more open and healthy ecosystem. To the above, I'd like us to be very careful on providing enough mentoring and guidance to new projects. Instead of incubation, which I'm find with getting rid of, it'd be great to have a program/team responsible for providing this technical mentoring. Other than that, this looks quite good to me. Cheers, Flavio -- @flaper87 Flavio Percoco ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 09/18/2014 02:53 PM, Monty Taylor wrote: > Hey all, > > I've recently been thinking a lot about Sean's Layers stuff. So I wrote > a blog post which Jim Blair and Devananda were kind enough to help me edit. > > http://inaugust.com/post/108 When I first read Monty's post, my basic reaction was "yes, please". I think there are plenty of devils in the details, but all of which we can work through, we're mostly all reasonable people. A couple of follow ups for parts of the thread: Concerning the summit: while I understand ttx's concerns at - http://lists.openstack.org/pipermail/openstack-dev/2014-September/046504.html - my experience with the summits is the in project alignment isn't being well served by current format. The absolutely most valuable parts of the last summit for me were the Operator meetup sessions, and some of the cross project sessions. I think there is an interesting question of what does the TC govern. Honestly, I'm more in the camp that the TC focus should be on the Foundational Infrastructure (hey, new words, not sure if they are any better than layer 1 or ring 0), and have the ecosystem largely outside TC governance per Joe / Vish's ASF model - http://lists.openstack.org/pipermail/openstack-dev/2014-September/046877.html. There are pragmatic reasons for that, which is a TC that's based around that Foundation will tend to have more shared context about how we make that better and move forward. I'm not sure I see the point of a TC who's main job is ranking 100s of ecosystem projects on their production readiness... when most of them don't run production clouds. I really like markmc's point about Production Ready being something the User Committee should probably have more of a hand in - http://lists.openstack.org/pipermail/openstack-dev/2014-September/046653.html. I actually think some kind of self certification by projects to make them easy to evaluate by potential consumers would be really handy. This template might be a good thing to co-evolve between the User Committee and the TC. I'm completely happy getting rid of incubation, given that we're talking about a basically static foundation. I think the process of raising TC expectations on projects this past year exposed an interesting fact that there were things some of us felt were core values of OpenStack, that a lot of projects weren't doing. Our approach was "they are doing it wrong" and to put them on an improvement plan. But I think Monty's slicing up of things brings out an interesting point. Maybe they were doing it fine, they just weren't part of the particular shared culture needed to build foundational infrastructure. Maybe that was ok, because they weren't actually part of that. So, honestly, I'd say full speed ahead on Monty's plan. Is it perfect, probably not. But I think it's a demonstrable move towards a more sustainable base, a more inclusive ecosystem, and a better consumption experience by our users. So how do we put a big stamp on it and make this the direction we are headed in? -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
I think Joe's idea pretty sums it up, ASF model is definitely worth following (Mesos is awesome). Non layer #1 projects will still be shepherded but not that closely coupled to make OpenStack over-bloated. Incubation projects can't be just dropped. On Wed, Sep 24, 2014 at 2:46 AM, Joe Gordon wrote: > > > On Tue, Sep 23, 2014 at 9:50 AM, Vishvananda Ishaya > wrote: > >> >> On Sep 23, 2014, at 8:40 AM, Doug Hellmann wrote: >> >> > If we are no longer incubating *programs*, which are the teams of >> people who we would like to ensure are involved in OpenStack governance, >> then how do we make that decision? From a practical standpoint, how do we >> make a list of eligible voters for a TC election? Today we pull a list of >> committers from the git history from the projects associated with “official >> programs", but if we are dropping “official programs” we need some other >> way to build the list. >> >> Joe Gordon mentioned an interesting idea to address this (which I am >> probably totally butchering), which is that we make incubation more similar >> to the ASF Incubator. In other words make it more lightweight with no >> promise of governance or infrastructure support. >> > > you only slightly butchered it :). From what I gather the Apache Software > Foundation primary goals are to: > > " > * provide a foundation for open, collaborative software development > projects by supplying hardware, communication, and business infrastructure > * create an independent legal entity to which companies and individuals > can donate resources and be assured that those resources will be used for > the public benefit > * provide a means for individual volunteers to be sheltered from legal > suits directed at the Foundation's projects > * protect the 'Apache' brand, as applied to its software products, from > being abused by other organizations > "[0] > > This roughly translates into: JIRA, SVN, Bugzilla and Confluence etc. > for infrastructure resources. So ASF provides infrastructure, legal > support, a trademark and some basic oversight. > > > "The [Apache] incubator is responsible for: > * filtering the proposals about the creation of a new project or > sub-project > * help the creation of the project and the infrastructure that it needs to > operate > * supervise and mentor the incubated community in order for them to reach > an open meritocratic environment > * evaluate the maturity of the incubated project, either promoting it to > official project/ sub-project status or by retiring it, in case of failure. > > It must be noted that the incubator (just like the board) does not perform > filtering on the basis of technical issues. This is because the foundation > respects and suggests variety of technical approaches. It doesn't fear > innovation or even internal confrontation between projects which overlap in > functionality." [1] > > So my idea, which is very similar to Monty's, is to make move all the > non-layer 1 projects into something closer to an ASF model where there is > still incubation and graduation. But the only things a project receives out > of this process is: > > * Legal support > * A trademark > * Mentorship > * Infrastructure to use > * Basic oversight via the incubation/graduation process with respect to > the health of the community. > > They do not get: > > * Required co-gating or integration with any other projects > * People to right there docs for them, etc. > * Technical review/oversight > * Technical requirements > * Evaluation on how the project fits into a bigger picture > * Language requirements > * etc. > > Note: this is just an idea, not a fully formed proposal > > [0] http://www.apache.org/foundation/how-it-works.html#what > [1] http://www.apache.org/foundation/how-it-works.html#incubator > > >> >> It is also interesting to consider that we may not need much governance >> for things outside of layer1. Of course, this may be dancing around the >> actual problem to some extent, because there are a bunch of projects that >> are not layer1 that are already a part of the community, and we need a >> solution that includes them somehow. >> >> Vish >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Zhipeng Huang Research Assistant Mobile Ad-Hoc Network Lab, Calit2 University of California, Irvine Email: zhipe...@uci.edu Office: Calit2 Building Room 2402 OpenStack, OpenDaylight, OpenCompute affcienado ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
No one helped me edit this :) http://rbtcollins.wordpress.com/2014/09/24/what-poles-for-the-tent/ I hope I haven't zoned out and just channelled someone else here ;) -Rob On 19 September 2014 06:53, Monty Taylor wrote: > Hey all, > > I've recently been thinking a lot about Sean's Layers stuff. So I wrote > a blog post which Jim Blair and Devananda were kind enough to help me edit. > > http://inaugust.com/post/108 > > Enjoy. > > Monty > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Robert Collins Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Sep 23, 2014, at 12:52 PM, Thierry Carrez wrote: > Devananda van der Veen wrote: >> On Tue, Sep 23, 2014 at 8:40 AM, Doug Hellmann wrote: >>> One is a technical discussion that has nothing at all to do with >>> governance. The other is entirely about governance. >>> >>> If we are no longer incubating *programs*, which are the teams of people >>> who we would like to ensure are involved in OpenStack governance, then how >>> do we make that decision? From a practical standpoint, how do we make a >>> list of eligible voters for a TC election? Today we pull a list of >>> committers from the git history from the projects associated with “official >>> programs", but if we are dropping “official programs” we need some other >>> way to build the list. >> >> I don't think incubation ever applied to programs. Any program listed >> in >> http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml >> is "official" and gets voting rights starting in the election after it >> was added to that file. > > I confirm there never was incubation for programs. The only thing that > goes through incubation are projects that want to become part of the > integrated release. I’m having some trouble making my question clear. We’re getting hung up on terminology, and part of that is because I think I’m using old terms for describing things and their states of being under the proposed changed system. Let me try rephrasing my concern, but first let me frame the question by saying that whatever else we say it is, OpenStack is made of people. The proposal includes a number of relatively small changes that affect how those people work and what they work on, combined with one big change to how we decide who those people are. I’m asking for more details about that big change. While the proposal has many concrete components, it also is purposefully more vague on some points as a way to lower the barrier for defining new groups of contributors and creating new projects. I agree with many of those goals, but I think that some of the points on which the proposal is vague are important for us to spell out clearly before they can be implemented. One such point is, how do new people — not their code, but the contributors themselves — obtain a status within the overall OpenStack project granting them the option of participating in our governance? In the past, we have had formal votes by members of the Technical Committee to decide whether or not to accept new groups (at different times called projects and programs) as being official parts of the overall OpenStack project. There are guidelines for what standards those groups need to follow once they are official [1] and other guidelines for the software created by those groups at designated points in its lifecycle [2]. While I’m not finding a specific procedure documented the TC has historically voted on the official status of groups either in IRC meetings or using gerrit based on changes to files in the governance repository [3]. The request for a vote from the TC is usually initiated by a leader in the group asking for official status. The official status of the group, and the git repositories it manages, are used to build our community voter list, and that is why it is important to understand how the proposal we’re discussing is different from what we do now. The “Who we are and what we all work on” section touches on this, but does not actually describe a process to be followed. The guidelines listed there seem easy to meet, but who decides if they actually are met for a given person or group of people? What, if any, expectations are placed on the group after they are given official status? What, if any, benefits come with the designation? The proposal includes process changes, testing configuration changes, and feature proposals. Most of the items it covers are not related to governance, and evaluating them all together masks some of the more important aspects of the governance changes. We could just go change the way we define the integrated gate without changing anything else we do, for example. It would be easier to evaluate the proposed governance changes if they were a patch on the existing repository, leaving out all of the unrelated suggestions to be evaluated separately. Doug [1] http://git.openstack.org/cgit/openstack/governance/tree/reference/new-programs-requirements.rst [2] http://git.openstack.org/cgit/openstack/governance/tree/reference/incubation-integration-requirements.rst [3] http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml > >> I also don't think that Monty's proposal suggests that we drop >> programs. I think it's suggesting the opposite -- we allow *more* >> programs (and the projects associated with them) into the openstack/* >> fold without requiring them to join the integrated gating of the >> "layer #1" projects. > > Although the proposal might make t
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 09/23/2014 02:18 AM, Thierry Carrez wrote:
> The main goal of incubation, as we did it in the past cycles, is a
> learning period where the new project aligns enough with the existing
> ones so that it integrates with them (Horizon shows Sahara dashboard)
> and won't break them around release time (stability, co-gate, respect of
> release deadlines).
>
> If we have a strict set of projects in layer #1, I don't see the point
> of keeping incubation. We wouldn't add new projects to layer #1 (only
> project splits which do not really require incubations), and additions
> to the big tent are considered on social alignment only ("are you
> vaguely about cloud and do you follow the OpenStack way"). If there is
> nothing to graduate to, there is no need for incubation.
There's no need for incubation, as such, but it's worth taking the time
to think about the technical and social functions that incubation and
integration served (sometimes ineffectively, or only as side-effects),
and what will replace them. You've identified a few there:
- learning period for new projects
- alignment with existing projects
- stability (in which gating served as a weak crutch, and the real
answer will likely lie in more extensive cross-project communication,
also carefully filtered to avoid information overload)
- respect of release deadlines (which doesn't necessarily mean releasing
all at the same time, just being cognizant of network-effects of
releases, and the cadence of other projects in an up-or-down dependency
relationship with yours)
> In Monty's proposal, ATC status would be linked to contributions to the
> big tent. Projects apply to become part of it, subject themselves to the
> oversight of the Technical Committee, and get the right to elect TC
> members in return.
And, a few more here:
- transitioning from "island" to part of the big tent
- accepting oversight of TC
- accepting responsibility to participate in TC election
Allison
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Tue, Sep 23, 2014 at 9:50 AM, Vishvananda Ishaya wrote: > > On Sep 23, 2014, at 8:40 AM, Doug Hellmann wrote: > > > If we are no longer incubating *programs*, which are the teams of people > who we would like to ensure are involved in OpenStack governance, then how > do we make that decision? From a practical standpoint, how do we make a > list of eligible voters for a TC election? Today we pull a list of > committers from the git history from the projects associated with “official > programs", but if we are dropping “official programs” we need some other > way to build the list. > > Joe Gordon mentioned an interesting idea to address this (which I am > probably totally butchering), which is that we make incubation more similar > to the ASF Incubator. In other words make it more lightweight with no > promise of governance or infrastructure support. > you only slightly butchered it :). From what I gather the Apache Software Foundation primary goals are to: " * provide a foundation for open, collaborative software development projects by supplying hardware, communication, and business infrastructure * create an independent legal entity to which companies and individuals can donate resources and be assured that those resources will be used for the public benefit * provide a means for individual volunteers to be sheltered from legal suits directed at the Foundation's projects * protect the 'Apache' brand, as applied to its software products, from being abused by other organizations "[0] This roughly translates into: JIRA, SVN, Bugzilla and Confluence etc. for infrastructure resources. So ASF provides infrastructure, legal support, a trademark and some basic oversight. "The [Apache] incubator is responsible for: * filtering the proposals about the creation of a new project or sub-project * help the creation of the project and the infrastructure that it needs to operate * supervise and mentor the incubated community in order for them to reach an open meritocratic environment * evaluate the maturity of the incubated project, either promoting it to official project/ sub-project status or by retiring it, in case of failure. It must be noted that the incubator (just like the board) does not perform filtering on the basis of technical issues. This is because the foundation respects and suggests variety of technical approaches. It doesn't fear innovation or even internal confrontation between projects which overlap in functionality." [1] So my idea, which is very similar to Monty's, is to make move all the non-layer 1 projects into something closer to an ASF model where there is still incubation and graduation. But the only things a project receives out of this process is: * Legal support * A trademark * Mentorship * Infrastructure to use * Basic oversight via the incubation/graduation process with respect to the health of the community. They do not get: * Required co-gating or integration with any other projects * People to right there docs for them, etc. * Technical review/oversight * Technical requirements * Evaluation on how the project fits into a bigger picture * Language requirements * etc. Note: this is just an idea, not a fully formed proposal [0] http://www.apache.org/foundation/how-it-works.html#what [1] http://www.apache.org/foundation/how-it-works.html#incubator > > It is also interesting to consider that we may not need much governance > for things outside of layer1. Of course, this may be dancing around the > actual problem to some extent, because there are a bunch of projects that > are not layer1 that are already a part of the community, and we need a > solution that includes them somehow. > > Vish > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Devananda van der Veen wrote: > On Tue, Sep 23, 2014 at 8:40 AM, Doug Hellmann wrote: >> One is a technical discussion that has nothing at all to do with governance. >> The other is entirely about governance. >> >> If we are no longer incubating *programs*, which are the teams of people who >> we would like to ensure are involved in OpenStack governance, then how do we >> make that decision? From a practical standpoint, how do we make a list of >> eligible voters for a TC election? Today we pull a list of committers from >> the git history from the projects associated with “official programs", but >> if we are dropping “official programs” we need some other way to build the >> list. > > I don't think incubation ever applied to programs. Any program listed > in > http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml > is "official" and gets voting rights starting in the election after it > was added to that file. I confirm there never was incubation for programs. The only thing that goes through incubation are projects that want to become part of the integrated release. > I also don't think that Monty's proposal suggests that we drop > programs. I think it's suggesting the opposite -- we allow *more* > programs (and the projects associated with them) into the openstack/* > fold without requiring them to join the integrated gating of the > "layer #1" projects. Although the proposal might make them so cheap we wouldn't need a formal word for them. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Sep 23, 2014, at 8:40 AM, Doug Hellmann wrote: > If we are no longer incubating *programs*, which are the teams of people who > we would like to ensure are involved in OpenStack governance, then how do we > make that decision? From a practical standpoint, how do we make a list of > eligible voters for a TC election? Today we pull a list of committers from > the git history from the projects associated with “official programs", but if > we are dropping “official programs” we need some other way to build the list. Joe Gordon mentioned an interesting idea to address this (which I am probably totally butchering), which is that we make incubation more similar to the ASF Incubator. In other words make it more lightweight with no promise of governance or infrastructure support. It is also interesting to consider that we may not need much governance for things outside of layer1. Of course, this may be dancing around the actual problem to some extent, because there are a bunch of projects that are not layer1 that are already a part of the community, and we need a solution that includes them somehow. Vish signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Doug Hellmann wrote:
> On Sep 23, 2014, at 5:18 AM, Thierry Carrez wrote:
>
>> Devananda van der Veen wrote:
>>> On Mon, Sep 22, 2014 at 2:27 PM, Doug Hellmann
>>> wrote:
On Sep 22, 2014, at 5:10 PM, Devananda van der Veen
wrote:
> One of the primary effects of integration, as far as the release
> process is concerned, is being allowed to co-gate with other
> integrated projects, and having those projects accept your changes
> (integrate back with the other project). That shouldn't be a TC
The point of integration is to add the projects to the integrated
*release*, not just the gate, because the release is the thing we have
said is OpenStack. Integration was about our overall project identity and
governance. The testing was a requirement to be accepted, not a goal.
>>>
>>> We have plenty of things which are clearly part of OpenStack, and yet
>>> which are not part of the Integrated Release. Oslo. Devstack. Zuul...
>>> As far as I can tell, the only time when "integrated release" equals
>>> "the thing we say is OpenStack" is when we're talking about the
>>> trademark.
>>
>> The main goal of incubation, as we did it in the past cycles, is a
>> learning period where the new project aligns enough with the existing
>> ones so that it integrates with them (Horizon shows Sahara dashboard)
>> and won't break them around release time (stability, co-gate, respect of
>> release deadlines).
>>
>> If we have a strict set of projects in layer #1, I don't see the point
>> of keeping incubation. We wouldn't add new projects to layer #1 (only
>> project splits which do not really require incubations), and additions
>> to the big tent are considered on social alignment only ("are you
>> vaguely about cloud and do you follow the OpenStack way"). If there is
>> nothing to graduate to, there is no need for incubation.
>>
Integration was about our overall project identity and governance. The
testing was a requirement to be accepted, not a goal.
>>>
>>> Project identity and governance are presently addressed by the
>>> creation of "Programs" and a fully-elected TC. Integration is not
>>> addressing these things at all, as far as I can tell, though I agree
>>> that it was initially intended to.
>>>
If there is no incubation process, and only a fixed list of projects will
be in that new layer 1 group, then do contributors to the other projects
have ATC status and vote for the TC? What is the basis for the TC
accepting any responsibility for the project, and for the project agreeing
to the TC’s leadership?
>>>
>>> I think a good basis for this is simply whether the developers of the
>>> project are part of our community, doing things in the way that we do
>>> things, and want this to happen. Voting and ATC status is already
>>> decoupled [0] from the integrated gate and the integrated release --
>>> it's based on the accepted list of Programs [1], which actually has
>>> nothing to do with incubation or integration [2].
>>
>> In Monty's proposal, ATC status would be linked to contributions to the
>> big tent. Projects apply to become part of it, subject themselves to the
>> oversight of the Technical Committee, and get the right to elect TC
>> members in return.
>
> That’s the part that wasn’t clear. If we’re not “incubating” those projects,
> then what criteria do we use to make decisions about the applications? Is a
> declaration of intent enough?
In Monty's proposal, the big tent is pretty welcoming. The bar is "are
you one of us":
> Some examples of community that we care about: being on stackforge rather
> than github; having a PTL who you elect rather than a BDFL; having meetings
> on IRC. "Do any of the people who hack on the project also hack on any other
> existing OpenStack projects, or are the people completely unconnected?" is a
> potential social touchstone as well.
>
> All in all, meeting the requirements for "being one of us" is not
> particularly hard, nor should it be.
It's a community values assessment... I don't see what "incubating"
would give us there, apart from preserving red tape.
--
Thierry Carrez (ttx)
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Tue, Sep 23, 2014 at 8:40 AM, Doug Hellmann wrote: > > On Sep 22, 2014, at 8:05 PM, Devananda van der Veen > wrote: > >> On Mon, Sep 22, 2014 at 2:27 PM, Doug Hellmann wrote: >>> >>> On Sep 22, 2014, at 5:10 PM, Devananda van der Veen >>> wrote: >>> One of the primary effects of integration, as far as the release process is concerned, is being allowed to co-gate with other integrated projects, and having those projects accept your changes (integrate back with the other project). That shouldn't be a TC >>> >>> The point of integration is to add the projects to the integrated >>> *release*, not just the gate, because the release is the thing we have said >>> is OpenStack. Integration was about our overall project identity and >>> governance. The testing was a requirement to be accepted, not a goal. >> >> We have plenty of things which are clearly part of OpenStack, and yet >> which are not part of the Integrated Release. Oslo. Devstack. Zuul... >> As far as I can tell, the only time when "integrated release" equals >> "the thing we say is OpenStack" is when we're talking about the >> trademark. >> >>> Integration was about our overall project identity and governance. The >>> testing was a requirement to be accepted, not a goal. >> >> Project identity and governance are presently addressed by the >> creation of "Programs" and a fully-elected TC. Integration is not >> addressing these things at all, as far as I can tell, though I agree >> that it was initially intended to. > > Good point: I’m mixing terms here. Programs and projects have tended to be > incubated at the same time. We’ve insisted that it doesn’t make sense to have > a program if there is nothing being produced, and that a project can’t be > incubated if the program isn’t also incubated. The fact that we’ve also had > 1:1 coupling between programs and projects is unfortunate, but orthogonal to > the fact that we have been evaluating the teams as well as the code. > >> >>> If there is no incubation process, and only a fixed list of projects will >>> be in that new layer 1 group, then do contributors to the other projects >>> have ATC status and vote for the TC? What is the basis for the TC accepting >>> any responsibility for the project, and for the project agreeing to the >>> TC’s leadership? >> >> I think a good basis for this is simply whether the developers of the >> project are part of our community, doing things in the way that we do >> things, and want this to happen. Voting and ATC status is already >> decoupled [0] from the integrated gate and the integrated release -- >> it's based on the accepted list of Programs [1], which actually has >> nothing to do with incubation or integration [2]. > > I’m concerned that we’re combining changes to the way we decide what we > include in the gate with changes to the way we decide which groups of people > have a say in how the overall OpenStack project is run. These things already weren't related. > One is a technical discussion that has nothing at all to do with governance. > The other is entirely about governance. > > If we are no longer incubating *programs*, which are the teams of people who > we would like to ensure are involved in OpenStack governance, then how do we > make that decision? From a practical standpoint, how do we make a list of > eligible voters for a TC election? Today we pull a list of committers from > the git history from the projects associated with “official programs", but if > we are dropping “official programs” we need some other way to build the list. I don't think incubation ever applied to programs. Any program listed in http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml is "official" and gets voting rights starting in the election after it was added to that file. I also don't think that Monty's proposal suggests that we drop programs. I think it's suggesting the opposite -- we allow *more* programs (and the projects associated with them) into the openstack/* fold without requiring them to join the integrated gating of the "layer #1" projects. -Devananda ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 22 September 2014 23:14, Robert Collins wrote: > I am not at all sure we've prevented other flowers blooming - > and I hate the idea that we have done that. I've certainly sat around at discussions which shut down hard with somebody making the statement that 'that is TripleO's field and they don't like ', it is a genuine if entirely unintentional problem. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Sep 22, 2014, at 8:05 PM, Devananda van der Veen wrote: > On Mon, Sep 22, 2014 at 2:27 PM, Doug Hellmann wrote: >> >> On Sep 22, 2014, at 5:10 PM, Devananda van der Veen >> wrote: >> >>> One of the primary effects of integration, as far as the release >>> process is concerned, is being allowed to co-gate with other >>> integrated projects, and having those projects accept your changes >>> (integrate back with the other project). That shouldn't be a TC >> >> The point of integration is to add the projects to the integrated *release*, >> not just the gate, because the release is the thing we have said is >> OpenStack. Integration was about our overall project identity and >> governance. The testing was a requirement to be accepted, not a goal. > > We have plenty of things which are clearly part of OpenStack, and yet > which are not part of the Integrated Release. Oslo. Devstack. Zuul... > As far as I can tell, the only time when "integrated release" equals > "the thing we say is OpenStack" is when we're talking about the > trademark. > >> Integration was about our overall project identity and governance. The >> testing was a requirement to be accepted, not a goal. > > Project identity and governance are presently addressed by the > creation of "Programs" and a fully-elected TC. Integration is not > addressing these things at all, as far as I can tell, though I agree > that it was initially intended to. Good point: I’m mixing terms here. Programs and projects have tended to be incubated at the same time. We’ve insisted that it doesn’t make sense to have a program if there is nothing being produced, and that a project can’t be incubated if the program isn’t also incubated. The fact that we’ve also had 1:1 coupling between programs and projects is unfortunate, but orthogonal to the fact that we have been evaluating the teams as well as the code. > >> If there is no incubation process, and only a fixed list of projects will be >> in that new layer 1 group, then do contributors to the other projects have >> ATC status and vote for the TC? What is the basis for the TC accepting any >> responsibility for the project, and for the project agreeing to the TC’s >> leadership? > > I think a good basis for this is simply whether the developers of the > project are part of our community, doing things in the way that we do > things, and want this to happen. Voting and ATC status is already > decoupled [0] from the integrated gate and the integrated release -- > it's based on the accepted list of Programs [1], which actually has > nothing to do with incubation or integration [2]. I’m concerned that we’re combining changes to the way we decide what we include in the gate with changes to the way we decide which groups of people have a say in how the overall OpenStack project is run. One is a technical discussion that has nothing at all to do with governance. The other is entirely about governance. If we are no longer incubating *programs*, which are the teams of people who we would like to ensure are involved in OpenStack governance, then how do we make that decision? From a practical standpoint, how do we make a list of eligible voters for a TC election? Today we pull a list of committers from the git history from the projects associated with “official programs", but if we are dropping “official programs” we need some other way to build the list. Doug > > > -Devananda > > [0] > http://git.openstack.org/cgit/openstack/governance/tree/reference/charter.rst#n132 > > [1] > http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml > > [2] > http://git.openstack.org/cgit/openstack/governance/tree/reference/new-programs-requirements.rst > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Sep 23, 2014, at 5:18 AM, Thierry Carrez wrote:
> Devananda van der Veen wrote:
>> On Mon, Sep 22, 2014 at 2:27 PM, Doug Hellmann wrote:
>>> On Sep 22, 2014, at 5:10 PM, Devananda van der Veen
>>> wrote:
>>>
One of the primary effects of integration, as far as the release
process is concerned, is being allowed to co-gate with other
integrated projects, and having those projects accept your changes
(integrate back with the other project). That shouldn't be a TC
>>>
>>> The point of integration is to add the projects to the integrated
>>> *release*, not just the gate, because the release is the thing we have said
>>> is OpenStack. Integration was about our overall project identity and
>>> governance. The testing was a requirement to be accepted, not a goal.
>>
>> We have plenty of things which are clearly part of OpenStack, and yet
>> which are not part of the Integrated Release. Oslo. Devstack. Zuul...
>> As far as I can tell, the only time when "integrated release" equals
>> "the thing we say is OpenStack" is when we're talking about the
>> trademark.
>
> The main goal of incubation, as we did it in the past cycles, is a
> learning period where the new project aligns enough with the existing
> ones so that it integrates with them (Horizon shows Sahara dashboard)
> and won't break them around release time (stability, co-gate, respect of
> release deadlines).
>
> If we have a strict set of projects in layer #1, I don't see the point
> of keeping incubation. We wouldn't add new projects to layer #1 (only
> project splits which do not really require incubations), and additions
> to the big tent are considered on social alignment only ("are you
> vaguely about cloud and do you follow the OpenStack way"). If there is
> nothing to graduate to, there is no need for incubation.
>
>>> Integration was about our overall project identity and governance. The
>>> testing was a requirement to be accepted, not a goal.
>>
>> Project identity and governance are presently addressed by the
>> creation of "Programs" and a fully-elected TC. Integration is not
>> addressing these things at all, as far as I can tell, though I agree
>> that it was initially intended to.
>>
>>> If there is no incubation process, and only a fixed list of projects will
>>> be in that new layer 1 group, then do contributors to the other projects
>>> have ATC status and vote for the TC? What is the basis for the TC accepting
>>> any responsibility for the project, and for the project agreeing to the
>>> TC’s leadership?
>>
>> I think a good basis for this is simply whether the developers of the
>> project are part of our community, doing things in the way that we do
>> things, and want this to happen. Voting and ATC status is already
>> decoupled [0] from the integrated gate and the integrated release --
>> it's based on the accepted list of Programs [1], which actually has
>> nothing to do with incubation or integration [2].
>
> In Monty's proposal, ATC status would be linked to contributions to the
> big tent. Projects apply to become part of it, subject themselves to the
> oversight of the Technical Committee, and get the right to elect TC
> members in return.
That’s the part that wasn’t clear. If we’re not “incubating” those projects,
then what criteria do we use to make decisions about the applications? Is a
declaration of intent enough?
Doug
>
> --
> Thierry Carrez (ttx)
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Robert Collins wrote: > On 19 September 2014 22:29, Thierry Carrez wrote: > ... >> current Heat team is not really interested in maintaining them. What's >> the point of being under the same program then ? And TripleO is not the >> only way to deploy OpenStack, but its mere existence (and name) >> prevented other flowers to bloom in our community. > > So I've a small issue here - *after* TripleO became an official > program Tuskar was started - and we embraced and shuffled things to > collaborate. There's a tripleo puppet repository, an ansible one > coming soon I believe, and I'm hearing rumours of a spike using > another thing altogether (whose thunder I don't want to steal so I'm > going to be vague :)). We've collaborated to a degree with Fuel and > Crowbar: I am not at all sure we've prevented other flowers blooming - > and I hate the idea that we have done that. I agree that you went out of your way to be inclusive, but having a single PTL for a group of competing alternatives is a bit weird. As far as "preventing blooming", that's something we've heard from the Board of Directors (as part of an objection to calling the TripleO program "OpenStack Deployment"). > ... >> ## The release and the development cycle >> >> You touch briefly on the consequences of your model for the "common >> release" and our development cycle. Obviously we would release the ring >> 0 projects in an integrated manner on a time-based schedule. > > I'm not at all sure we need to do that - I've long been suspicious of > the coordinated release. I see benefits to users in being able to grab > a new set of projects all at once, but they can do that irrespective > of our release process, as long as: > > *) we do releases > *) we do at least one per project per 6 month period > > Tying all our things together makes for hugely destablising waves of > changes and rushes: if we aimed at really smooth velocity and frequent > independent releases I think we could do a lot better: contracts and > interfaces are *useful* things for large scale architectures, and we > need to stop kidding ourselves - OpenStack is not a lean little > beastie anymore: its a large scale distributed system. Swift is doing > exactly the right thing today - I'd like to see more of that. Note that I'm only advocating that Monty's layer #1 things would be released in an integrated manner on a time-based schedule. That's not "all our things". All the other things would be released as-needed. That lets us focus on cleaning up interfaces between layer #1 and other things first. Swift is doing the right thing today because its loosely-integrated with layer #1 (and has a clean interface for doing so). So Swift is doing the right thing for an non-layer#1 project. Once our layer#1-internal interfaces are simple/stable enough that we can safely compose something that works at any time by picking the latest version of everything, we can discuss the benefits and the drawbacks of the "common release" model. I still think there are huge community benefits to sharing the same cycle, and gathering around a common "release" that can be advertised and consumed downstream... as far as layer #1 is concerned. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Devananda van der Veen wrote:
> On Mon, Sep 22, 2014 at 2:27 PM, Doug Hellmann wrote:
>> On Sep 22, 2014, at 5:10 PM, Devananda van der Veen
>> wrote:
>>
>>> One of the primary effects of integration, as far as the release
>>> process is concerned, is being allowed to co-gate with other
>>> integrated projects, and having those projects accept your changes
>>> (integrate back with the other project). That shouldn't be a TC
>>
>> The point of integration is to add the projects to the integrated *release*,
>> not just the gate, because the release is the thing we have said is
>> OpenStack. Integration was about our overall project identity and
>> governance. The testing was a requirement to be accepted, not a goal.
>
> We have plenty of things which are clearly part of OpenStack, and yet
> which are not part of the Integrated Release. Oslo. Devstack. Zuul...
> As far as I can tell, the only time when "integrated release" equals
> "the thing we say is OpenStack" is when we're talking about the
> trademark.
The main goal of incubation, as we did it in the past cycles, is a
learning period where the new project aligns enough with the existing
ones so that it integrates with them (Horizon shows Sahara dashboard)
and won't break them around release time (stability, co-gate, respect of
release deadlines).
If we have a strict set of projects in layer #1, I don't see the point
of keeping incubation. We wouldn't add new projects to layer #1 (only
project splits which do not really require incubations), and additions
to the big tent are considered on social alignment only ("are you
vaguely about cloud and do you follow the OpenStack way"). If there is
nothing to graduate to, there is no need for incubation.
>> Integration was about our overall project identity and governance. The
>> testing was a requirement to be accepted, not a goal.
>
> Project identity and governance are presently addressed by the
> creation of "Programs" and a fully-elected TC. Integration is not
> addressing these things at all, as far as I can tell, though I agree
> that it was initially intended to.
>
>> If there is no incubation process, and only a fixed list of projects will be
>> in that new layer 1 group, then do contributors to the other projects have
>> ATC status and vote for the TC? What is the basis for the TC accepting any
>> responsibility for the project, and for the project agreeing to the TC’s
>> leadership?
>
> I think a good basis for this is simply whether the developers of the
> project are part of our community, doing things in the way that we do
> things, and want this to happen. Voting and ATC status is already
> decoupled [0] from the integrated gate and the integrated release --
> it's based on the accepted list of Programs [1], which actually has
> nothing to do with incubation or integration [2].
In Monty's proposal, ATC status would be linked to contributions to the
big tent. Projects apply to become part of it, subject themselves to the
oversight of the Technical Committee, and get the right to elect TC
members in return.
--
Thierry Carrez (ttx)
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Mon, Sep 22, 2014 at 2:27 PM, Doug Hellmann wrote: > > On Sep 22, 2014, at 5:10 PM, Devananda van der Veen > wrote: > >> One of the primary effects of integration, as far as the release >> process is concerned, is being allowed to co-gate with other >> integrated projects, and having those projects accept your changes >> (integrate back with the other project). That shouldn't be a TC > > The point of integration is to add the projects to the integrated *release*, > not just the gate, because the release is the thing we have said is > OpenStack. Integration was about our overall project identity and governance. > The testing was a requirement to be accepted, not a goal. We have plenty of things which are clearly part of OpenStack, and yet which are not part of the Integrated Release. Oslo. Devstack. Zuul... As far as I can tell, the only time when "integrated release" equals "the thing we say is OpenStack" is when we're talking about the trademark. > Integration was about our overall project identity and governance. The > testing was a requirement to be accepted, not a goal. Project identity and governance are presently addressed by the creation of "Programs" and a fully-elected TC. Integration is not addressing these things at all, as far as I can tell, though I agree that it was initially intended to. > If there is no incubation process, and only a fixed list of projects will be > in that new layer 1 group, then do contributors to the other projects have > ATC status and vote for the TC? What is the basis for the TC accepting any > responsibility for the project, and for the project agreeing to the TC’s > leadership? I think a good basis for this is simply whether the developers of the project are part of our community, doing things in the way that we do things, and want this to happen. Voting and ATC status is already decoupled [0] from the integrated gate and the integrated release -- it's based on the accepted list of Programs [1], which actually has nothing to do with incubation or integration [2]. -Devananda [0] http://git.openstack.org/cgit/openstack/governance/tree/reference/charter.rst#n132 [1] http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml [2] http://git.openstack.org/cgit/openstack/governance/tree/reference/new-programs-requirements.rst ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 23 September 2014 10:32, Dean Troyer wrote: > tl;dr: we're not broken, but under stress...changing (outside) expectations > requires changing the expression of the model...while it's called a 'stack' > maybe it's multiple tiered stacks. MultiStack! ... > > This is one reason for multiple layers. The original 4 layer stack was > meant as a technical dependency stack but has morphed into a social/project > organizational stack. I don't think it is total coincidence that the > technical hierarchy was interpreted as a social/governance hierarchy by some > as there is a lot of similarity. The mapping between the two in my mind is > fairly easy, but those details is not what is important. Are you familiar with Conway's law? http://en.wikipedia.org/wiki/Conway's_law - I think it works both ways. When we drive towards a particular structure be it technical or social, expect the other side to follow suit. Its very hard to to have a social structure decoupled from the technical structure. One of the reasons Launchpad has such poor cross-component (bugs/code hosting/translations) features compared to the in-component features is that for a long time there were separate teams for each thing - which resulted in great domain knowledge, but the vast amount of work being done in-component. If we want to be better at 'cross project' things, I think we need to factor Conway's law into our *social* design process. I don't doubt we need separate teams - we're way past Dunbar's number, for any estimate of it - but I think we need to seriously consider making our teams align with our user needs, not our codebases. Codebases are an implementation detail - an important one, but a detail. Our goals, our focus, *our structure* should be on meeting user needs [including those users that do deployments], rather than structured around our code. > * IaaS thing: the stuff that builds excellent clouds > * PaaS thing: the stuff that does amazing things that may or may not be > built on top of excellent clouds > * XaaS thing(s): more things I can not visualize through the fog of > antihistamines > * Non-aas developer things: what enables us s developers to make the above > things (infra, qa, etc) > * Non-aas consumer[1] things: what enables the rest of the world to enjoy > the above things (docs, SDKs, clients, etc) > > This separates the technical hierarchical from the organizational > relationships. All of the above things are still called OpenStack. But > maybe it's a 'MultiStack'. I think this would result in excellent intra-layer features/quality etc, but perhaps cross-layer would become the poor stepchild. And - I think thats ok. I think it would help overall. -Rob -- Robert Collins Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
tl;dr: we're not broken, but under stress...changing (outside) expectations requires changing the expression of the model...while it's called a 'stack' maybe it's multiple tiered stacks. MultiStack! On Mon, Sep 22, 2014 at 4:27 PM, Doug Hellmann wrote: > The point of integration is to add the projects to the integrated > *release*, not just the gate, because the release is the thing we have said > is OpenStack. Integration was about our overall project identity and > governance. The testing was a requirement to be accepted, not a goal. > > If there is no incubation process, and only a fixed list of projects will > be in that new layer 1 group, then do contributors to the other projects > have ATC status and vote for the TC? What is the basis for the TC accepting > any responsibility for the project, and for the project agreeing to the > TC’s leadership? > This is one reason for multiple layers. The original 4 layer stack was meant as a technical dependency stack but has morphed into a social/project organizational stack. I don't think it is total coincidence that the technical hierarchy was interpreted as a social/governance hierarchy by some as there is a lot of similarity. The mapping between the two in my mind is fairly easy, but those details is not what is important. We love to look at the Apache Foundation for inspiration. In the current set of Apache projects most of them are not focused on adding value to a web server. They grew beyond that in multiple directions. I'm selfish and want to keep the layer nomenclature for the technical organization that helped re-structure DevStack and propose we think of these as *thing*[0] where we have Multiple *Things*: * IaaS thing: the stuff that builds excellent clouds * PaaS thing: the stuff that does amazing things that may or may not be built on top of excellent clouds * XaaS thing(s): more things I can not visualize through the fog of antihistamines * Non-aas developer things: what enables us s developers to make the above things (infra, qa, etc) * Non-aas consumer[1] things: what enables the rest of the world to enjoy the above things (docs, SDKs, clients, etc) This separates the technical hierarchical from the organizational relationships. All of the above things are still called OpenStack. But maybe it's a 'MultiStack'. - New projects wanting to join can apply and receive a 'provisional' attribute that tells the world we (OpenStack people) thing this looks promising and want to see if it matures to our standards. Similar to incubation but maybe the bar has some differences between the things. It _should_ require a higher bar to add to the foundation than a new deck on the side. - The integrated release still applies to a subset of the projects and/or things. The IaaS thing establishes the base release cycle other things apply common sense and follow along or release more often. - The test matrix is built using the technical layers and not the above organizational structure. I'm getting a bit hand-wavy here, but the point is to clearly state to the world that it isn't the organizational things that determine testing beyond having at least the 'provisional' attribute on a project. If the above feels very familiar it should. Some of it is terminology changes to help change expectations and some divisions based on use cases. What we have isn't totally broken, it is under growth related stress. Taking a different perspective on it will help expose where things can be improved. And what we are doing right. dt [0] TODO: cut-n-paste in actual allegorical noun [1] Yuck, better word here too! -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 19 September 2014 22:29, Thierry Carrez wrote: ... > current Heat team is not really interested in maintaining them. What's > the point of being under the same program then ? And TripleO is not the > only way to deploy OpenStack, but its mere existence (and name) > prevented other flowers to bloom in our community. So I've a small issue here - *after* TripleO became an official program Tuskar was started - and we embraced and shuffled things to collaborate. There's a tripleo puppet repository, an ansible one coming soon I believe, and I'm hearing rumours of a spike using another thing altogether (whose thunder I don't want to steal so I'm going to be vague :)). We've collaborated to a degree with Fuel and Crowbar: I am not at all sure we've prevented other flowers blooming - and I hate the idea that we have done that. ... > ## The release and the development cycle > > You touch briefly on the consequences of your model for the "common > release" and our development cycle. Obviously we would release the ring > 0 projects in an integrated manner on a time-based schedule. I'm not at all sure we need to do that - I've long been suspicious of the coordinated release. I see benefits to users in being able to grab a new set of projects all at once, but they can do that irrespective of our release process, as long as: *) we do releases *) we do at least one per project per 6 month period Tying all our things together makes for hugely destablising waves of changes and rushes: if we aimed at really smooth velocity and frequent independent releases I think we could do a lot better: contracts and interfaces are *useful* things for large scale architectures, and we need to stop kidding ourselves - OpenStack is not a lean little beastie anymore: its a large scale distributed system. Swift is doing exactly the right thing today - I'd like to see more of that. -Rob -- Robert Collins Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Sep 22, 2014, at 5:10 PM, Devananda van der Veen wrote: > On Mon, Sep 22, 2014 at 12:33 PM, Doug Hellmann wrote: >> >> On Sep 22, 2014, at 3:11 PM, Tim Bell wrote: >> >>> >>> On 22 Sep 2014, at 20:53, Doug Hellmann wrote: >>> On Sep 19, 2014, at 6:29 AM, Thierry Carrez wrote: > Monty Taylor wrote: >> I've recently been thinking a lot about Sean's Layers stuff. So I wrote >> a blog post which Jim Blair and Devananda were kind enough to help me >> edit. >> >> http://inaugust.com/post/108 > > Hey Monty, > > As you can imagine, I read that post with great attention. I generally > like the concept of a tightly integrated, limited-by-design layer #1 > (I'd personally call it "Ring 0") and a large collection of "OpenStack" > things gravitating around it. That would at least solve the attraction > of the integrated release, suppress the need for incubation, foster I’m not sure I see this change reducing the number of incubated projects unless we no longer incubate and graduate projects at all. Would everything just live on stackforge and have a quality designation instead of an “officialness” designation? Or would we have both? ATC status seems to imply we need some sort of officialness designation, as you mention below. >>> >>> The quality designation is really important for the operator community who >>> are trying to work out what we can give to our end users. >>> >>> Offering early helps to establish the real-life experience and give good >>> feedback on the designs. However, the operator then risks leaving their >>> users orphaned if the project does not get a sustainable following or >>> significant disruption if the APIs change. >>> >>> The packaging teams are key here as well. When do Ubuntu and Red Hat work >>> out the chain of pre-reqs etc. to produce installable packages, >>> packstack/juju tool support ? >>> >>> We do need to have some way to show that an layer #2 package is ready for >>> prime time production and associated criteria (packages available, docs >>> available, >1 company communities, models for HA and scale, …) >> >> >> Right. I’m trying to understand if we are talking about doing that *instead* >> of our existing incubation/graduation process, or in addition to that >> process as a new thing. I like the idea of adding a quality designation. I’m >> not sure replacing our existing process with that designation is a good idea. >> > > A "production ready" tag wouldn't replace the incubation process > because incubation isn't about that. Also, the incubation process > would go away because the TC wouldn't be in the business of adding > projects to the integrated gate any more. > > The content of integrated release would be based on a use-case-based > scope definition, which is satisfied by the projects in the "Layer #1" > list. I think it's interesting to think about other use-cases, and > those could well inform the trademark discussion in the long term, but > I think we need to start with one use case, and I would like to think > this is one we can all agree on as central to the mission of > OpenStack. That doesn't invalidate other use cases. And it doesn't > make things not in Layer #1 suddenly become not-OpenStack. (A triple > negative? Yep.) > > One of the primary effects of integration, as far as the release > process is concerned, is being allowed to co-gate with other > integrated projects, and having those projects accept your changes > (integrate back with the other project). That shouldn't be a TC The point of integration is to add the projects to the integrated *release*, not just the gate, because the release is the thing we have said is OpenStack. Integration was about our overall project identity and governance. The testing was a requirement to be accepted, not a goal. If there is no incubation process, and only a fixed list of projects will be in that new layer 1 group, then do contributors to the other projects have ATC status and vote for the TC? What is the basis for the TC accepting any responsibility for the project, and for the project agreeing to the TC’s leadership? Based on what you’ve said, I’m afraid that this change makes project governance murkier, and introduces challenges to accomplish some of the unifying cross-project work that we have recently discussed as being so important. > decision, and the decision to cogate with another project should be up > to the projects themselves. If Nova feels that supporting the Ironic > virt driver is important, Nova would agree to a two-project gate with > Ironic. Other projects that Nova also gates with would not need to > participate (eg, a change to Glance would also test Nova, but wouldn't > necessarily run the nova-ironic test). I think this can be extended > out to a "big tent" far better than our current gating system. > > Whether or not there is
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Mon, Sep 22, 2014 at 12:33 PM, Doug Hellmann wrote: > > On Sep 22, 2014, at 3:11 PM, Tim Bell wrote: > >> >> On 22 Sep 2014, at 20:53, Doug Hellmann wrote: >> >>> >>> On Sep 19, 2014, at 6:29 AM, Thierry Carrez wrote: >>> Monty Taylor wrote: > I've recently been thinking a lot about Sean's Layers stuff. So I wrote > a blog post which Jim Blair and Devananda were kind enough to help me > edit. > > http://inaugust.com/post/108 Hey Monty, As you can imagine, I read that post with great attention. I generally like the concept of a tightly integrated, limited-by-design layer #1 (I'd personally call it "Ring 0") and a large collection of "OpenStack" things gravitating around it. That would at least solve the attraction of the integrated release, suppress the need for incubation, foster >>> >>> I’m not sure I see this change reducing the number of incubated projects >>> unless we no longer incubate and graduate projects at all. Would everything >>> just live on stackforge and have a quality designation instead of an >>> “officialness” designation? Or would we have both? ATC status seems to >>> imply we need some sort of officialness designation, as you mention below. >>> >> >> The quality designation is really important for the operator community who >> are trying to work out what we can give to our end users. >> >> Offering early helps to establish the real-life experience and give good >> feedback on the designs. However, the operator then risks leaving their >> users orphaned if the project does not get a sustainable following or >> significant disruption if the APIs change. >> >> The packaging teams are key here as well. When do Ubuntu and Red Hat work >> out the chain of pre-reqs etc. to produce installable packages, >> packstack/juju tool support ? >> >> We do need to have some way to show that an layer #2 package is ready for >> prime time production and associated criteria (packages available, docs >> available, >1 company communities, models for HA and scale, …) > > > Right. I’m trying to understand if we are talking about doing that *instead* > of our existing incubation/graduation process, or in addition to that process > as a new thing. I like the idea of adding a quality designation. I’m not sure > replacing our existing process with that designation is a good idea. > A "production ready" tag wouldn't replace the incubation process because incubation isn't about that. Also, the incubation process would go away because the TC wouldn't be in the business of adding projects to the integrated gate any more. The content of integrated release would be based on a use-case-based scope definition, which is satisfied by the projects in the "Layer #1" list. I think it's interesting to think about other use-cases, and those could well inform the trademark discussion in the long term, but I think we need to start with one use case, and I would like to think this is one we can all agree on as central to the mission of OpenStack. That doesn't invalidate other use cases. And it doesn't make things not in Layer #1 suddenly become not-OpenStack. (A triple negative? Yep.) One of the primary effects of integration, as far as the release process is concerned, is being allowed to co-gate with other integrated projects, and having those projects accept your changes (integrate back with the other project). That shouldn't be a TC decision, and the decision to cogate with another project should be up to the projects themselves. If Nova feels that supporting the Ironic virt driver is important, Nova would agree to a two-project gate with Ironic. Other projects that Nova also gates with would not need to participate (eg, a change to Glance would also test Nova, but wouldn't necessarily run the nova-ironic test). I think this can be extended out to a "big tent" far better than our current gating system. Whether or not there is a process to apply for / get the "production ready" sticker would no longer be related to being part of the release process or gate tests. Of course, it's essential this includes feedback from users and operators. For what it's worth, I would want to see at least one production deployment of a project - and get feedback from the operators of said deployment - before I would feel comfortable calling it production-ready. -Devananda ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Sep 22, 2014, at 3:11 PM, Tim Bell wrote: > > On 22 Sep 2014, at 20:53, Doug Hellmann wrote: > >> >> On Sep 19, 2014, at 6:29 AM, Thierry Carrez wrote: >> >>> Monty Taylor wrote: I've recently been thinking a lot about Sean's Layers stuff. So I wrote a blog post which Jim Blair and Devananda were kind enough to help me edit. http://inaugust.com/post/108 >>> >>> Hey Monty, >>> >>> As you can imagine, I read that post with great attention. I generally >>> like the concept of a tightly integrated, limited-by-design layer #1 >>> (I'd personally call it "Ring 0") and a large collection of "OpenStack" >>> things gravitating around it. That would at least solve the attraction >>> of the integrated release, suppress the need for incubation, foster >> >> I’m not sure I see this change reducing the number of incubated projects >> unless we no longer incubate and graduate projects at all. Would everything >> just live on stackforge and have a quality designation instead of an >> “officialness” designation? Or would we have both? ATC status seems to imply >> we need some sort of officialness designation, as you mention below. >> > > The quality designation is really important for the operator community who > are trying to work out what we can give to our end users. > > Offering early helps to establish the real-life experience and give good > feedback on the designs. However, the operator then risks leaving their > users orphaned if the project does not get a sustainable following or > significant disruption if the APIs change. > > The packaging teams are key here as well. When do Ubuntu and Red Hat work out > the chain of pre-reqs etc. to produce installable packages, packstack/juju > tool support ? > > We do need to have some way to show that an layer #2 package is ready for > prime time production and associated criteria (packages available, docs > available, >1 company communities, models for HA and scale, …) Right. I’m trying to understand if we are talking about doing that *instead* of our existing incubation/graduation process, or in addition to that process as a new thing. I like the idea of adding a quality designation. I’m not sure replacing our existing process with that designation is a good idea. Doug > > Tim > > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 22 Sep 2014, at 20:53, Doug Hellmann wrote: > > On Sep 19, 2014, at 6:29 AM, Thierry Carrez wrote: > >> Monty Taylor wrote: >>> I've recently been thinking a lot about Sean's Layers stuff. So I wrote >>> a blog post which Jim Blair and Devananda were kind enough to help me edit. >>> >>> http://inaugust.com/post/108 >> >> Hey Monty, >> >> As you can imagine, I read that post with great attention. I generally >> like the concept of a tightly integrated, limited-by-design layer #1 >> (I'd personally call it "Ring 0") and a large collection of "OpenStack" >> things gravitating around it. That would at least solve the attraction >> of the integrated release, suppress the need for incubation, foster > > I’m not sure I see this change reducing the number of incubated projects > unless we no longer incubate and graduate projects at all. Would everything > just live on stackforge and have a quality designation instead of an > “officialness” designation? Or would we have both? ATC status seems to imply > we need some sort of officialness designation, as you mention below. > The quality designation is really important for the operator community who are trying to work out what we can give to our end users. Offering early helps to establish the real-life experience and give good feedback on the designs. However, the operator then risks leaving their users orphaned if the project does not get a sustainable following or significant disruption if the APIs change. The packaging teams are key here as well. When do Ubuntu and Red Hat work out the chain of pre-reqs etc. to produce installable packages, packstack/juju tool support ? We do need to have some way to show that an layer #2 package is ready for prime time production and associated criteria (packages available, docs available, >1 company communities, models for HA and scale, …) Tim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Sep 18, 2014, at 2:53 PM, Monty Taylor wrote: > Hey all, > > I've recently been thinking a lot about Sean's Layers stuff. So I wrote > a blog post which Jim Blair and Devananda were kind enough to help me edit. > > http://inaugust.com/post/108 > > Enjoy. I’ve read through this a few times now, and I think I can support most of it. It doesn’t address some of the issues we have, but most of the concrete proposals seem like they take us to incrementally better places than where we are now. I definitely like the idea of making the integrated gate different for each project, based on the other projects it actually integrates with. I could see extending the two-project gate idea for projects outside of layer 1 to include more than 2 projects eventually. The assumption that all layer 1 projects can depend on the other members of layer 1 being present may have ramifications for the trademark “core” definition. I don’t think those cause problems, based on the mechanisms for defining capabilities and designated sections being worked out now, but it’s worth pointing out as something we’ll need to keep in mind. The self-organizing groupings that Vish, John, and others have mentioned may well lead us to create additional trademarks in a more naturally evolving way than the single big mark we’re trying to squeeze everything into now. Does the unified client SDK fit into layer 1 as one of the “common libraries … necessary for these”? Or do we anticipate the services still using their own individual libraries for talking to each other? Having a quality designation will help distros and deployers. I’m not sure we want Cern specifically to be our arbiter of that quality, but I do like the idea of having users be involved in the determination. Maybe it's something the User Committee could help with in a more general way. This proposal only addresses some of the challenges we have right now. If we maintain a big tent approach, and I think we should, we still need a way to implement cross-project policies and initiatives outside of the scope of any one of our existing programs. I agree with Vish that we need a different name for Layer 1. A name that doesn’t imply “leveling” or “layering” would be good, since some of the cloud-native services don’t build on those layer 1 services. Doug ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Sep 19, 2014, at 10:37 PM, Monty Taylor wrote: > On 09/19/2014 03:29 AM, Thierry Carrez wrote: >> Monty Taylor wrote: >>> I've recently been thinking a lot about Sean's Layers stuff. So I wrote >>> a blog post which Jim Blair and Devananda were kind enough to help me edit. >>> >>> http://inaugust.com/post/108 >> >> Hey Monty, >> >> As you can imagine, I read that post with great attention. I generally >> like the concept of a tightly integrated, limited-by-design layer #1 >> (I'd personally call it "Ring 0") and a large collection of "OpenStack" >> things gravitating around it. That would at least solve the attraction >> of the integrated release, suppress the need for incubation, foster >> competition/innovation within our community, and generally address the >> problem of community scaling. There are a few details on the >> consequences though, and in those as always the devil lurks. >> >> ## The Technical Committee >> >> The Technical Committee is defined in the OpenStack bylaws, and is the >> representation of the contributors to "the project". Teams work on code >> repositories, and at some point ask their work to be recognized as part >> of "OpenStack". In doing so, they place their work under the oversight >> of the Technical Committee. In return, team members get to participate >> in electing the technical committee members (they become "ATC"). It's a >> balanced system, where both parties need to agree: the TC can't force >> itself as overseer of a random project, and a random project can't just >> decide by itself it is "OpenStack". >> >> I don't see your proposal breaking that balanced system, but it changes >> its dynamics a bit. The big tent would contain a lot more members. And >> while the TC would arguably bring a significant share of its attention >> to Ring 0, its voters constituency would mostly consist of developers >> who do not participate in Ring 0 development. I don't really see it as >> changing dramatically the membership of the TC, but it's a consequence >> worth mentioning. > > Agree. I'm willing to bet it'll be better not worse to have a large > constituency - but it's also problem that it's a giant disaster. I'm > still on board with going for it. > >> ## Programs >> >> Programs were created relatively recently as a way to describe which >> teams are "in OpenStack" vs. which ones aren't. They directly tie into >> the ATC system: if you contribute to code repositories under a blessed >> program, then you're an ATC, you vote in TC elections and the TC has >> some oversight over your code repositories. Previously, this was granted >> at a code repository level, but that failed to give flexibility for >> teams to organize their code in the most convenient manner for them. So >> we started to bless teams rather than specific code repositories. >> >> Now, that didn't work out so well. Some programs were a 'theme', like >> Infrastructure, or Docs. For those, competing efforts do not really make >> sense: there can only be one, and competition should happen inside those >> efforts rather than outside. Some programs were a 'team', like >> Orchestration/Heat or Deployment/TripleO. And that's where the model >> started to break: some new orchestration things need space, but the >> current Heat team is not really interested in maintaining them. What's >> the point of being under the same program then ? And TripleO is not the >> only way to deploy OpenStack, but its mere existence (and name) >> prevented other flowers to bloom in our community. >> >> You don't talk much about programs in your proposal. In particular, you >> only mention layer 1, "Cloud Native" applications, "User Interface" >> applications, and "Operator" applications. So I'm unsure of where, if >> anywhere, would "Infrastructure" or "Docs" repositories live. >> >> Here is how I see it could work. We could keep 'theme' programs (think >> Infra, Release cycle management, Docs, QA) with their current structure >> (collection of code repositories under a single team/PTL). We would get >> rid of 'team' programs completely, and just have a registry of >> "OpenStack" code repositories (openstack*/*, big tent). Each of those >> could have a specific PTL, or explicitely inherit its PTL from another >> code repository. Under that PTL, they could have separate or same core >> review team, whatever maps reality and how they want to work (so we >> could say that openstack/python-novaclient inherits its PTL from >> openstack/nova and doesn't need a specific one). That would allow us to >> map anything that may come in. Oslo falls a bit in between, could be >> considered a 'theme' program or several repos sharing PTL. > > I think we can do what you're saying and generalize a little bit. What > if we declared programs, as needed, when we think there is a need to > "pick a winner". (I think we can all agree that early winner picking is > an unintended but very real side effect of the current system) > > And when I say
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Sep 19, 2014, at 6:29 AM, Thierry Carrez wrote: > Monty Taylor wrote: >> I've recently been thinking a lot about Sean's Layers stuff. So I wrote >> a blog post which Jim Blair and Devananda were kind enough to help me edit. >> >> http://inaugust.com/post/108 > > Hey Monty, > > As you can imagine, I read that post with great attention. I generally > like the concept of a tightly integrated, limited-by-design layer #1 > (I'd personally call it "Ring 0") and a large collection of "OpenStack" > things gravitating around it. That would at least solve the attraction > of the integrated release, suppress the need for incubation, foster I’m not sure I see this change reducing the number of incubated projects unless we no longer incubate and graduate projects at all. Would everything just live on stackforge and have a quality designation instead of an “officialness” designation? Or would we have both? ATC status seems to imply we need some sort of officialness designation, as you mention below. > competition/innovation within our community, and generally address the > problem of community scaling. There are a few details on the > consequences though, and in those as always the devil lurks. > > ## The Technical Committee > > The Technical Committee is defined in the OpenStack bylaws, and is the > representation of the contributors to "the project". Teams work on code > repositories, and at some point ask their work to be recognized as part > of "OpenStack". In doing so, they place their work under the oversight > of the Technical Committee. In return, team members get to participate > in electing the technical committee members (they become "ATC"). It's a > balanced system, where both parties need to agree: the TC can't force > itself as overseer of a random project, and a random project can't just > decide by itself it is "OpenStack". > > I don't see your proposal breaking that balanced system, but it changes > its dynamics a bit. The big tent would contain a lot more members. And > while the TC would arguably bring a significant share of its attention > to Ring 0, its voters constituency would mostly consist of developers > who do not participate in Ring 0 development. I don't really see it as > changing dramatically the membership of the TC, but it's a consequence > worth mentioning. > > ## Programs > > Programs were created relatively recently as a way to describe which > teams are "in OpenStack" vs. which ones aren't. They directly tie into > the ATC system: if you contribute to code repositories under a blessed > program, then you're an ATC, you vote in TC elections and the TC has > some oversight over your code repositories. Previously, this was granted > at a code repository level, but that failed to give flexibility for > teams to organize their code in the most convenient manner for them. So > we started to bless teams rather than specific code repositories. > > Now, that didn't work out so well. Some programs were a 'theme', like > Infrastructure, or Docs. For those, competing efforts do not really make > sense: there can only be one, and competition should happen inside those > efforts rather than outside. Some programs were a 'team', like > Orchestration/Heat or Deployment/TripleO. And that's where the model > started to break: some new orchestration things need space, but the > current Heat team is not really interested in maintaining them. What's > the point of being under the same program then ? And TripleO is not the > only way to deploy OpenStack, but its mere existence (and name) > prevented other flowers to bloom in our community. > > You don't talk much about programs in your proposal. In particular, you > only mention layer 1, "Cloud Native" applications, "User Interface" > applications, and "Operator" applications. So I'm unsure of where, if > anywhere, would "Infrastructure" or "Docs" repositories live. > > Here is how I see it could work. We could keep 'theme' programs (think > Infra, Release cycle management, Docs, QA) with their current structure > (collection of code repositories under a single team/PTL). We would get > rid of 'team' programs completely, and just have a registry of > "OpenStack" code repositories (openstack*/*, big tent). Each of those > could have a specific PTL, or explicitely inherit its PTL from another > code repository. Under that PTL, they could have separate or same core > review team, whatever maps reality and how they want to work (so we > could say that openstack/python-novaclient inherits its PTL from > openstack/nova and doesn't need a specific one). That would allow us to > map anything that may come in. Oslo falls a bit in between, could be > considered a 'theme' program or several repos sharing PTL. I like the idea of chartered programs as a way to tackling our cross-project issues. We need fewer of programs, but for the ones we do need we still need to integrate them with the rest of our governance. > > ## The release and the d
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
FWIW, I think this is a great approach to evolving our thinking of the projects and ecosystem around OpenStack. I’m far too removed these days from the details of the day-to-day running of the programs and projects to comment on details. But, I’ve long felt a need to go beyond the simple core + everyone else thinking. This is moving in the right direction IMHO. Troy > On Sep 22, 2014, at 5:49 AM, Mark McLoughlin wrote: > > Hey > > On Thu, 2014-09-18 at 11:53 -0700, Monty Taylor wrote: >> Hey all, >> >> I've recently been thinking a lot about Sean's Layers stuff. So I wrote >> a blog post which Jim Blair and Devananda were kind enough to help me edit. >> >> http://inaugust.com/post/108 > > Lots of great stuff here, but too much to respond to everything in > detail. > > I love the way you've framed this in terms of the needs of developers, > distributors, deployers and end users. I'd like to make sure we're > focused on tackling those places where we're failing these groups, so: > > > - Developers > > I think we're catering pretty well to developers with the "big tent" > concept of Programs. There's been some good discussion about how > Programs could be better at embracing projects in their related area, > and that would be great to pursue. But the general concept - of > endorsing and empowering teams of people collaborating in the > OpenStack way - has a lot of legs, I think. > > I also think our release cadence does a pretty good job of serving > developers. We've talked many times about the benefit of it, and I'd > like to see it applied to more than just the "server projects". > > OTOH, the integrated gate is straining, and a source of frustration > for everyone. You raise the question of whether everything currently > in the integrated release needs to be co-gated, and I totally agree > that needs re-visiting. > > > - Distributors > > We may be doing a better job of catering to distributors than any > other group. For example, things like the release cadence, stable > branch and common set of dependencies works pretty well. > > . The concept of an integrated release (with an incubation process) is > great, because it nicely defines a set of stuff that distributors > should ship. Certainly, life would be more difficult for distributors > if there was a smaller set of projects in the release and a whole > bunch of other projects which are interesting to distro users, but > with an ambiguous level of commitment from our community. Right now, > our integration process has a huge amount of influence over what > gets shipped by distros, and that in turn serves distro users by > ensuring a greater level of commonality between distros. > > > - Operators > > I think the feedback we've been getting over the past few cycles > suggests we are failing this group the most. > > Operators want to offer a compelling set of services to their users, > but they want those services to be stable, performant, and perhaps > most importantly, cost-effective. No operator wants to have to > invest a huge amount of time in getting a new service running. > > You suggest a "Production Ready" tag. Absolutely - our graduation of > projects has been interpreted as meaning "production ready", when > it's actually more useful as a signal to distros rather than > operators. Graduation does not necessarily imply that a service is > ready for "production", no matter how you define "production". > > I'd like to think we could give more nuanced advice to operators than > a simple tag, but perhaps the information gathering process that > projects would need to go through to be awarded that tag would > uncover the more detailed information for operators. > > You could question whether the TC is the right body for this > process. How might it work if the User Committee owned this? > > There are many other ways we can and should help operators, > obviously, but this "setting expectations" is the aspect most > relevant to this discussion. > > > - End users > > You're right that we don't pay sufficient attention to this group. > For me, the highest priority challenge here is interoperability. > Particularly interoperability between public clouds. > > The only real interop effort to date you can point to is the > board-owned DefCore and RefStack efforts. The idea being that a > trademark program with API testing requirements will focus minds on > interoperability. I'd love us (as a community) to be making more > rapid progress on interoperability, but at least there are no > encouraging signs that we should make some definite progress soon. > > Your end-user focused concrete suggestions (#7-#10) are interesting, > and I find myself thinking about how much of a positive effect on > interop each of them would have. For example, making our tools > multi-cloud aware would help e
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Great conversations here. I'd like to echo Dean Troyer's comment on Suggestion 9, for multi-cloud span node pooling ,we need standards. It'll make life easier when user tools could be configured against a limit as well as standard set of rules, instead of numerous different rules by vendors. It is gonna be a difficult task, since this kind of thing is always hard to do, but some form of intra cloud standard has to be introduced and worked on so users could actually have a resource "pool" Best Regards Zhipeng -- Zhipeng Huang Research Assistant Mobile Ad-Hoc Network Lab, Calit2 University of California, Irvine Email: zhipe...@uci.edu Office: Calit2 Building Room 2402 OpenStack, OpenDaylight, OpenCompute affcienado ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Hey On Thu, 2014-09-18 at 11:53 -0700, Monty Taylor wrote: > Hey all, > > I've recently been thinking a lot about Sean's Layers stuff. So I wrote > a blog post which Jim Blair and Devananda were kind enough to help me edit. > > http://inaugust.com/post/108 Lots of great stuff here, but too much to respond to everything in detail. I love the way you've framed this in terms of the needs of developers, distributors, deployers and end users. I'd like to make sure we're focused on tackling those places where we're failing these groups, so: - Developers I think we're catering pretty well to developers with the "big tent" concept of Programs. There's been some good discussion about how Programs could be better at embracing projects in their related area, and that would be great to pursue. But the general concept - of endorsing and empowering teams of people collaborating in the OpenStack way - has a lot of legs, I think. I also think our release cadence does a pretty good job of serving developers. We've talked many times about the benefit of it, and I'd like to see it applied to more than just the "server projects". OTOH, the integrated gate is straining, and a source of frustration for everyone. You raise the question of whether everything currently in the integrated release needs to be co-gated, and I totally agree that needs re-visiting. - Distributors We may be doing a better job of catering to distributors than any other group. For example, things like the release cadence, stable branch and common set of dependencies works pretty well. . The concept of an integrated release (with an incubation process) is great, because it nicely defines a set of stuff that distributors should ship. Certainly, life would be more difficult for distributors if there was a smaller set of projects in the release and a whole bunch of other projects which are interesting to distro users, but with an ambiguous level of commitment from our community. Right now, our integration process has a huge amount of influence over what gets shipped by distros, and that in turn serves distro users by ensuring a greater level of commonality between distros. - Operators I think the feedback we've been getting over the past few cycles suggests we are failing this group the most. Operators want to offer a compelling set of services to their users, but they want those services to be stable, performant, and perhaps most importantly, cost-effective. No operator wants to have to invest a huge amount of time in getting a new service running. You suggest a "Production Ready" tag. Absolutely - our graduation of projects has been interpreted as meaning "production ready", when it's actually more useful as a signal to distros rather than operators. Graduation does not necessarily imply that a service is ready for "production", no matter how you define "production". I'd like to think we could give more nuanced advice to operators than a simple tag, but perhaps the information gathering process that projects would need to go through to be awarded that tag would uncover the more detailed information for operators. You could question whether the TC is the right body for this process. How might it work if the User Committee owned this? There are many other ways we can and should help operators, obviously, but this "setting expectations" is the aspect most relevant to this discussion. - End users You're right that we don't pay sufficient attention to this group. For me, the highest priority challenge here is interoperability. Particularly interoperability between public clouds. The only real interop effort to date you can point to is the board-owned DefCore and RefStack efforts. The idea being that a trademark program with API testing requirements will focus minds on interoperability. I'd love us (as a community) to be making more rapid progress on interoperability, but at least there are no encouraging signs that we should make some definite progress soon. Your end-user focused concrete suggestions (#7-#10) are interesting, and I find myself thinking about how much of a positive effect on interop each of them would have. For example, making our tools multi-cloud aware would help encourage people to demand interop from their providers. I also agree that end-user tools should support older versions of our APIs, but don't think that necessarily implies rolling releases. So, if I was to pick the areas which I think would address our most pressing challenges: 1) Shrinking the integrated gate, and allowing per-project testing strategies other than shoving every integrated project into the gate. 2) Giving more direction to operators about the readiness of our projects for different use cases. A process a
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
John Dickinson wrote: > I propose that we can get the benefits of Monty's proposal and implement all > of his concrete suggestions (which are fantastic) by slightly adjusting our > usage of the program/project concepts. > > I had originally hoped that the "program" concept would have been a little > higher-level instead of effectively spelling "project" as "program". I'd love > to see a hierarchy of openstack->program->project/team->repos. Right now, we > have added the "program" layer but have effectively mapped it 1:1 to the > project. For example, we used to have a few repos in the Swift project > managed by the same group of people, and now we have a few repos in the > "object storage" program, all managed by the same group of people. And every > time something is added to OpenStack, its added as a new program, effectively > putting us exactly where we were before we called it a program with the same > governance and management scaling problems. > > Today, I'd group existing OpenStack projects into programs as follows: > > Compute: nova, sahara, ironic > Storage: swift, cinder, glance, trove > Network: neutron, designate, zaqar > Deployment/management: heat, triple-o, horizon, ceilometer > Identity: keystone, barbican > Support (not user facing): infra, docs, tempest, devstack, oslo > (potentially even) stackforge: lots I'm not clear on what those $THINGS would actually represent. Would they have a single PTL ? (I guess not, otherwise we are back at the team/theme duality I called out in my answer to Monty). Who decides which $THINGS may exist ? (I suppose anybody can declare one, otherwise we are back with TC blessing fields). Can $THINGS contain alternative competing solutions ? (I suppose yes, otherwise we are back to preventing innovation). So what are they ? A tag you can apply to your project ? A category you can file yourself under ? I like Monty's "end-user" approach to defining layer #1. The use case being, you want a functioning compute instance. Your taxonomy seems more artificial. Swift, Cinder, Glance and Trove (and Manila) all represent very different end-user use cases. Yes they all "store" stuff, but that's about the only thing they have in common. And no user in their sane mind would use all of them at the same time (if they do, they are probably doing it wrong). I guess we could recognize more "basic use cases". I.e. Object storage is a end-user use case all by itself, and it only requires Swift + Keystone. So we could have a Layer #1bis that is Swift + Keystone. But then we are back to blessing stuff as essential/official, and next thing we know, Sahara will knock on the TC door to get Map/Reduce recognized as essential layer-1-like end-user use case. I can see how a "give-me-a-damn-instance" layer-1 definition is restrictive, but that's the beauty and the predictability of Monty's proposal. It limits integration where it really matters, unleashes competition where it will help, and removes almost all of the badge-granting role of the TC. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 09/19/2014 10:50 AM, Vishvananda Ishaya wrote: > > On Sep 19, 2014, at 10:14 AM, John Dickinson wrote: > >> >> On Sep 19, 2014, at 5:46 AM, John Griffith >> wrote: >> >>> >>> >>> On Fri, Sep 19, 2014 at 4:33 AM, Thierry Carrez >>> wrote: >>> Vishvananda Ishaya wrote: Great writeup. I think there are some great concrete suggestions here. A couple more: 1. I think we need a better name for Layer #1 that actually represents what the goal of it is: Infrastructure Services? 2. We need to be be open to having other Layer #1s within the community. We should allow for similar collaborations and group focus to grow up as well. Storage Services? Platform Services? Computation Services? >>> >>> I think that would nullify most of the benefits of Monty's proposal. If >>> we keep on blessing "themes" or special groups, we'll soon be back at >>> step 0, with projects banging on the TC door to become special, and >>> companies not allocating resources to anything that's not special. >>> >>> -- >>> Thierry Carrez (ttx) >>> >>> ___ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> Great stuff, mixed on point 2 raised by Vish but honestly I think that's >>> something that could evolve over time, but I looked at that differently as >>> in Cinder, SWIFT and some day Manilla live under a Storage Services >>> umbrella, and ideally at some point there's some convergence there. >>> >>> Anyway, I don't want to start a rat-hole on that, it's kind of irrelevant >>> right now. Bottom line is I think the direction and initial ideas in >>> Monty's post are what a lot of us have been thinking about and looking for. >>> I'm in!! >> >> >> I too am generally supportive of the concept, but I do want to think about >> the vishy/tts/jgriffith points above. >> >> Today, I'd group existing OpenStack projects into programs as follows: >> >> Compute: nova, sahara, ironic >> Storage: swift, cinder, glance, trove >> Network: neutron, designate, zaqar >> Deployment/management: heat, triple-o, horizon, ceilometer >> Identity: keystone, barbican >> Support (not user facing): infra, docs, tempest, devstack, oslo >> (potentially even) stackforge: lots > > There is a pretty different division of things in this breakdown than in what > monty was proposing. This divides things up by conceptual similarity which I > think is actually less useful than breaking things down by use case. I really > like the grouping and testing of things which are tightly coupled. > > If we say launching a VM and using it is the primary use case of our > community corrently then things group into monty’s layer #1. It seems fairly > clear that a large section of our community is focused on this use case so > this should be a primary focus of infrastructure resources. > > There are other use cases in our community, for example: > > Object Storage: Swift (depends on keystone) > Orchestrating Multiple VMs: Heat (depends on layer1) > DBaSS: Trove: (depends on heat) > > These are also important use cases for parts of our community, but swift has > demostrated that it isn’t required to be a part of an integrated release > schedule, so these could be managed by smaller groups in the community. Note > that these are primarily individual projects today, but I could see a future > where some of these projects decide to group and do an integrated release. In > the future we might see (totally making this up): > > Public Cloud Application services: Trove, Zaqar > Application Deployment services: Heat, Murano > Operations services: Ceilometer, Congress > > As I mentioned previously though, I don’t think we need to define these > groups in advance. These groups can organize as needed. I'm kinda interested to see what self-organization happens... ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 09/19/2014 10:14 AM, John Dickinson wrote: > > On Sep 19, 2014, at 5:46 AM, John Griffith > wrote: > >> >> >> On Fri, Sep 19, 2014 at 4:33 AM, Thierry Carrez >> wrote: Vishvananda Ishaya wrote: >>> Great writeup. I think there are some great concrete suggestions >>> here. >>> >>> A couple more: >>> >>> 1. I think we need a better name for Layer #1 that actually >>> represents what the goal of it is: Infrastructure Services? 2. We >>> need to be be open to having other Layer #1s within the >>> community. We should allow for similar collaborations and group >>> focus to grow up as well. Storage Services? Platform Services? >>> Computation Services? >> >> I think that would nullify most of the benefits of Monty's >> proposal. If we keep on blessing "themes" or special groups, we'll >> soon be back at step 0, with projects banging on the TC door to >> become special, and companies not allocating resources to anything >> that's not special. >> >> -- Thierry Carrez (ttx) >> >> ___ OpenStack-dev >> mailing list OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> Great stuff, mixed on point 2 raised by Vish but honestly I think >> that's something that could evolve over time, but I looked at that >> differently as in Cinder, SWIFT and some day Manilla live under a >> Storage Services umbrella, and ideally at some point there's some >> convergence there. >> >> Anyway, I don't want to start a rat-hole on that, it's kind of >> irrelevant right now. Bottom line is I think the direction and >> initial ideas in Monty's post are what a lot of us have been >> thinking about and looking for. I'm in!! > > > I too am generally supportive of the concept, but I do want to think > about the vishy/tts/jgriffith points above. > > It's interesting that the proposed "layer #1" stuff is very very > similar to what was originally in OpenStack at the very beginning as > Nova. Over time, many of these pieces of functionality required for > compute were split out (block, networking, image, etc), and I think > that's why so many people look at these pieces and say (rightly), "of > course these are required all together and tightly coupled". That's > how these projects started, and we still see evidence of their birth > today. > > For that reason, I do agree with Vish that there should be similar > collaborations for other things. While the "layer #1" (or "compute") > use case is very common, we can all see that it's not the only one > that people are solving with OpenStack parts. And this is reflected > in the products build and sold by companies, too. Some sell one > subset of openstack stuff as product X and maybe a different subset > as product Y. (The most common example here is "compute" vs "object > storage".) This reality has led to a lot of the angst around > definitions since there is effort to define openstack all as one > thing (or worse, as a "base" thing that others are defined as built > upon). > > I propose that we can get the benefits of Monty's proposal and > implement all of his concrete suggestions (which are fantastic) by > slightly adjusting our usage of the program/project concepts. > > I had originally hoped that the "program" concept would have been a > little higher-level instead of effectively spelling "project" as > "program". I'd love to see a hierarchy of > openstack->program->project/team->repos. Right now, we have added the > "program" layer but have effectively mapped it 1:1 to the project. > For example, we used to have a few repos in the Swift project managed > by the same group of people, and now we have a few repos in the > "object storage" program, all managed by the same group of people. > And every time something is added to OpenStack, its added as a new > program, effectively putting us exactly where we were before we > called it a program with the same governance and management scaling > problems. > > Today, I'd group existing OpenStack projects into programs as > follows: > > Compute: nova, sahara, ironic > Storage: swift, cinder, glance, trove > Network: neutron, designate, zaqar > Deployment/management: heat, triple-o, horizon, ceilometer > Identity: keystone, barbican Support > (not user facing): infra, docs, tempest, devstack, oslo (potentially > even) stackforge: lots > > I like two things about this. First, it allows people to compose a > solution. Second, it allows us as a community to thing more about the > strategic/product things. For example, it lets us as a community say, > "We think storage is important. How are we solving it today? What > gaps do we have in that? How can the various storage things we have > work together better?" > > Thierry makes the point that more "themes" will nullify the benefits > of Monty's proposal. I agree, if we continue to allow the explosion > of projects/programs/themes to continue. The benefit of what Monty is > proposing
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 09/19/2014 03:29 AM, Thierry Carrez wrote: > Monty Taylor wrote: >> I've recently been thinking a lot about Sean's Layers stuff. So I wrote >> a blog post which Jim Blair and Devananda were kind enough to help me edit. >> >> http://inaugust.com/post/108 > > Hey Monty, > > As you can imagine, I read that post with great attention. I generally > like the concept of a tightly integrated, limited-by-design layer #1 > (I'd personally call it "Ring 0") and a large collection of "OpenStack" > things gravitating around it. That would at least solve the attraction > of the integrated release, suppress the need for incubation, foster > competition/innovation within our community, and generally address the > problem of community scaling. There are a few details on the > consequences though, and in those as always the devil lurks. > > ## The Technical Committee > > The Technical Committee is defined in the OpenStack bylaws, and is the > representation of the contributors to "the project". Teams work on code > repositories, and at some point ask their work to be recognized as part > of "OpenStack". In doing so, they place their work under the oversight > of the Technical Committee. In return, team members get to participate > in electing the technical committee members (they become "ATC"). It's a > balanced system, where both parties need to agree: the TC can't force > itself as overseer of a random project, and a random project can't just > decide by itself it is "OpenStack". > > I don't see your proposal breaking that balanced system, but it changes > its dynamics a bit. The big tent would contain a lot more members. And > while the TC would arguably bring a significant share of its attention > to Ring 0, its voters constituency would mostly consist of developers > who do not participate in Ring 0 development. I don't really see it as > changing dramatically the membership of the TC, but it's a consequence > worth mentioning. Agree. I'm willing to bet it'll be better not worse to have a large constituency - but it's also problem that it's a giant disaster. I'm still on board with going for it. > ## Programs > > Programs were created relatively recently as a way to describe which > teams are "in OpenStack" vs. which ones aren't. They directly tie into > the ATC system: if you contribute to code repositories under a blessed > program, then you're an ATC, you vote in TC elections and the TC has > some oversight over your code repositories. Previously, this was granted > at a code repository level, but that failed to give flexibility for > teams to organize their code in the most convenient manner for them. So > we started to bless teams rather than specific code repositories. > > Now, that didn't work out so well. Some programs were a 'theme', like > Infrastructure, or Docs. For those, competing efforts do not really make > sense: there can only be one, and competition should happen inside those > efforts rather than outside. Some programs were a 'team', like > Orchestration/Heat or Deployment/TripleO. And that's where the model > started to break: some new orchestration things need space, but the > current Heat team is not really interested in maintaining them. What's > the point of being under the same program then ? And TripleO is not the > only way to deploy OpenStack, but its mere existence (and name) > prevented other flowers to bloom in our community. > > You don't talk much about programs in your proposal. In particular, you > only mention layer 1, "Cloud Native" applications, "User Interface" > applications, and "Operator" applications. So I'm unsure of where, if > anywhere, would "Infrastructure" or "Docs" repositories live. > > Here is how I see it could work. We could keep 'theme' programs (think > Infra, Release cycle management, Docs, QA) with their current structure > (collection of code repositories under a single team/PTL). We would get > rid of 'team' programs completely, and just have a registry of > "OpenStack" code repositories (openstack*/*, big tent). Each of those > could have a specific PTL, or explicitely inherit its PTL from another > code repository. Under that PTL, they could have separate or same core > review team, whatever maps reality and how they want to work (so we > could say that openstack/python-novaclient inherits its PTL from > openstack/nova and doesn't need a specific one). That would allow us to > map anything that may come in. Oslo falls a bit in between, could be > considered a 'theme' program or several repos sharing PTL. I think we can do what you're saying and generalize a little bit. What if we declared programs, as needed, when we think there is a need to "pick a winner". (I think we can all agree that early winner picking is an unintended but very real side effect of the current system) And when I say "need to" - I mean it in the same sense as "Production Ready" The themes you listed are excellent ones - it makes no sense to have two Infras, two QAs or
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 09/19/2014 01:15 PM, Vishvananda Ishaya wrote: > > On Sep 19, 2014, at 3:33 AM, Thierry Carrez > wrote: > >> Vishvananda Ishaya wrote: >>> Great writeup. I think there are some great concrete >>> suggestions here. >>> >>> A couple more: >>> >>> 1. I think we need a better name for Layer #1 that actually >>> represents what the goal of it is: Infrastructure Services? 2. >>> We need to be be open to having other Layer #1s within the >>> community. We should allow for similar collaborations and group >>> focus to grow up as well. Storage Services? Platform Services? >>> Computation Services? >> >> I think that would nullify most of the benefits of Monty's >> proposal. If we keep on blessing "themes" or special groups, >> we'll soon be back at step 0, with projects banging on the TC >> door to become special, and companies not allocating resources to >> anything that's not special. > > I was assuming that these would be self-organized rather than > managed by the TC. > > Vish > Some groups are able to self-organize better than others, I have been hoping the third party theme would self-organize for the last 10 months and while some folks are trying, their numbers don't keep up with the demand. This group insists that someone else tell them what to do. It is not a scalable model, it also doesn't inspire anyone outside of the theme to want to help out with the answering of questions either. Just as a data point. Thanks, Anita. > > > ___ OpenStack-dev > mailing list OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Sep 19, 2014, at 10:14 AM, John Dickinson wrote: > > On Sep 19, 2014, at 5:46 AM, John Griffith > wrote: > >> >> >> On Fri, Sep 19, 2014 at 4:33 AM, Thierry Carrez >> wrote: >> Vishvananda Ishaya wrote: >>> Great writeup. I think there are some great concrete suggestions here. >>> >>> A couple more: >>> >>> 1. I think we need a better name for Layer #1 that actually represents what >>> the goal of it is: Infrastructure Services? >>> 2. We need to be be open to having other Layer #1s within the community. We >>> should allow for similar collaborations and group focus to grow up as well. >>> Storage Services? Platform Services? Computation Services? >> >> I think that would nullify most of the benefits of Monty's proposal. If >> we keep on blessing "themes" or special groups, we'll soon be back at >> step 0, with projects banging on the TC door to become special, and >> companies not allocating resources to anything that's not special. >> >> -- >> Thierry Carrez (ttx) >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> Great stuff, mixed on point 2 raised by Vish but honestly I think that's >> something that could evolve over time, but I looked at that differently as >> in Cinder, SWIFT and some day Manilla live under a Storage Services >> umbrella, and ideally at some point there's some convergence there. >> >> Anyway, I don't want to start a rat-hole on that, it's kind of irrelevant >> right now. Bottom line is I think the direction and initial ideas in >> Monty's post are what a lot of us have been thinking about and looking for. >> I'm in!! > > > I too am generally supportive of the concept, but I do want to think about > the vishy/tts/jgriffith points above. > > Today, I'd group existing OpenStack projects into programs as follows: > > Compute: nova, sahara, ironic > Storage: swift, cinder, glance, trove > Network: neutron, designate, zaqar > Deployment/management: heat, triple-o, horizon, ceilometer > Identity: keystone, barbican > Support (not user facing): infra, docs, tempest, devstack, oslo > (potentially even) stackforge: lots There is a pretty different division of things in this breakdown than in what monty was proposing. This divides things up by conceptual similarity which I think is actually less useful than breaking things down by use case. I really like the grouping and testing of things which are tightly coupled. If we say launching a VM and using it is the primary use case of our community corrently then things group into monty’s layer #1. It seems fairly clear that a large section of our community is focused on this use case so this should be a primary focus of infrastructure resources. There are other use cases in our community, for example: Object Storage: Swift (depends on keystone) Orchestrating Multiple VMs: Heat (depends on layer1) DBaSS: Trove: (depends on heat) These are also important use cases for parts of our community, but swift has demostrated that it isn’t required to be a part of an integrated release schedule, so these could be managed by smaller groups in the community. Note that these are primarily individual projects today, but I could see a future where some of these projects decide to group and do an integrated release. In the future we might see (totally making this up): Public Cloud Application services: Trove, Zaqar Application Deployment services: Heat, Murano Operations services: Ceilometer, Congress As I mentioned previously though, I don’t think we need to define these groups in advance. These groups can organize as needed. Vish signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Fri, Sep 19, 2014 at 12:02 PM, Adam Lawson wrote: > Can someone do a small little Visio or other visual to explain what's > being > Sean's blog post included a nice diagram that is Monty's starting point: https://dague.net/2014/08/26/openstack-as-layers/ AIUI Monty's Layer 1 is basically the original layers 1+2. I had separated them originally as layer 2 is optional from a purely technical perspective, but not really from a cloud user perspective. layers tl;dr: For my purposes layers 1, 2 and a hand-wavey 'everything else' is useful. Others find combing layers 1 and 2 more useful, especially for organizational and other purposes. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Sep 19, 2014, at 3:33 AM, Thierry Carrez wrote: > Vishvananda Ishaya wrote: >> Great writeup. I think there are some great concrete suggestions here. >> >> A couple more: >> >> 1. I think we need a better name for Layer #1 that actually represents what >> the goal of it is: Infrastructure Services? >> 2. We need to be be open to having other Layer #1s within the community. We >> should allow for similar collaborations and group focus to grow up as well. >> Storage Services? Platform Services? Computation Services? > > I think that would nullify most of the benefits of Monty's proposal. If > we keep on blessing "themes" or special groups, we'll soon be back at > step 0, with projects banging on the TC door to become special, and > companies not allocating resources to anything that's not special. I was assuming that these would be self-organized rather than managed by the TC. Vish signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Sep 19, 2014, at 5:46 AM, John Griffith wrote: > > > On Fri, Sep 19, 2014 at 4:33 AM, Thierry Carrez wrote: > Vishvananda Ishaya wrote: > > Great writeup. I think there are some great concrete suggestions here. > > > > A couple more: > > > > 1. I think we need a better name for Layer #1 that actually represents what > > the goal of it is: Infrastructure Services? > > 2. We need to be be open to having other Layer #1s within the community. We > > should allow for similar collaborations and group focus to grow up as well. > > Storage Services? Platform Services? Computation Services? > > I think that would nullify most of the benefits of Monty's proposal. If > we keep on blessing "themes" or special groups, we'll soon be back at > step 0, with projects banging on the TC door to become special, and > companies not allocating resources to anything that's not special. > > -- > Thierry Carrez (ttx) > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > Great stuff, mixed on point 2 raised by Vish but honestly I think that's > something that could evolve over time, but I looked at that differently as in > Cinder, SWIFT and some day Manilla live under a Storage Services umbrella, > and ideally at some point there's some convergence there. > > Anyway, I don't want to start a rat-hole on that, it's kind of irrelevant > right now. Bottom line is I think the direction and initial ideas in Monty's > post are what a lot of us have been thinking about and looking for. I'm in!! I too am generally supportive of the concept, but I do want to think about the vishy/tts/jgriffith points above. It's interesting that the proposed "layer #1" stuff is very very similar to what was originally in OpenStack at the very beginning as Nova. Over time, many of these pieces of functionality required for compute were split out (block, networking, image, etc), and I think that's why so many people look at these pieces and say (rightly), "of course these are required all together and tightly coupled". That's how these projects started, and we still see evidence of their birth today. For that reason, I do agree with Vish that there should be similar collaborations for other things. While the "layer #1" (or "compute") use case is very common, we can all see that it's not the only one that people are solving with OpenStack parts. And this is reflected in the products build and sold by companies, too. Some sell one subset of openstack stuff as product X and maybe a different subset as product Y. (The most common example here is "compute" vs "object storage".) This reality has led to a lot of the angst around definitions since there is effort to define openstack all as one thing (or worse, as a "base" thing that others are defined as built upon). I propose that we can get the benefits of Monty's proposal and implement all of his concrete suggestions (which are fantastic) by slightly adjusting our usage of the program/project concepts. I had originally hoped that the "program" concept would have been a little higher-level instead of effectively spelling "project" as "program". I'd love to see a hierarchy of openstack->program->project/team->repos. Right now, we have added the "program" layer but have effectively mapped it 1:1 to the project. For example, we used to have a few repos in the Swift project managed by the same group of people, and now we have a few repos in the "object storage" program, all managed by the same group of people. And every time something is added to OpenStack, its added as a new program, effectively putting us exactly where we were before we called it a program with the same governance and management scaling problems. Today, I'd group existing OpenStack projects into programs as follows: Compute: nova, sahara, ironic Storage: swift, cinder, glance, trove Network: neutron, designate, zaqar Deployment/management: heat, triple-o, horizon, ceilometer Identity: keystone, barbican Support (not user facing): infra, docs, tempest, devstack, oslo (potentially even) stackforge: lots I like two things about this. First, it allows people to compose a solution. Second, it allows us as a community to thing more about the strategic/product things. For example, it lets us as a community say, "We think storage is important. How are we solving it today? What gaps do we have in that? How can the various storage things we have work together better?" Thierry makes the point that more "themes" will nullify the benefits of Monty's proposal. I agree, if we continue to allow the explosion of projects/programs/themes to continue. The benefit of what Monty is proposing is that it identifies and focusses on a particular use case (deploy a VM, add a volume, get an IP, configure a domain) so that we know we have solved it well. I think that focus is
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Can someone do a small little Visio or other visual to explain what's being proposed here? My head sported a small crack at around the 5-6th page... ; ) But seriously, I couldn't understand the proposal. Maybe I'm not the audience which is fine, just saying, the words got in the way. Sounds like a song! *Adam Lawson* AQORN, Inc. 427 North Tatnall Street Ste. 58461 Wilmington, Delaware 19801-2230 Toll-free: (844) 4-AQORN-NOW ext. 101 International: +1 302-387-4660 Direct: +1 916-246-2072 On Fri, Sep 19, 2014 at 5:46 AM, John Griffith wrote: > > > On Fri, Sep 19, 2014 at 4:33 AM, Thierry Carrez > wrote: > >> Vishvananda Ishaya wrote: >> > Great writeup. I think there are some great concrete suggestions here. >> > >> > A couple more: >> > >> > 1. I think we need a better name for Layer #1 that actually represents >> what the goal of it is: Infrastructure Services? >> > 2. We need to be be open to having other Layer #1s within the >> community. We should allow for similar collaborations and group focus to >> grow up as well. Storage Services? Platform Services? Computation Services? >> >> I think that would nullify most of the benefits of Monty's proposal. If >> we keep on blessing "themes" or special groups, we'll soon be back at >> step 0, with projects banging on the TC door to become special, and >> companies not allocating resources to anything that's not special. >> >> -- >> Thierry Carrez (ttx) >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > Great stuff, mixed on point 2 raised by Vish but honestly I think that's > something that could evolve over time, but I looked at that differently as > in Cinder, SWIFT and some day Manilla live under a Storage Services > umbrella, and ideally at some point there's some convergence there. > > Anyway, I don't want to start a rat-hole on that, it's kind of irrelevant > right now. Bottom line is I think the direction and initial ideas in > Monty's post are what a lot of us have been thinking about and looking for. > I'm in!! > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Fri, Sep 19, 2014 at 4:33 AM, Thierry Carrez wrote: > Vishvananda Ishaya wrote: > > Great writeup. I think there are some great concrete suggestions here. > > > > A couple more: > > > > 1. I think we need a better name for Layer #1 that actually represents > what the goal of it is: Infrastructure Services? > > 2. We need to be be open to having other Layer #1s within the community. > We should allow for similar collaborations and group focus to grow up as > well. Storage Services? Platform Services? Computation Services? > > I think that would nullify most of the benefits of Monty's proposal. If > we keep on blessing "themes" or special groups, we'll soon be back at > step 0, with projects banging on the TC door to become special, and > companies not allocating resources to anything that's not special. > > -- > Thierry Carrez (ttx) > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > Great stuff, mixed on point 2 raised by Vish but honestly I think that's something that could evolve over time, but I looked at that differently as in Cinder, SWIFT and some day Manilla live under a Storage Services umbrella, and ideally at some point there's some convergence there. Anyway, I don't want to start a rat-hole on that, it's kind of irrelevant right now. Bottom line is I think the direction and initial ideas in Monty's post are what a lot of us have been thinking about and looking for. I'm in!! ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Vishvananda Ishaya wrote: > Great writeup. I think there are some great concrete suggestions here. > > A couple more: > > 1. I think we need a better name for Layer #1 that actually represents what > the goal of it is: Infrastructure Services? > 2. We need to be be open to having other Layer #1s within the community. We > should allow for similar collaborations and group focus to grow up as well. > Storage Services? Platform Services? Computation Services? I think that would nullify most of the benefits of Monty's proposal. If we keep on blessing "themes" or special groups, we'll soon be back at step 0, with projects banging on the TC door to become special, and companies not allocating resources to anything that's not special. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Monty Taylor wrote: > I've recently been thinking a lot about Sean's Layers stuff. So I wrote > a blog post which Jim Blair and Devananda were kind enough to help me edit. > > http://inaugust.com/post/108 Hey Monty, As you can imagine, I read that post with great attention. I generally like the concept of a tightly integrated, limited-by-design layer #1 (I'd personally call it "Ring 0") and a large collection of "OpenStack" things gravitating around it. That would at least solve the attraction of the integrated release, suppress the need for incubation, foster competition/innovation within our community, and generally address the problem of community scaling. There are a few details on the consequences though, and in those as always the devil lurks. ## The Technical Committee The Technical Committee is defined in the OpenStack bylaws, and is the representation of the contributors to "the project". Teams work on code repositories, and at some point ask their work to be recognized as part of "OpenStack". In doing so, they place their work under the oversight of the Technical Committee. In return, team members get to participate in electing the technical committee members (they become "ATC"). It's a balanced system, where both parties need to agree: the TC can't force itself as overseer of a random project, and a random project can't just decide by itself it is "OpenStack". I don't see your proposal breaking that balanced system, but it changes its dynamics a bit. The big tent would contain a lot more members. And while the TC would arguably bring a significant share of its attention to Ring 0, its voters constituency would mostly consist of developers who do not participate in Ring 0 development. I don't really see it as changing dramatically the membership of the TC, but it's a consequence worth mentioning. ## Programs Programs were created relatively recently as a way to describe which teams are "in OpenStack" vs. which ones aren't. They directly tie into the ATC system: if you contribute to code repositories under a blessed program, then you're an ATC, you vote in TC elections and the TC has some oversight over your code repositories. Previously, this was granted at a code repository level, but that failed to give flexibility for teams to organize their code in the most convenient manner for them. So we started to bless teams rather than specific code repositories. Now, that didn't work out so well. Some programs were a 'theme', like Infrastructure, or Docs. For those, competing efforts do not really make sense: there can only be one, and competition should happen inside those efforts rather than outside. Some programs were a 'team', like Orchestration/Heat or Deployment/TripleO. And that's where the model started to break: some new orchestration things need space, but the current Heat team is not really interested in maintaining them. What's the point of being under the same program then ? And TripleO is not the only way to deploy OpenStack, but its mere existence (and name) prevented other flowers to bloom in our community. You don't talk much about programs in your proposal. In particular, you only mention layer 1, "Cloud Native" applications, "User Interface" applications, and "Operator" applications. So I'm unsure of where, if anywhere, would "Infrastructure" or "Docs" repositories live. Here is how I see it could work. We could keep 'theme' programs (think Infra, Release cycle management, Docs, QA) with their current structure (collection of code repositories under a single team/PTL). We would get rid of 'team' programs completely, and just have a registry of "OpenStack" code repositories (openstack*/*, big tent). Each of those could have a specific PTL, or explicitely inherit its PTL from another code repository. Under that PTL, they could have separate or same core review team, whatever maps reality and how they want to work (so we could say that openstack/python-novaclient inherits its PTL from openstack/nova and doesn't need a specific one). That would allow us to map anything that may come in. Oslo falls a bit in between, could be considered a 'theme' program or several repos sharing PTL. ## The release and the development cycle You touch briefly on the consequences of your model for the "common release" and our development cycle. Obviously we would release the ring 0 projects in an integrated manner on a time-based schedule. For the other projects, we have a choice: - the "ring 0" model (development milestones, coordinated final release) - the "Swift" model (release as-needed, coordinated final release) - the "Oslo" model (release as-needed, try to have one final one before end of cycle) - the "Client libraries" model (release as-needed) If possible, I would like to avoid the "Swift" model, which is the most costly from a release management standpoint. All projects following the ring 0 model are easy to keep track of, using common freezes etc. So it's easy to make sure they wil
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Great writeup. I think there are some great concrete suggestions here. A couple more: 1. I think we need a better name for Layer #1 that actually represents what the goal of it is: Infrastructure Services? 2. We need to be be open to having other Layer #1s within the community. We should allow for similar collaborations and group focus to grow up as well. Storage Services? Platform Services? Computation Services? Vish On Sep 18, 2014, at 11:53 AM, Monty Taylor wrote: > Hey all, > > I've recently been thinking a lot about Sean's Layers stuff. So I wrote > a blog post which Jim Blair and Devananda were kind enough to help me edit. > > http://inaugust.com/post/108 > > Enjoy. > > Monty > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On 09/18/2014 01:12 PM, Dean Troyer wrote: > On Thu, Sep 18, 2014 at 1:53 PM, Monty Taylor wrote: > >> I've recently been thinking a lot about Sean's Layers stuff. So I wrote >> a blog post which Jim Blair and Devananda were kind enough to help me edit. >> > > Thanks for writing that Monty. Sean took a concept meant for organizing > the relationships in DevStack and Grenade and made it relatable to > OpenStack as a whole. This brings it out where we can actually use it as a > model. > > I do think there is value in distinctions between the original layers 1,2 > and 3 (you combined 1 and 2). But for non-technical purposes 1 and 2 are > indeed the same. > > Suggestion 6: Isn't it funny how everything old is new again? The DevStack > exercises started out as exactly this, then grew more functional over time > until Tempest came along. How hipster is that? Dude. So hipster. > Suggestion 9: Wouldn't it be wonderful if a small set of cloud definition > files could be used for the myriad of user tools out there? Standards, we > don't have enough of them! > > In the long term I would like to see more of our base (toolsets and > services) be pluggable so that the less-blessed projects can participate > without requiring additions to the base repos. This should also be true > for user-facing tools. OpenStackClient already picks up installed plugins > with the proper entry points configured so everything doesn't need to be in > the primary repo to play along. +1000 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Thu, Sep 18, 2014 at 1:53 PM, Monty Taylor wrote: > I've recently been thinking a lot about Sean's Layers stuff. So I wrote > a blog post which Jim Blair and Devananda were kind enough to help me edit. > Thanks for writing that Monty. Sean took a concept meant for organizing the relationships in DevStack and Grenade and made it relatable to OpenStack as a whole. This brings it out where we can actually use it as a model. I do think there is value in distinctions between the original layers 1,2 and 3 (you combined 1 and 2). But for non-technical purposes 1 and 2 are indeed the same. Suggestion 6: Isn't it funny how everything old is new again? The DevStack exercises started out as exactly this, then grew more functional over time until Tempest came along. How hipster is that? Suggestion 9: Wouldn't it be wonderful if a small set of cloud definition files could be used for the myriad of user tools out there? Standards, we don't have enough of them! In the long term I would like to see more of our base (toolsets and services) be pluggable so that the less-blessed projects can participate without requiring additions to the base repos. This should also be true for user-facing tools. OpenStackClient already picks up installed plugins with the proper entry points configured so everything doesn't need to be in the primary repo to play along. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
Hey all, I've recently been thinking a lot about Sean's Layers stuff. So I wrote a blog post which Jim Blair and Devananda were kind enough to help me edit. http://inaugust.com/post/108 Enjoy. Monty ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
