[openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
I'm going to go ahead and ask the difficult question now as the answer is relevant to the attached proposal... Should we reconsider whether the big tent is the right approach going forward? There have been some major downsides I think to the Big Tent approach, such as: * Projects not working together because of fear of adding extra dependencies Ops don't already have * Reimplementing features, badly, over and over again in different projects instead of standardizing something. * More projects being created due to politics and not technical reasons. * Less cross project communication * Greater Operator pain at trying to assemble a more loose confederation of projects into something useful. * General pushing off more and more work to Operators/Users to deal with. * Worse User experience as cross project issues aren't being addressed. * Previously incubated projects such as Nova, Swift, etc getting a disproportionate say in things as they have a greater current user base, and its increasingly hard now for new projects to gain any traction. * Much less community pressure on projects to work together to elevate everyone. Architectural decisions are now made at individual project level with little done at the OpenStack level. * etc... The overall goal of the Big Tent was to make the community more inclusive. That I think has mostly happened. Which is good. But It also seems to have fractured the community more into insular islands and made the system harder for our operators/users. Which is bad. Maybe the benefits outweigh the drawbacks. I'm not sure. But I think its probably time to consider if it has been a net positive and should be further refined to try and address some of these problems, or a net negative and different approaches be explored. Thanks, Kevin From: Hayes, Graham [graham.ha...@hpe.com] Sent: Thursday, July 14, 2016 12:21 PM To: OpenStack Development Mailing List (not for usage questions); openstack...@lists.openstack.org Subject: [openstack-dev] [tc][all] Plugins for all I just proposed a review to openstack/governance repo [0] that aims to have everything across OpenStack be plugin based for all cross project interaction, or allow all projects access to the same internal APIs and I wanted to give a bit of background on my motivation, and how it came about. Coming from a smaller project, I can see issues for new projects, smaller projects, and projects that may not be seen as "important". As a smaller project trying to fit into cross project initiatives, (and yes, make sure our software looks at least OK in the Project Navigator) the process can be difficult. A lot of projects / repositories have plugin interfaces, but also have project integrations in tree, that do not follow the plugin interface. This makes it difficult to see what a plugin can, and should do. When we moved to the big tent, we wanted as a community to move to a flatter model, removing the old integrated status. Unfortunately we still have areas when some projects are more equal - there is a lingering set of projects who were integrated at the point in time that we moved, and have preferential status. A lot of the effects are hard to see, and are not insurmountable, but do cause projects to re-invent the wheel. For example, quotas - there is no way for a project that is not nova, neutron, cinder to hook into the standard CLI, or UI for setting quotas. They can be done as either extra commands (openstack dns quota set --foo bar) or as custom panels, but not the way other quotas get set. Tempest plugins are another example. Approximately 30 of the 36 current plugins are using resources that are not supposed to be used, and are an unstable interface. Projects in tree in tempest are at a much better position, as any change to the internal API will have to be fixed before the gate merges, but other out of tree plugins are in a place where they can be broken at any point. None of this is meant to single out projects, or teams. A lot of the projects that are in this situation have inordinate amounts of work placed on them by the big-tent, and I can emphasize with why things are this way. These were the examples that currently stick out in my mind, and I think we have come to a point where we need to make a change as a community. By moving to a "plugins for all" model, these issues are reduced. It undoubtedly will cause more, but it is closer to our goal of Recognizing all our community is part of OpenStack, and differentiate projects by tags. This won't be a change that happens tomorrow, next week, or even next cycle, but think as a goal, we should start moving in this direction as soon as we can, and start building momentum. Thanks, Graham 0 - https://review.openstack.org/342366 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Kevin, can you please be *specific* about your complaints below? Saying things like "less project communication" and "projects not working together because of fear of adding dependencies" and "worse user experience" are your personal opinions. Please back those opinions up with specific examples of what you are talking about so that we may address specific things and not vague ideas. Also, the overall goal of the Big Tent, as I've said repeatedly and people keep willfully ignoring, was *not* to "make the community more inclusive". It was to replace the inconsistently-applied-by-the-TC *subjective* criteria for project applications to OpenStack with an *objective* list of application requirements that could be *consistently* reviewed by the TC. Thanks, -jay On 07/14/2016 01:30 PM, Fox, Kevin M wrote: I'm going to go ahead and ask the difficult question now as the answer is relevant to the attached proposal... Should we reconsider whether the big tent is the right approach going forward? There have been some major downsides I think to the Big Tent approach, such as: * Projects not working together because of fear of adding extra dependencies Ops don't already have * Reimplementing features, badly, over and over again in different projects instead of standardizing something. * More projects being created due to politics and not technical reasons. * Less cross project communication * Greater Operator pain at trying to assemble a more loose confederation of projects into something useful. * General pushing off more and more work to Operators/Users to deal with. * Worse User experience as cross project issues aren't being addressed. * Previously incubated projects such as Nova, Swift, etc getting a disproportionate say in things as they have a greater current user base, and its increasingly hard now for new projects to gain any traction. * Much less community pressure on projects to work together to elevate everyone. Architectural decisions are now made at individual project level with little done at the OpenStack level. * etc... The overall goal of the Big Tent was to make the community more inclusive. That I think has mostly happened. Which is good. But It also seems to have fractured the community more into insular islands and made the system harder for our operators/users. Which is bad. Maybe the benefits outweigh the drawbacks. I'm not sure. But I think its probably time to consider if it has been a net positive and should be further refined to try and address some of these problems, or a net negative and different approaches be explored. Thanks, Kevin From: Hayes, Graham [graham.ha...@hpe.com] Sent: Thursday, July 14, 2016 12:21 PM To: OpenStack Development Mailing List (not for usage questions); openstack...@lists.openstack.org Subject: [openstack-dev] [tc][all] Plugins for all I just proposed a review to openstack/governance repo [0] that aims to have everything across OpenStack be plugin based for all cross project interaction, or allow all projects access to the same internal APIs and I wanted to give a bit of background on my motivation, and how it came about. Coming from a smaller project, I can see issues for new projects, smaller projects, and projects that may not be seen as "important". As a smaller project trying to fit into cross project initiatives, (and yes, make sure our software looks at least OK in the Project Navigator) the process can be difficult. A lot of projects / repositories have plugin interfaces, but also have project integrations in tree, that do not follow the plugin interface. This makes it difficult to see what a plugin can, and should do. When we moved to the big tent, we wanted as a community to move to a flatter model, removing the old integrated status. Unfortunately we still have areas when some projects are more equal - there is a lingering set of projects who were integrated at the point in time that we moved, and have preferential status. A lot of the effects are hard to see, and are not insurmountable, but do cause projects to re-invent the wheel. For example, quotas - there is no way for a project that is not nova, neutron, cinder to hook into the standard CLI, or UI for setting quotas. They can be done as either extra commands (openstack dns quota set --foo bar) or as custom panels, but not the way other quotas get set. Tempest plugins are another example. Approximately 30 of the 36 current plugins are using resources that are not supposed to be used, and are an unstable interface. Projects in tree in tempest are at a much better position, as any change to the internal API will have to be fixed before the gate merges, but other out of tree plugins are in a place where they can be broken at any point. None of this is meant to single out projects, or teams. A lot of the projects that are in this situation have inordinate amounts of work placed on them by the big
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On 14/07/2016 21:33, Fox, Kevin M wrote: > I'm going to go ahead and ask the difficult question now as the answer is > relevant to the attached proposal... > > Should we reconsider whether the big tent is the right approach going forward? > > There have been some major downsides I think to the Big Tent approach, such > as: > * Projects not working together because of fear of adding extra dependencies > Ops don't already have > * Reimplementing features, badly, over and over again in different projects > instead of standardizing something. > * More projects being created due to politics and not technical reasons. > * Less cross project communication > * Greater Operator pain at trying to assemble a more loose confederation of > projects into something useful. > * General pushing off more and more work to Operators/Users to deal with. > * Worse User experience as cross project issues aren't being addressed. > * Previously incubated projects such as Nova, Swift, etc getting a > disproportionate say in things as they have a greater current user base, and > its increasingly hard now for new projects to gain any traction. > * Much less community pressure on projects to work together to elevate > everyone. Architectural decisions are now made at individual project level > with little done at the OpenStack level. > * etc... > > The overall goal of the Big Tent was to make the community more inclusive. > That I think has mostly happened. Which is good. But It also seems to have > fractured the community more into insular islands and made the system harder > for our operators/users. Which is bad. > > Maybe the benefits outweigh the drawbacks. I'm not sure. But I think its > probably time to consider if it has been a net positive and should be further > refined to try and address some of these problems, or a net negative and > different approaches be explored. > > Thanks, > Kevin Sure - having a review of a major governance change like the big-tent is a good idea, and pretty good practice. I personally don't think we have reached the end of the transition yet, and proposals like mine will start to come along and deal with issues outlined above. We should be sure that we have gone as far as we can before we claim defeat. That said, there is serious issues - with cross project fairness being one of the big ones (from my perspective anyway). I think getting concrete examples of problems and tackling them is a good road forward. - Graham > > From: Hayes, Graham [graham.ha...@hpe.com] > Sent: Thursday, July 14, 2016 12:21 PM > To: OpenStack Development Mailing List (not for usage questions); > openstack...@lists.openstack.org > Subject: [openstack-dev] [tc][all] Plugins for all > > I just proposed a review to openstack/governance repo [0] that aims > to have everything across OpenStack be plugin based for all cross > project interaction, or allow all projects access to the same internal > APIs and I wanted to give a bit of background on my motivation, and how > it came about. > > Coming from a smaller project, I can see issues for new projects, > smaller projects, and projects that may not be seen as "important". > > As a smaller project trying to fit into cross project initiatives, > (and yes, make sure our software looks at least OK in the > Project Navigator) the process can be difficult. > > A lot of projects / repositories have plugin interfaces, but also > have project integrations in tree, that do not follow the plugin > interface. This makes it difficult to see what a plugin can, and > should do. > > When we moved to the big tent, we wanted as a community to move to > a flatter model, removing the old integrated status. > > Unfortunately we still have areas when some projects are more equal - > there is a lingering set of projects who were integrated at the point > in time that we moved, and have preferential status. > > A lot of the effects are hard to see, and are not insurmountable, but > do cause projects to re-invent the wheel. > > For example, quotas - there is no way for a project that is not nova, > neutron, cinder to hook into the standard CLI, or UI for setting > quotas. They can be done as either extra commands > (openstack dns quota set --foo bar) or as custom panels, but not > the way other quotas get set. > > Tempest plugins are another example. Approximately 30 of the 36 > current plugins are using resources that are not supposed to be > used, and are an unstable interface. Projects in tree in tempest > are at a much better position, as any change to the internal > API will have to be fixed before the gate merges, but other > out of tree plugins are in a place where they can be broken at any > point. > > None of this is meant to single out projects, or teams. A lot > of the projects that are in this situation have inordinate amounts of > work placed on them by the big-tent, and I can emphasize with why things > are this way. The
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Some specific things: Magnum trying to not use Barbican as it adds an addition dependency. See the discussion on the devel mailing list for details. Horizon discussions at the summit around wanting to use Zaqar for dynamic ui updates instead of polling, but couldn't depend on a non widely deployed subsystem. Each Advanced OpenStack Service implementing a guest controller communication channel that are incompatible with each other and work around communications issues differently. This makes a lot more pain for Ops to debug or architect a viable solution. For example: * Sahara uses ssh from the controllers to the vms. This does not play well with tenant networks. They have tried to work around this several ways: * require every vm to have a floating ip. (Unnecessary attack surface) * require the controller to be on the one and only network node (Uses ip netns exec to tunnel. Doesn't work for large clouds) * Double tunnel ssh via the controller vm's. so some vms have fips, some don't. Better then all, but still not good. * Trove uses Rabbit for the guest agent to talk back to the controllers. This has traffic going the right direction to work well with tenant networks. * But Rabbit is not multitenant so a security risk if any user can get into any one of the database vm's. Really, I believe the right solution is to use a multitenant aware message queue so that the guest agent can pull in the right direction for tenant networks, and not have the security risk. We have such a system already, Zaqar, but its not widely deployed and projects don't want to depend on other projects that aren't widely deployed. The lack of Instance Users has caused lots of projects to try and work around the lack thereof. I know for sure, Mangum, Heat, and Trove work around the lack. I'm positive others have too. As an operator, it makes me have to very carefully consider all the tradeoffs each project made, and decide if I can accept the same risk they assumed. Since each is different, thats much harder. I'm sure there are more examples. but I hope you get I'm not just trying to troll. The TC did apply inconsistant rules on letting projects in. That was for sure a negative before the big tent. But it also provided a way to apply pressure to projects to fix some of the issues that multiple projects face, and that plague user/operators and raise the whole community up, and that has fallen to the wayside since. Which is a big negative now. Maybe that could be bolted on top of the Big Tent I don't know. I could write a very long description about the state of being an OpenStack App developer too that touches on all the problems with getting a consistent target and all the cross project communication issues there of. But thats probably for some other time. Thanks, Kevin From: Jay Pipes [jaypi...@gmail.com] Sent: Friday, July 15, 2016 8:17 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all) Kevin, can you please be *specific* about your complaints below? Saying things like "less project communication" and "projects not working together because of fear of adding dependencies" and "worse user experience" are your personal opinions. Please back those opinions up with specific examples of what you are talking about so that we may address specific things and not vague ideas. Also, the overall goal of the Big Tent, as I've said repeatedly and people keep willfully ignoring, was *not* to "make the community more inclusive". It was to replace the inconsistently-applied-by-the-TC *subjective* criteria for project applications to OpenStack with an *objective* list of application requirements that could be *consistently* reviewed by the TC. Thanks, -jay On 07/14/2016 01:30 PM, Fox, Kevin M wrote: > I'm going to go ahead and ask the difficult question now as the answer is > relevant to the attached proposal... > > Should we reconsider whether the big tent is the right approach going forward? > > There have been some major downsides I think to the Big Tent approach, such > as: > * Projects not working together because of fear of adding extra > dependencies Ops don't already have > * Reimplementing features, badly, over and over again in different projects > instead of standardizing something. > * More projects being created due to politics and not technical reasons. > * Less cross project communication > * Greater Operator pain at trying to assemble a more loose confederation of > projects into something useful. > * General pushing off more and more work to Operators/Users to deal with. > * Worse User experience as cross project issues aren't being addressed. > * Previously incubated projects suc
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
y long description about the state of being an > OpenStack App developer too that touches on all the problems with getting a > consistent target and all the cross project communication issues there of. > But thats probably for some other time. > > Thanks, > Kevin > > > From: Jay Pipes [jaypi...@gmail.com] > Sent: Friday, July 15, 2016 8:17 AM > To: openstack-dev@lists.openstack.org > Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for > all) > > Kevin, can you please be *specific* about your complaints below? Saying > things like "less project communication" and "projects not working > together because of fear of adding dependencies" and "worse user > experience" are your personal opinions. Please back those opinions up > with specific examples of what you are talking about so that we may > address specific things and not vague ideas. > > Also, the overall goal of the Big Tent, as I've said repeatedly and > people keep willfully ignoring, was *not* to "make the community more > inclusive". It was to replace the inconsistently-applied-by-the-TC > *subjective* criteria for project applications to OpenStack with an > *objective* list of application requirements that could be > *consistently* reviewed by the TC. > > Thanks, > -jay > > On 07/14/2016 01:30 PM, Fox, Kevin M wrote: > > I'm going to go ahead and ask the difficult question now as the answer > is relevant to the attached proposal... > > > > Should we reconsider whether the big tent is the right approach going > forward? > > > > There have been some major downsides I think to the Big Tent approach, > such as: > > * Projects not working together because of fear of adding extra > dependencies Ops don't already have > > * Reimplementing features, badly, over and over again in different > projects instead of standardizing something. > > * More projects being created due to politics and not technical > reasons. > > * Less cross project communication > > * Greater Operator pain at trying to assemble a more loose > confederation of projects into something useful. > > * General pushing off more and more work to Operators/Users to deal > with. > > * Worse User experience as cross project issues aren't being addressed. > > * Previously incubated projects such as Nova, Swift, etc getting a > disproportionate say in things as they have a greater current user base, > and its increasingly hard now for new projects to gain any traction. > > * Much less community pressure on projects to work together to elevate > everyone. Architectural decisions are now made at individual project level > with little done at the OpenStack level. > > * etc... > > > > The overall goal of the Big Tent was to make the community more > inclusive. That I think has mostly happened. Which is good. But It also > seems to have fractured the community more into insular islands and made > the system harder for our operators/users. Which is bad. > > > > Maybe the benefits outweigh the drawbacks. I'm not sure. But I think its > probably time to consider if it has been a net positive and should be > further refined to try and address some of these problems, or a net > negative and different approaches be explored. > > > > Thanks, > > Kevin > > > > From: Hayes, Graham [graham.ha...@hpe.com] > > Sent: Thursday, July 14, 2016 12:21 PM > > To: OpenStack Development Mailing List (not for usage questions); > openstack...@lists.openstack.org > > Subject: [openstack-dev] [tc][all] Plugins for all > > > > I just proposed a review to openstack/governance repo [0] that aims > > to have everything across OpenStack be plugin based for all cross > > project interaction, or allow all projects access to the same internal > > APIs and I wanted to give a bit of background on my motivation, and how > > it came about. > > > > Coming from a smaller project, I can see issues for new projects, > > smaller projects, and projects that may not be seen as "important". > > > > As a smaller project trying to fit into cross project initiatives, > > (and yes, make sure our software looks at least OK in the > > Project Navigator) the process can be difficult. > > > > A lot of projects / repositories have plugin interfaces, but also > > have project integrations in tree, that do not follow the plugin > > interface. This makes it difficult to see what a plugin can, and > > should do. > > > > When we moved to the big
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
tem. >> >> Each Advanced OpenStack Service implementing a guest controller >> communication channel that are incompatible with each other and work around >> communications issues differently. This makes a lot more pain for Ops to >> debug or architect a viable solution. For example: >> * Sahara uses ssh from the controllers to the vms. This does not play >> well with tenant networks. They have tried to work around this several ways: >> * require every vm to have a floating ip. (Unnecessary attack surface) >> * require the controller to be on the one and only network node (Uses >> ip netns exec to tunnel. Doesn't work for large clouds) >> * Double tunnel ssh via the controller vm's. so some vms have fips, >> some don't. Better then all, but still not good. >> * Trove uses Rabbit for the guest agent to talk back to the >> controllers. This has traffic going the right direction to work well with >> tenant networks. >> * But Rabbit is not multitenant so a security risk if any user can >> get into any one of the database vm's. >> Really, I believe the right solution is to use a multitenant aware >> message queue so that the guest agent can pull in the right direction for >> tenant networks, and not have the security risk. We have such a system >> already, Zaqar, but its not widely deployed and projects don't want to >> depend on other projects that aren't widely deployed. >> >> The lack of Instance Users has caused lots of projects to try and work >> around the lack thereof. I know for sure, Mangum, Heat, and Trove work >> around the lack. I'm positive others have too. As an operator, it makes me >> have to very carefully consider all the tradeoffs each project made, and >> decide if I can accept the same risk they assumed. Since each is different, >> thats much harder. >> >> I'm sure there are more examples. but I hope you get I'm not just trying >> to troll. >> >> The TC did apply inconsistant rules on letting projects in. That was for >> sure a negative before the big tent. But it also provided a way to apply >> pressure to projects to fix some of the issues that multiple projects face, >> and that plague user/operators and raise the whole community up, and that >> has fallen to the wayside since. Which is a big negative now. Maybe that >> could be bolted on top of the Big Tent I don't know. >> >> I could write a very long description about the state of being an >> OpenStack App developer too that touches on all the problems with getting a >> consistent target and all the cross project communication issues there of. >> But thats probably for some other time. >> >> Thanks, >> Kevin >> >> >> From: Jay Pipes [jaypi...@gmail.com] >> Sent: Friday, July 15, 2016 8:17 AM >> To: openstack-dev@lists.openstack.org >> Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for >> all) >> >> Kevin, can you please be *specific* about your complaints below? Saying >> things like "less project communication" and "projects not working >> together because of fear of adding dependencies" and "worse user >> experience" are your personal opinions. Please back those opinions up >> with specific examples of what you are talking about so that we may >> address specific things and not vague ideas. >> >> Also, the overall goal of the Big Tent, as I've said repeatedly and >> people keep willfully ignoring, was *not* to "make the community more >> inclusive". It was to replace the inconsistently-applied-by-the-TC >> *subjective* criteria for project applications to OpenStack with an >> *objective* list of application requirements that could be >> *consistently* reviewed by the TC. >> >> Thanks, >> -jay >> >> On 07/14/2016 01:30 PM, Fox, Kevin M wrote: >> > I'm going to go ahead and ask the difficult question now as the answer >> is relevant to the attached proposal... >> > >> > Should we reconsider whether the big tent is the right approach going >> forward? >> > >> > There have been some major downsides I think to the Big Tent approach, >> such as: >> > * Projects not working together because of fear of adding extra >> dependencies Ops don't already have >> > * Reimplementing features, badly, over and over again in different >> projects instead of standardizing something. >> > * More projects being created due to politics
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
No, Magnum still uses Barbican as an optional dependency, and I believe nobody has proposed to remove Barbican entirely. I have no position about big tent but using Magnum as an example of "projects are not working together" is inappropriate. Best regards, Hongbin > -Original Message- > From: Fox, Kevin M [mailto:kevin@pnnl.gov] > Sent: July-15-16 2:37 PM > To: OpenStack Development Mailing List (not for usage questions) > Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins > for all) > > Some specific things: > > Magnum trying to not use Barbican as it adds an addition dependency. > See the discussion on the devel mailing list for details. > > Horizon discussions at the summit around wanting to use Zaqar for > dynamic ui updates instead of polling, but couldn't depend on a non > widely deployed subsystem. > > Each Advanced OpenStack Service implementing a guest controller > communication channel that are incompatible with each other and work > around communications issues differently. This makes a lot more pain > for Ops to debug or architect a viable solution. For example: > * Sahara uses ssh from the controllers to the vms. This does not play > well with tenant networks. They have tried to work around this several > ways: > * require every vm to have a floating ip. (Unnecessary attack > surface) > * require the controller to be on the one and only network node > (Uses ip netns exec to tunnel. Doesn't work for large clouds) > * Double tunnel ssh via the controller vm's. so some vms have fips, > some don't. Better then all, but still not good. > * Trove uses Rabbit for the guest agent to talk back to the > controllers. This has traffic going the right direction to work well > with tenant networks. > * But Rabbit is not multitenant so a security risk if any user can > get into any one of the database vm's. > Really, I believe the right solution is to use a multitenant aware > message queue so that the guest agent can pull in the right direction > for tenant networks, and not have the security risk. We have such a > system already, Zaqar, but its not widely deployed and projects don't > want to depend on other projects that aren't widely deployed. > > The lack of Instance Users has caused lots of projects to try and work > around the lack thereof. I know for sure, Mangum, Heat, and Trove work > around the lack. I'm positive others have too. As an operator, it makes > me have to very carefully consider all the tradeoffs each project made, > and decide if I can accept the same risk they assumed. Since each is > different, thats much harder. > > I'm sure there are more examples. but I hope you get I'm not just > trying to troll. > > The TC did apply inconsistant rules on letting projects in. That was > for sure a negative before the big tent. But it also provided a way to > apply pressure to projects to fix some of the issues that multiple > projects face, and that plague user/operators and raise the whole > community up, and that has fallen to the wayside since. Which is a big > negative now. Maybe that could be bolted on top of the Big Tent I don't > know. > > I could write a very long description about the state of being an > OpenStack App developer too that touches on all the problems with > getting a consistent target and all the cross project communication > issues there of. But thats probably for some other time. > > Thanks, > Kevin > > > From: Jay Pipes [jaypi...@gmail.com] > Sent: Friday, July 15, 2016 8:17 AM > To: openstack-dev@lists.openstack.org > Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins > for all) > > Kevin, can you please be *specific* about your complaints below? Saying > things like "less project communication" and "projects not working > together because of fear of adding dependencies" and "worse user > experience" are your personal opinions. Please back those opinions up > with specific examples of what you are talking about so that we may > address specific things and not vague ideas. > > Also, the overall goal of the Big Tent, as I've said repeatedly and > people keep willfully ignoring, was *not* to "make the community more > inclusive". It was to replace the inconsistently-applied-by-the-TC > *subjective* criteria for project applications to OpenStack with an > *objective* list of application requirements that could be > *consistently* reviewed by the TC. > > Thanks, > -jay > > On 07/14/2016 01:30 PM, Fox, Kevin M wrote: > > I'm going to go ahead and ask the dif
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
> -Original Message- > From: Fox, Kevin M [mailto:kevin@pnnl.gov] > Sent: Friday, July 15, 2016 2:37 PM > To: OpenStack Development Mailing List (not for usage questions) > > Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for > all) > > Some specific things: > > Magnum trying to not use Barbican as it adds an addition dependency. See > the discussion on the devel mailing list for details. > > Horizon discussions at the summit around wanting to use Zaqar for dynamic > ui updates instead of polling, but couldn't depend on a non widely > deployed subsystem. > > Each Advanced OpenStack Service implementing a guest controller > communication channel that are incompatible with each other and work > around communications issues differently. This makes a lot more pain for > Ops to debug or architect a viable solution. For example: > * Sahara uses ssh from the controllers to the vms. This does not play > well with tenant networks. They have tried to work around this several > ways: > * require every vm to have a floating ip. (Unnecessary attack surface) > * require the controller to be on the one and only network node (Uses > ip netns exec to tunnel. Doesn't work for large clouds) > * Double tunnel ssh via the controller vm's. so some vms have fips, > some don't. Better then all, but still not good. > * Trove uses Rabbit for the guest agent to talk back to the controllers. > This has traffic going the right direction to work well with tenant > networks. > * But Rabbit is not multitenant so a security risk if any user can get > into any one of the database vm's. > Really, I believe the right solution is to use a multitenant aware message > queue so that the guest agent can pull in the right direction for tenant > networks, and not have the security risk. We have such a system already, > Zaqar, but its not widely deployed and projects don't want to depend on > other projects that aren't widely deployed. [amrith] Kevin, to the best of my knowledge, Zaqar does not eliminate Trove's security risks if one can compromise a database's VM's. It merely reduces the extent of the compromise (to the tenant whose vm's have been compromised). This understanding is based on a conversation with the Zaqar folks in Vancouver, and later in Tokyo on possible integrations with Trove. I will observe that it is perfectly possible to operate Trove with a service tenant and thereby eliminate any opportunity for a user to be able to compromise a guest instance that has been created with a rational image (disabled ssh, etc.,). Since the image creation and registration is an operator activity, it places the ability to secure the Trove deployment well within the control of the operator. It is also the case that one can further tighten things up by making each guest instance use its own private RabbitMQ credentials (something that the code currently does not support). Setting those things aside, let me also point out that if one wants to implement a database as a service, then the controller needs to be able to perform some management of the database instances. One way of doing that is the current architecture that has the guest agent on the guest instance, and another is to have a remote guest agent. One way or another, there is going to have to be communication between the controller and the guest instance. I believe that there is no way around that. I'm unclear why it is unacceptable that the guest instance have a network interface on a controller network with tight firewall protections on what traffic is allowed there. I believe that each project should be allowed to adopt the solution(s) that are best suited to their own situation, within reason. Mandating that projects shall use a specific solution will, I believe, lead to a worse overall outcome. > > The lack of Instance Users has caused lots of projects to try and work > around the lack thereof. I know for sure, Mangum, Heat, and Trove work > around the lack. I'm positive others have too. As an operator, it makes me > have to very carefully consider all the tradeoffs each project made, and > decide if I can accept the same risk they assumed. Since each is > different, thats much harder. > > I'm sure there are more examples. but I hope you get I'm not just trying > to troll. > > The TC did apply inconsistant rules on letting projects in. That was for > sure a negative before the big tent. But it also provided a way to apply > pressure to projects to fix some of the issues that multiple projects > face, and that plague user/operators and raise the whole community up, and > that has fallen to the wayside since. Which is a big negative now. Maybe > that could be
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On Fri, Jul 15, 2016 at 8:36 PM, Fox, Kevin M wrote: > Some specific things: > > Magnum trying to not use Barbican as it adds an addition dependency. See the > discussion on the devel mailing list for details. > > Horizon discussions at the summit around wanting to use Zaqar for dynamic ui > updates instead of polling, but couldn't depend on a non widely deployed > subsystem. > > Each Advanced OpenStack Service implementing a guest controller communication > channel that are incompatible with each other and work around communications > issues differently. This makes a lot more pain for Ops to debug or architect > a viable solution. For example: > * Sahara uses ssh from the controllers to the vms. This does not play well > with tenant networks. They have tried to work around this several ways: > * require every vm to have a floating ip. (Unnecessary attack surface) > * require the controller to be on the one and only network node (Uses ip > netns exec to tunnel. Doesn't work for large clouds) > * Double tunnel ssh via the controller vm's. so some vms have fips, some > don't. Better then all, but still not good. > * Trove uses Rabbit for the guest agent to talk back to the controllers. > This has traffic going the right direction to work well with tenant networks. > * But Rabbit is not multitenant so a security risk if any user can get > into any one of the database vm's. > Really, I believe the right solution is to use a multitenant aware message > queue so that the guest agent can pull in the right direction for tenant > networks, and not have the security risk. We have such a system already, > Zaqar, but its not widely deployed and projects don't want to depend on other > projects that aren't widely deployed. While I agree using Barbican/Zaqar for those would be fantastic, this is easily solved: just optionally depend on it. Heat provides features that use Zaqar, which are not just present when Zaqar is not there. For example, if people in the Trove project were convinced that Zaqar was the best solution to their problem, I think they would find a way to provide an implementation using it. I don't think they are, though. Whatever the reasons may be. > The lack of Instance Users has caused lots of projects to try and work around > the lack thereof. I know for sure, Mangum, Heat, and Trove work around the > lack. I'm positive others have too. As an operator, it makes me have to very > carefully consider all the tradeoffs each project made, and decide if I can > accept the same risk they assumed. Since each is different, thats much harder. Instance users would be nice. Nobody just provided a good solution. I know you tried, but I don't think you succeeded. We need a good implementation, easy to understand, and maybe this will move forward. > I'm sure there are more examples. but I hope you get I'm not just trying to > troll. > > The TC did apply inconsistant rules on letting projects in. That was for sure > a negative before the big tent. But it also provided a way to apply pressure > to projects to fix some of the issues that multiple projects face, and that > plague user/operators and raise the whole community up, and that has fallen > to the wayside since. Which is a big negative now. Maybe that could be bolted > on top of the Big Tent I don't know. So I think that's the root of the issue here. You assume we can "pressure" people do something you think is right. But that's not how opensource works. You either implement the solution to your problem, or convince someone else, but you don't force anybody. I think that's what the TC recognized and changed, to the correct path. Could project be better integrated, so that we have a more coherent "product"? Sure. But you don't achieve that with more governance and processes. -- Thomas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On 07/15/2016 08:52 AM, Hayes, Graham wrote: I think getting concrete examples of problems and tackling them is a good road forward. Could not agree more. I eagerly await this. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On 15/07/16 18:36 +, Fox, Kevin M wrote: Some specific things: Magnum trying to not use Barbican as it adds an addition dependency. See the discussion on the devel mailing list for details. Horizon discussions at the summit around wanting to use Zaqar for dynamic ui updates instead of polling, but couldn't depend on a non widely deployed subsystem. Are the two points above a big tent issue? We did have that problem before the big tent because distributions wouldn't package projects that weren't integrated and OPs wouldn't deploy projects that weren't integrated. I do not think that's true anymore. In fact, projects like TripleO and Heat are already using Zaqar, despite the fact it's not widely deployed. As Thomas mentioned in his reply to this email, optional dependencies are the answer here. Re-inventing the wheel to avoid adding a new dependency is the wrong approach but that's not an issue originated by the big tent itself. Each Advanced OpenStack Service implementing a guest controller communication channel that are incompatible with each other and work around communications issues differently. This makes a lot more pain for Ops to debug or architect a viable solution. For example: * Sahara uses ssh from the controllers to the vms. This does not play well with tenant networks. They have tried to work around this several ways: * require every vm to have a floating ip. (Unnecessary attack surface) * require the controller to be on the one and only network node (Uses ip netns exec to tunnel. Doesn't work for large clouds) * Double tunnel ssh via the controller vm's. so some vms have fips, some don't. Better then all, but still not good. * Trove uses Rabbit for the guest agent to talk back to the controllers. This has traffic going the right direction to work well with tenant networks. * But Rabbit is not multitenant so a security risk if any user can get into any one of the database vm's. Really, I believe the right solution is to use a multitenant aware message queue so that the guest agent can pull in the right direction for tenant networks, and not have the security risk. We have such a system already, Zaqar, but its not widely deployed and projects don't want to depend on other projects that aren't widely deployed. I believe what you're describing above is not really related to how widely deployed Zaqar is, but I might be being too naive here. Even if it were, though, I still don't think that issue was caused by the big tent change. If anything, the big tent change alleviated part of the issue. The lack of Instance Users has caused lots of projects to try and work around the lack thereof. I know for sure, Mangum, Heat, and Trove work around the lack. I'm positive others have too. As an operator, it makes me have to very carefully consider all the tradeoffs each project made, and decide if I can accept the same risk they assumed. Since each is different, thats much harder. I'd really appreciate if you'd ellaborate on why this issue was caused by the big tent. I'm sure there are more examples. but I hope you get I'm not just trying to troll. I don't think you are but I'm lacking of some context that I get the feeling you have. Hopefully I'm not being short-minded. The TC did apply inconsistant rules on letting projects in. That was for sure a negative before the big tent. But it also provided a way to apply pressure to projects to fix some of the issues that multiple projects face, and that plague user/operators and raise the whole community up, and that has fallen to the wayside since. Which is a big negative now. Maybe that could be bolted on top of the Big Tent I don't know. I could write a very long description about the state of being an OpenStack App developer too that touches on all the problems with getting a consistent target and all the cross project communication issues there of. But thats probably for some other time. TBH, I'd be interested in such email but I'd beg you to keep it in a separate thread as I don't think such email relates entirely to this one. I believe the things you're describing are side-effects caused by the growth of our community, which I consider a good thing. As Graham said in one of his replies, we've not seen the end of the big tent yet and I believe there are still good and bad things to come. How fast we'll adapt/react to those will determine the success of the big tent. I agree there are things that need to be changed or reflected on already and your email, Graham's and Clynt's are a good example of the community trying/willing to keep up with the changes. Looking forward to your reply, Flavio -- @flaper87 Flavio Percoco signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.ope
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On 16 Jul 2016 1:27 PM, "Thomas Herve" wrote: > > On Fri, Jul 15, 2016 at 8:36 PM, Fox, Kevin M wrote: > > Some specific things: > > > > Magnum trying to not use Barbican as it adds an addition dependency. See the discussion on the devel mailing list for details. > > > > Horizon discussions at the summit around wanting to use Zaqar for dynamic ui updates instead of polling, but couldn't depend on a non widely deployed subsystem. > > > > Each Advanced OpenStack Service implementing a guest controller communication channel that are incompatible with each other and work around communications issues differently. This makes a lot more pain for Ops to debug or architect a viable solution. For example: > > * Sahara uses ssh from the controllers to the vms. This does not play well with tenant networks. They have tried to work around this several ways: > > * require every vm to have a floating ip. (Unnecessary attack surface) > > * require the controller to be on the one and only network node (Uses ip netns exec to tunnel. Doesn't work for large clouds) > > * Double tunnel ssh via the controller vm's. so some vms have fips, some don't. Better then all, but still not good. > > * Trove uses Rabbit for the guest agent to talk back to the controllers. This has traffic going the right direction to work well with tenant networks. > > * But Rabbit is not multitenant so a security risk if any user can get into any one of the database vm's. > > Really, I believe the right solution is to use a multitenant aware message queue so that the guest agent can pull in the right direction for tenant networks, and not have the security risk. We have such a system already, Zaqar, but its not widely deployed and projects don't want to depend on other projects that aren't widely deployed. > > While I agree using Barbican/Zaqar for those would be fantastic, this > is easily solved: just optionally depend on it. Heat provides features > that use Zaqar, which are not just present when Zaqar is not there. > For example, if people in the Trove project were convinced that Zaqar > was the best solution to their problem, I think they would find a way > to provide an implementation using it. I don't think they are, though. > Whatever the reasons may be. > > > The lack of Instance Users has caused lots of projects to try and work around the lack thereof. I know for sure, Mangum, Heat, and Trove work around the lack. I'm positive others have too. As an operator, it makes me have to very carefully consider all the tradeoffs each project made, and decide if I can accept the same risk they assumed. Since each is different, thats much harder. > > Instance users would be nice. Nobody just provided a good solution. I > know you tried, but I don't think you succeeded. We need a good > implementation, easy to understand, and maybe this will move forward. I think I need a more complete definition of instance users to know what you're talking about here. Is this the instance locking stuff Bruno proposed a while ago? > > I'm sure there are more examples. but I hope you get I'm not just trying to troll. > > > > The TC did apply inconsistant rules on letting projects in. That was for sure a negative before the big tent. But it also provided a way to apply pressure to projects to fix some of the issues that multiple projects face, and that plague user/operators and raise the whole community up, and that has fallen to the wayside since. Which is a big negative now. Maybe that could be bolted on top of the Big Tent I don't know. > > So I think that's the root of the issue here. You assume we can > "pressure" people do something you think is right. But that's not how > opensource works. You either implement the solution to your problem, > or convince someone else, but you don't force anybody. I think that's > what the TC recognized and changed, to the correct path. Could project > be better integrated, so that we have a more coherent "product"? Sure. > But you don't achieve that with more governance and processes. > > -- > Thomas > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Michael __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
I don't object to the need of an OpenStack controller and a vm need to communicate. I object to every project coming up with their own solution for that communication channel, when there really is no benefit to them being different. It is one thing amongst many that has made being an Operator's life very difficult as the individual OpenStack projects don't consider all the stuff they are asking an op to do, in aggregate. This has parallels with, say, college professors. They often do not talk to each other, and will give out a fair amount of homework, which, by itself doesn't sound unreasonable, but when you have a lot of classes, they can add up sometimes to periods of time where you have an unreasonable workload. Its not such a problem in academia, as students are usually kind of a captive audience. Operators can choose to move on though. Other then the specific messages being sent (payload) between heat and vm, trove and vm, sahara and vm, etc, are the needs of the base communication channel any different between the projects? I don't think there are. But today each projects workarounds/solutions/monitoring are all different. I feel like I'm just the messenger here. Please don't shoot me. I'm generally trying to provide good, though hard feedback to better a perceived problem. Many others would just stay silent or leave rather then have the difficult discussion. For reference, have a look here: https://www.google.com/trends/explore#q=%2Fm%2F0cm87w_&cmpt=q&tz=Etc%2FGMT%2B7 The slope of the line has dramatically changed. I'm sure there are many reasons for it, but the difficulty of being an OpenStack operator or app developer in a world where there are increasing numbers of alternatives I'm sure plays a part. (Some other things, docker and docker COE's staring to gain traction, the introduction of the big tent) The openness of the big tent has allowed more choice in the ecosystem, which seems good, but I think it has also stalled development on the sorts of cross project issues that were important to our users and are now looking elsewhere for solutions. I have to admit, I've gotten frustrated after years of waiting, and trying to fix things, I'm starting to switch some systems away from OpenStack to things that are easier to deal with. Again, please understand, there is no malice here, just stating the way things are. We should try and fix it so this is not the case. As for the specific Trove security risk mentioned below, you can have a zaqar queue per vm, not per tenant, and compromising the vm only compromise control of that one vm. Thats much preferable to the current situation. But again, this is in no way Trove specific. We should be talking this issue through with all the related projects. Service tenants are not an option for us today, as we have users that buy hardware that gets dedicated to them. they use flavors with permissions to land vm's onto their hardware pools. Service tenants don't have access to those flavors. Again, this is a solution that was thought up and provided by a single project (Trove) but should be considered at the general OpenStack level. It isn't specific to Trove, but affects Sahara, Magnum, etc too. There isn't anything Trove specific about the issue or the solution. There needs to be a general place... technical committee? that helps design these sorts of solutions that deal with the needs of multiple OpenStack projects and operators and pushes through features (like nova service vm's) that fills in the current implementation gaps to get good solutions. This, everyone for themselves way of solving things isn't working so good. Thanks, Kevin From: Amrith Kumar [amr...@tesora.com] Sent: Saturday, July 16, 2016 8:41 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all) > -Original Message- > From: Fox, Kevin M [mailto:kevin@pnnl.gov] > Sent: Friday, July 15, 2016 2:37 PM > To: OpenStack Development Mailing List (not for usage questions) > > Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for > all) > > Some specific things: > > Magnum trying to not use Barbican as it adds an addition dependency. See > the discussion on the devel mailing list for details. > > Horizon discussions at the summit around wanting to use Zaqar for dynamic > ui updates instead of polling, but couldn't depend on a non widely > deployed subsystem. > > Each Advanced OpenStack Service implementing a guest controller > communication channel that are incompatible with each other and work > around communications issues differently. This makes a lot more pain for > Ops to debug or architect a viable solu
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
I'm arguing the opposite. It should be a requirement to use the OpenStack Secret Store. Instead, we're trying to make it optional and either reimplementing large swaths of it in individual projects to make it optional, or skip security all together. If it wasn't optional, then: * Keystone could rely on it being there for password hashing * Magnum can rely on it for security. * lbaas can rely on it * App developers can rely on it being there * ... Same with Zaqar. It was a requirement then, * Guest agents of heat, sahara, mangum, trove, etc could all rely on it being there and wouldn't have to deal with workarounds * Horizon could rely on it being there for dynamic ui * App developers can rely on it being there. Yes, it seems like more work for an operator to have to install "one more thing", but the savings it provides by not having to do that "one more thing" in 7 different projects pays for it. Rather then focus on sharing code by adding required project dependencies, we're just implementing things in each project to workaround the lack thereof in a general project and then everyone suffers for it. Thanks, Kevin From: Hongbin Lu [hongbin...@huawei.com] Sent: Friday, July 15, 2016 10:41 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all) No, Magnum still uses Barbican as an optional dependency, and I believe nobody has proposed to remove Barbican entirely. I have no position about big tent but using Magnum as an example of "projects are not working together" is inappropriate. Best regards, Hongbin > -Original Message- > From: Fox, Kevin M [mailto:kevin@pnnl.gov] > Sent: July-15-16 2:37 PM > To: OpenStack Development Mailing List (not for usage questions) > Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins > for all) > > Some specific things: > > Magnum trying to not use Barbican as it adds an addition dependency. > See the discussion on the devel mailing list for details. > > Horizon discussions at the summit around wanting to use Zaqar for > dynamic ui updates instead of polling, but couldn't depend on a non > widely deployed subsystem. > > Each Advanced OpenStack Service implementing a guest controller > communication channel that are incompatible with each other and work > around communications issues differently. This makes a lot more pain > for Ops to debug or architect a viable solution. For example: > * Sahara uses ssh from the controllers to the vms. This does not play > well with tenant networks. They have tried to work around this several > ways: > * require every vm to have a floating ip. (Unnecessary attack > surface) > * require the controller to be on the one and only network node > (Uses ip netns exec to tunnel. Doesn't work for large clouds) > * Double tunnel ssh via the controller vm's. so some vms have fips, > some don't. Better then all, but still not good. > * Trove uses Rabbit for the guest agent to talk back to the > controllers. This has traffic going the right direction to work well > with tenant networks. > * But Rabbit is not multitenant so a security risk if any user can > get into any one of the database vm's. > Really, I believe the right solution is to use a multitenant aware > message queue so that the guest agent can pull in the right direction > for tenant networks, and not have the security risk. We have such a > system already, Zaqar, but its not widely deployed and projects don't > want to depend on other projects that aren't widely deployed. > > The lack of Instance Users has caused lots of projects to try and work > around the lack thereof. I know for sure, Mangum, Heat, and Trove work > around the lack. I'm positive others have too. As an operator, it makes > me have to very carefully consider all the tradeoffs each project made, > and decide if I can accept the same risk they assumed. Since each is > different, thats much harder. > > I'm sure there are more examples. but I hope you get I'm not just > trying to troll. > > The TC did apply inconsistant rules on letting projects in. That was > for sure a negative before the big tent. But it also provided a way to > apply pressure to projects to fix some of the issues that multiple > projects face, and that plague user/operators and raise the whole > community up, and that has fallen to the wayside since. Which is a big > negative now. Maybe that could be bolted on top of the Big Tent I don't > know. > > I could write a very long description about the state of being an > OpenStack App developer too that touches on all the
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On Mon, Jul 18, 2016 at 7:13 PM, Michael Still wrote: > On 16 Jul 2016 1:27 PM, "Thomas Herve" wrote: >> >> On Fri, Jul 15, 2016 at 8:36 PM, Fox, Kevin M wrote: >> > The lack of Instance Users has caused lots of projects to try and work >> > around the lack thereof. I know for sure, Mangum, Heat, and Trove work >> > around the lack. I'm positive others have too. As an operator, it makes me >> > have to very carefully consider all the tradeoffs each project made, and >> > decide if I can accept the same risk they assumed. Since each is different, >> > thats much harder. >> >> Instance users would be nice. Nobody just provided a good solution. I >> know you tried, but I don't think you succeeded. We need a good >> implementation, easy to understand, and maybe this will move forward. > > I think I need a more complete definition of instance users to know what > you're talking about here. Is this the instance locking stuff Bruno proposed > a while ago? I believe this is what's Kevin talks about: https://review.openstack.org/#/c/93/ -- Thomas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Kfox> Responding to this inline. Sorry, its outlook. Not my choice. :/ -Original Message- From: Thomas Herve [mailto:the...@redhat.com] Sent: Saturday, July 16, 2016 1:22 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all) On Fri, Jul 15, 2016 at 8:36 PM, Fox, Kevin M wrote: > Some specific things: > > Magnum trying to not use Barbican as it adds an addition dependency. See the > discussion on the devel mailing list for details. > > Horizon discussions at the summit around wanting to use Zaqar for dynamic ui > updates instead of polling, but couldn't depend on a non widely deployed > subsystem. > > Each Advanced OpenStack Service implementing a guest controller communication > channel that are incompatible with each other and work around communications > issues differently. This makes a lot more pain for Ops to debug or architect > a viable solution. For example: > * Sahara uses ssh from the controllers to the vms. This does not play well > with tenant networks. They have tried to work around this several ways: > * require every vm to have a floating ip. (Unnecessary attack surface) > * require the controller to be on the one and only network node (Uses ip > netns exec to tunnel. Doesn't work for large clouds) > * Double tunnel ssh via the controller vm's. so some vms have fips, some > don't. Better then all, but still not good. > * Trove uses Rabbit for the guest agent to talk back to the controllers. > This has traffic going the right direction to work well with tenant networks. > * But Rabbit is not multitenant so a security risk if any user can get > into any one of the database vm's. > Really, I believe the right solution is to use a multitenant aware message > queue so that the guest agent can pull in the right direction for tenant > networks, and not have the security risk. We have such a system already, > Zaqar, but its not widely deployed and projects don't want to depend on other > projects that aren't widely deployed. While I agree using Barbican/Zaqar for those would be fantastic, this is easily solved: just optionally depend on it. Heat provides features that use Zaqar, which are not just present when Zaqar is not there. For example, if people in the Trove project were convinced that Zaqar was the best solution to their problem, I think they would find a way to provide an implementation using it. I don't think they are, though. Whatever the reasons may be. kfox> The problem with making it optional, is it's a chicken and the egg problem. Everyone makes it optional, so they all have to implement fallback code. Operators see its optional, so they don't deploy the optional stuff as that's "more work". Users can't depend on it, since its not there on most systems. They don't tend to ask for it to be installed. Then the new project never gains any traction. It's a vicious cycle. > The lack of Instance Users has caused lots of projects to try and work around > the lack thereof. I know for sure, Mangum, Heat, and Trove work around the > lack. I'm positive others have too. As an operator, it makes me have to very > carefully consider all the tradeoffs each project made, and decide if I can > accept the same risk they assumed. Since each is different, thats much harder. Instance users would be nice. Nobody just provided a good solution. I know you tried, but I don't think you succeeded. We need a good implementation, easy to understand, and maybe this will move forward. kfox> The problem is, it's a hard thing to solve, and I believe the current organization of OpenStack provides significant barrier to solving it. I've tried for years to solve it, and have basically given up. This is part of the big tent issue. I'm moving more and more stuff that requires Instance Users to non OpenStack systems, which will be a majority of my workload at some point if things continue. This is the kind of thing OpenStack is really suffering from right now, and will only get worse as it continues to be swept under the rug. > I'm sure there are more examples. but I hope you get I'm not just trying to > troll. > > The TC did apply inconsistant rules on letting projects in. That was for sure > a negative before the big tent. But it also provided a way to apply pressure > to projects to fix some of the issues that multiple projects face, and that > plague user/operators and raise the whole community up, and that has fallen > to the wayside since. Which is a big negative now. Maybe that could be bolted > on top of the Big Tent I don't know. So I think that's the root of the issue here. You assume
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
The current incarnation of the Instance User spec: https://review.openstack.org/#/c/93/ Thanks, Kevin From: Michael Still [mi...@stillhq.com] Sent: Monday, July 18, 2016 10:13 AM To: OpenStack Development Mailing List Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all) On 16 Jul 2016 1:27 PM, "Thomas Herve" mailto:the...@redhat.com>> wrote: > > On Fri, Jul 15, 2016 at 8:36 PM, Fox, Kevin M > mailto:kevin@pnnl.gov>> wrote: > > Some specific things: > > > > Magnum trying to not use Barbican as it adds an addition dependency. See > > the discussion on the devel mailing list for details. > > > > Horizon discussions at the summit around wanting to use Zaqar for dynamic > > ui updates instead of polling, but couldn't depend on a non widely deployed > > subsystem. > > > > Each Advanced OpenStack Service implementing a guest controller > > communication channel that are incompatible with each other and work around > > communications issues differently. This makes a lot more pain for Ops to > > debug or architect a viable solution. For example: > > * Sahara uses ssh from the controllers to the vms. This does not play well > > with tenant networks. They have tried to work around this several ways: > > * require every vm to have a floating ip. (Unnecessary attack surface) > > * require the controller to be on the one and only network node (Uses > > ip netns exec to tunnel. Doesn't work for large clouds) > > * Double tunnel ssh via the controller vm's. so some vms have fips, > > some don't. Better then all, but still not good. > > * Trove uses Rabbit for the guest agent to talk back to the controllers. > > This has traffic going the right direction to work well with tenant > > networks. > > * But Rabbit is not multitenant so a security risk if any user can get > > into any one of the database vm's. > > Really, I believe the right solution is to use a multitenant aware message > > queue so that the guest agent can pull in the right direction for tenant > > networks, and not have the security risk. We have such a system already, > > Zaqar, but its not widely deployed and projects don't want to depend on > > other projects that aren't widely deployed. > > While I agree using Barbican/Zaqar for those would be fantastic, this > is easily solved: just optionally depend on it. Heat provides features > that use Zaqar, which are not just present when Zaqar is not there. > For example, if people in the Trove project were convinced that Zaqar > was the best solution to their problem, I think they would find a way > to provide an implementation using it. I don't think they are, though. > Whatever the reasons may be. > > > The lack of Instance Users has caused lots of projects to try and work > > around the lack thereof. I know for sure, Mangum, Heat, and Trove work > > around the lack. I'm positive others have too. As an operator, it makes me > > have to very carefully consider all the tradeoffs each project made, and > > decide if I can accept the same risk they assumed. Since each is different, > > thats much harder. > > Instance users would be nice. Nobody just provided a good solution. I > know you tried, but I don't think you succeeded. We need a good > implementation, easy to understand, and maybe this will move forward. I think I need a more complete definition of instance users to know what you're talking about here. Is this the instance locking stuff Bruno proposed a while ago? > > I'm sure there are more examples. but I hope you get I'm not just trying to > > troll. > > > > The TC did apply inconsistant rules on letting projects in. That was for > > sure a negative before the big tent. But it also provided a way to apply > > pressure to projects to fix some of the issues that multiple projects face, > > and that plague user/operators and raise the whole community up, and that > > has fallen to the wayside since. Which is a big negative now. Maybe that > > could be bolted on top of the Big Tent I don't know. > > So I think that's the root of the issue here. You assume we can > "pressure" people do something you think is right. But that's not how > opensource works. You either implement the solution to your problem, > or convince someone else, but you don't force anybody. I think that's > what the TC recognized and changed, to the correct path. Could project > be better integrated, so that we have a more coherent "
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Fox, Kevin M wrote: [...] I feel like I'm just the messenger here. Please don't shoot me. I'm generally trying to provide good, though hard feedback to better a perceived problem. Many others would just stay silent or leave rather then have the difficult discussion. And I think we all appreciate that :) [...] The openness of the big tent has allowed more choice in the ecosystem, which seems good, but I think it has also stalled development on the sorts of cross project issues that were important to our users and are now looking elsewhere for solutions. As others have mentioned, we are always adapting and continuously improving. Now that we completed the community-centric redefinition of OpenStack (the "big tent"), we are I think in a better position to push the sort of cross-project initiatives and normalization that you are looking for. For example, we talked recently about having the Technical Committee set "release goals" to drive simple coherence and visible improvements across all the components of the stack. -- Thierry Carrez (ttx) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
I applaud this (since I know this kind of question and work is really really hard). So thanks for starting and keeping 'hard' questions going because without someone pushing the limit here (and/or raising the questions) I too fear what u fear and that we may be obsoleting ourselves by continuing down the path we are on (death by a thousand cuts anyone?) In general I'd also like to know how cross-project things will be any different with the TC work, tags, release goals or other? I have a glass is half full feeling that those kind of things are 'to soft' or 'to late' but maybe I should just think more positive, but then I'm sort of in the camp that much *much* more aggressive action must be taken here. Thus why I think the starting of the architecture working group is a good thing; because I have a believe that people are forgetting among all of this that such a group holds a lot of the keys to the kingdom (whether u, the reader, want to admit that or not is well the readers problem) in openstack (sorry and no disrespect to independent folks & contributors), but most of us work for large companies that have architects (and leads) and if those architects (and leads) can get together cross-company to aggregate and (agree on) and solve actual problems then that really is IMHO the only way to keep our projects healthy (assuming we can even do that at this stage). Sadly I'm not sure if it's already to late for an architecture working group of which one of its goals (IMHO) is to do said analysis & aggregation of 'worker bees' to actually make fixes and solutions appear; but I'll remain hopefully on that (for now). Fox, Kevin M wrote: I feel like I'm just the messenger here. Please don't shoot me. I'm generally trying to provide good, though hard feedback to better a perceived problem. Many others would just stay silent or leave rather then have the difficult discussion. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On Jul 18, 2016, at 12:39 PM, Fox, Kevin M wrote: > I'm arguing the opposite. It should be a requirement to use the OpenStack > Secret Store. > > Instead, we're trying to make it optional and either reimplementing large > swaths of it in individual projects to make it optional, or skip security all > together. > > If it wasn't optional, then: > * Keystone could rely on it being there for password hashing > * Magnum can rely on it for security. > * lbaas can rely on it > * App developers can rely on it being there > * ... > > Same with Zaqar. It was a requirement then, > * Guest agents of heat, sahara, mangum, trove, etc could all rely on it being > there and wouldn't have to deal with workarounds > * Horizon could rely on it being there for dynamic ui > * App developers can rely on it being there. > > Yes, it seems like more work for an operator to have to install "one more > thing", but the savings it provides by not having to do that "one more thing" > in 7 different projects pays for it. There is a lot to be said for code simplification. When Glance split out of Nova, the knowledge that Glance would be required to be there meant the the code in Nova for interacting with it didn’t have to contain tons of ‘if : else ”. Knowing that that code was unnecessary made writing the interaction much simpler (and less tedious to read). So if we added, say, Barbican as a requirement, all of the other projects that need to handle secrets (a lot of them!) could simply program to the Barbican interface, and be done with it. The discussion of whether project X should be required is a whole ‘nother topic. But I want to emphasize that often the utility of having a standard is grossly underestimated. In this discussion, the issue that seems to be arising is that the Big Tent doesn’t provide for a path for a project to be reconsidered as an OpenStack standard, in the way that Nova and Glance and Swift are. The Small Tent “incubation” process designed to identify which projects were mature enough to be considered “one of us”, when “us” was the group of core projects. This was unnecessarily harsh, and pushed away many worthy efforts. In an attempt to improve this, the definition of “one of us” was broadened so that lots of interesting projects could be part of the community. What I think is causing some confusion (and friction) is that OpenStack needs two “us”s: what projects are part of our ecosystem and build software in the same open way (the Big Tent concept), and which are a fundamental component of the overall OpenStack software, which all other projects can assume are available (the Small Tent concept). To that end, it seems necessary to define a migration path for a project to become required. Of course, most will never need to be required, but I think a case could be made that several projects (Barbican and Zaqar are the obvious ones here) have matured to the point where it makes sense to standardize on them. They are mature, and no other projects have come along to challenge their designs. At some point, it makes sense to stop pretending that there are options, and establish a standard instead. -- Ed Leafe __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On Mon, Jul 18 2016, Joshua Harlow wrote: > Thus why I think the starting of the architecture working group is a good > thing; because I have a believe that people are forgetting among all of this > that such a group holds a lot of the keys to the kingdom (whether u, the > reader, want to admit that or not is well the readers problem) in openstack > (sorry and no disrespect to independent folks & contributors), but most of us > work for large companies that have architects (and leads) and if those > architects (and leads) can get together cross-company to aggregate and (agree > on) and solve actual problems then that really is IMHO the only way to keep > our > projects healthy (assuming we can even do that at this stage). I think it is a bit naive to think any working group is going to fix architectural problems. You know first hand what happened¹ with the Nova service group and tooz for example. ¹ Nothing. -- Julien Danjou // Free Software hacker // https://julien.danjou.info signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Excerpts from Ed Leafe's message of 2016-07-18 21:59:44 -0700: > On Jul 18, 2016, at 12:39 PM, Fox, Kevin M wrote: > > > I'm arguing the opposite. It should be a requirement to use the OpenStack > > Secret Store. > > > > Instead, we're trying to make it optional and either reimplementing large > > swaths of it in individual projects to make it optional, or skip security > > all together. > > > > If it wasn't optional, then: > > * Keystone could rely on it being there for password hashing > > * Magnum can rely on it for security. > > * lbaas can rely on it > > * App developers can rely on it being there > > * ... > > > > Same with Zaqar. It was a requirement then, > > * Guest agents of heat, sahara, mangum, trove, etc could all rely on it > > being there and wouldn't have to deal with workarounds > > * Horizon could rely on it being there for dynamic ui > > * App developers can rely on it being there. > > > > Yes, it seems like more work for an operator to have to install "one more > > thing", but the savings it provides by not having to do that "one more > > thing" in 7 different projects pays for it. > > There is a lot to be said for code simplification. When Glance split out of > Nova, the knowledge that Glance would be required to be there meant the the > code in Nova for interacting with it didn’t have to contain tons of ‘if > : else ”. Knowing that that > code was unnecessary made writing the interaction much simpler (and less > tedious to read). So if we added, say, Barbican as a requirement, all of the > other projects that need to handle secrets (a lot of them!) could simply > program to the Barbican interface, and be done with it. > > The discussion of whether project X should be required is a whole ‘nother > topic. But I want to emphasize that often the utility of having a standard is > grossly underestimated. > > In this discussion, the issue that seems to be arising is that the Big Tent > doesn’t provide for a path for a project to be reconsidered as an OpenStack > standard, in the way that Nova and Glance and Swift are. The Small Tent > “incubation” process designed to identify which projects were mature enough > to be considered “one of us”, when “us” was the group of core projects. This > was unnecessarily harsh, and pushed away many worthy efforts. In an attempt > to improve this, the definition of “one of us” was broadened so that lots of > interesting projects could be part of the community. This came up during the discussion of co-gating that was part of the big tent implementation, IIRC we settled on the policy that teams are encouraged to rely on the projects produced by other community teams *when they are comfortable adding co-gating to ensure continued compatibility*, but there was no rule that such relationships must be put in place because we don't want projects to depend on new, potentially unstable, services before they are ready. So, the nova and glance teams are currently comfortable co-gating, and nova can therefore depend on glance being present. Projects handling secrets would need to negotiate similar functional test co-gating relationships with the barbican team. At that point it would be safe to have barbican as a required dependency of those other projects. Doug > > What I think is causing some confusion (and friction) is that OpenStack needs > two “us”s: what projects are part of our ecosystem and build software in the > same open way (the Big Tent concept), and which are a fundamental component > of the overall OpenStack software, which all other projects can assume are > available (the Small Tent concept). > > To that end, it seems necessary to define a migration path for a project to > become required. Of course, most will never need to be required, but I think > a case could be made that several projects (Barbican and Zaqar are the > obvious ones here) have matured to the point where it makes sense to > standardize on them. They are mature, and no other projects have come along > to challenge their designs. At some point, it makes sense to stop pretending > that there are options, and establish a standard instead. > > > -- Ed Leafe > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Julien Danjou wrote: On Mon, Jul 18 2016, Joshua Harlow wrote: Thus why I think the starting of the architecture working group is a good thing; because I have a believe that people are forgetting among all of this that such a group holds a lot of the keys to the kingdom (whether u, the reader, want to admit that or not is well the readers problem) in openstack (sorry and no disrespect to independent folks& contributors), but most of us work for large companies that have architects (and leads) and if those architects (and leads) can get together cross-company to aggregate and (agree on) and solve actual problems then that really is IMHO the only way to keep our projects healthy (assuming we can even do that at this stage). I think it is a bit naive to think any working group is going to fix architectural problems. You know first hand what happened¹ with the Nova service group and tooz for example. ¹ Nothing. Totally fair, but I still 'somewhat believe' that something good may come out of this. No I don't have a lot (I can probably count them on one hand) success stories with regards to this kind of cross-project work but it's not so easy to pin-point why things happened this way (without getting into speculation/theorizing territory). __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
+1. I think there have been a few efforts around the issue of the small tent thing described below, such as the http://www.openstack.org/brand/interop/ effort. But those profiles seemed centred around standardizing the minimal common denominator parts of what's widely deployed today rather then coming up with a new standard for the pieces that should be required for a next generation iteration of the system. Without the latter, we just keep the same old stuff in the small tent, and no new stuff gets added. Thanks, Kevin From: Ed Leafe [e...@leafe.com] Sent: Monday, July 18, 2016 9:59 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all) On Jul 18, 2016, at 12:39 PM, Fox, Kevin M wrote: > I'm arguing the opposite. It should be a requirement to use the OpenStack > Secret Store. > > Instead, we're trying to make it optional and either reimplementing large > swaths of it in individual projects to make it optional, or skip security all > together. > > If it wasn't optional, then: > * Keystone could rely on it being there for password hashing > * Magnum can rely on it for security. > * lbaas can rely on it > * App developers can rely on it being there > * ... > > Same with Zaqar. It was a requirement then, > * Guest agents of heat, sahara, mangum, trove, etc could all rely on it being > there and wouldn't have to deal with workarounds > * Horizon could rely on it being there for dynamic ui > * App developers can rely on it being there. > > Yes, it seems like more work for an operator to have to install "one more > thing", but the savings it provides by not having to do that "one more thing" > in 7 different projects pays for it. There is a lot to be said for code simplification. When Glance split out of Nova, the knowledge that Glance would be required to be there meant the the code in Nova for interacting with it didn’t have to contain tons of ‘if : else ”. Knowing that that code was unnecessary made writing the interaction much simpler (and less tedious to read). So if we added, say, Barbican as a requirement, all of the other projects that need to handle secrets (a lot of them!) could simply program to the Barbican interface, and be done with it. The discussion of whether project X should be required is a whole ‘nother topic. But I want to emphasize that often the utility of having a standard is grossly underestimated. In this discussion, the issue that seems to be arising is that the Big Tent doesn’t provide for a path for a project to be reconsidered as an OpenStack standard, in the way that Nova and Glance and Swift are. The Small Tent “incubation” process designed to identify which projects were mature enough to be considered “one of us”, when “us” was the group of core projects. This was unnecessarily harsh, and pushed away many worthy efforts. In an attempt to improve this, the definition of “one of us” was broadened so that lots of interesting projects could be part of the community. What I think is causing some confusion (and friction) is that OpenStack needs two “us”s: what projects are part of our ecosystem and build software in the same open way (the Big Tent concept), and which are a fundamental component of the overall OpenStack software, which all other projects can assume are available (the Small Tent concept). To that end, it seems necessary to define a migration path for a project to become required. Of course, most will never need to be required, but I think a case could be made that several projects (Barbican and Zaqar are the obvious ones here) have matured to the point where it makes sense to standardize on them. They are mature, and no other projects have come along to challenge their designs. At some point, it makes sense to stop pretending that there are options, and establish a standard instead. -- Ed Leafe __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Excerpts from Julien Danjou's message of 2016-07-19 09:30:36 +0200: > On Mon, Jul 18 2016, Joshua Harlow wrote: > > > Thus why I think the starting of the architecture working group is a good > > thing; because I have a believe that people are forgetting among all of this > > that such a group holds a lot of the keys to the kingdom (whether u, the > > reader, want to admit that or not is well the readers problem) in openstack > > (sorry and no disrespect to independent folks & contributors), but most of > > us > > work for large companies that have architects (and leads) and if those > > architects (and leads) can get together cross-company to aggregate and > > (agree > > on) and solve actual problems then that really is IMHO the only way to keep > > our > > projects healthy (assuming we can even do that at this stage). > > I think it is a bit naive to think any working group is going to fix > architectural problems. You know first hand what happened¹ with the Nova > service group and tooz for example. > Perhaps if we form and start working together as a group, we can disect why nothing happened, build consensus on the most important thing to do next, and actually fix some architectural problems. The social structure that teams have is a huge part of the deadlock we find ourselves in with certain controversial changes. The idea is to unroll the dependency loop and start _somewhere_ rather than where a lot of these efforts die: starting _everywhere_. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On 14/07/16 16:30, Fox, Kevin M wrote: I'm going to go ahead and ask the difficult question now as the answer is relevant to the attached proposal... Should we reconsider whether the big tent is the right approach going forward? There have been some major downsides I think to the Big Tent approach, such as: * Projects not working together because of fear of adding extra dependencies Ops don't already have * Reimplementing features, badly, over and over again in different projects instead of standardizing something. * More projects being created due to politics and not technical reasons. * Less cross project communication * Greater Operator pain at trying to assemble a more loose confederation of projects into something useful. * General pushing off more and more work to Operators/Users to deal with. * Worse User experience as cross project issues aren't being addressed. * Previously incubated projects such as Nova, Swift, etc getting a disproportionate say in things as they have a greater current user base, and its increasingly hard now for new projects to gain any traction. * Much less community pressure on projects to work together to elevate everyone. Architectural decisions are now made at individual project level with little done at the OpenStack level. * etc... The thing is, all of these problems pre-date the big tent. Arguably they were even worse before because so many projects were not only not blessed by the TC as hard dependencies, but officially weren't even part of OpenStack. Say what you want about the big tent, but at least we haven't been treated to another Zaqar graduation review farce. I think your complaint here is that the big tent hasn't done enough to solve these problems. That's a fair complaint. I think what you're suggesting is missing is some sort of TC imprimatur to say that it's acceptable to take a hard dependency on a particular project, which is effectively what graduation from incubation meant previously (e.g. distributors were effectively, though not actually, obliged to package it at this point if they weren't already). Of course the only thing that can really _force_ people to adopt a project is DefCore, and that comes with a major chicken-and-egg dilemma. It's true that we've hollowed out most of the levels between just being "one of us" and being DefCore-approved. I worried about this myself at the time the big tent was proposed. But I don't think it's a silver bullet - developers of new projects have always been reluctant to take hard dependencies (source: first-hand experience starting in 2012). It's *hard* to get operators to adopt your project. To get operators to adopt your project plus some other project is... well it's definitely never easier. That said, I don't think any of the projects you mentioned elsewhere (e.g. the 4 with the own implementations of guest agents) are being deliberately intransigent. They're each just trying to find a good-enough solution in isolation. If a standard way of doing guest agents had existed before they started then I'm confident they (we) all would have used it, and if one cropped up now I hope none of them would resist efforts to at least support it as an option (i.e. with only soft dependencies). The thing that's not happening, and has never happened, is that nobody is getting those groups together and trying to come up with a common standard that everything can move toward. I think Clint's proposed architecture working group is a step in the right direction here. I don't know if such questions, which in a sense concern the user-visible architecture as much as the backend architecture, would be considered in scope (or high-priority) by that group. It's possible we might need a separate working group to address the needs of what I've been calling autonomous applications (i.e. cloud-resident applications that use the cloud APIs to control their own infrastructure). But I'm reluctant to formally propose such a thing until we have had a chance to let the architecture group pioneer the territory :) cheers, Zane. The overall goal of the Big Tent was to make the community more inclusive. That I think has mostly happened. Which is good. But It also seems to have fractured the community more into insular islands and made the system harder for our operators/users. Which is bad. Maybe the benefits outweigh the drawbacks. I'm not sure. But I think its probably time to consider if it has been a net positive and should be further refined to try and address some of these problems, or a net negative and different approaches be explored. Thanks, Kevin __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Yeah. I'm not saying any project has done it out of malice. Everyone's just doing whats best for their project. But it does not seem like there is an overarching entity watching over the whole or (pushing, encouraging, enticing, whatever words are appropriate here) projects to work on the commons anymore. It use to be that incubating projects were pushed to help the other projects integrate with them by the incubating project being strongly encouraged to write the integrations themselves as part of the incubation process. Now it seems like each project just spawns and then hopes someone else will do the legwork. Using the carrot of incubation to further the commons is not an ideal solution, but it was at least something. Linux has an overarching entity, Linus for that task. He's there to make sure that someone is really paying attention to integration of the whole thing into a cohesive, usable whole. Sometimes he pushes back and makes sure commons work happens as part of features getting in to ensure commons work still gets done. I'm not advocating a benevolent dictator for OpenStack though. But what I thought what the TC's job was, was benevolent dictators, which each subproject (or subsystem in linux terms) are required to give up final say to, so that sometimes the projects have to sacrifice a bit so that the whole can flourish and those benevolent dictators are elected for a time, by the OpenStack community. (Actually, I think that kind of makes it a Democratic Republic... but I digress) Maybe I misunderstood what the TC's about. But I think we still do need some folks elected by the community to be involved in making sure OpenStack as a whole has a cohesive technical architecture that actually addresses user problems and that have some power to to stop the "this feature belongs in this project", "no, it belongs in that project", "no, lets spawn 3 new projects to cover that case" kinds of things, make the difficult decision, and ask a project to help the community out by going along with "a solution" and we all can move on. Critical features have been stuck in this space for years and OpenStacks competitors have had solutions for years. Interest in OpenStack as a whole has leveled off in about: From: Zane Bitter [zbit...@redhat.com] Sent: Tuesday, July 19, 2016 1:08 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all) On 14/07/16 16:30, Fox, Kevin M wrote: > I'm going to go ahead and ask the difficult question now as the answer is > relevant to the attached proposal... > > Should we reconsider whether the big tent is the right approach going forward? > > There have been some major downsides I think to the Big Tent approach, such > as: > * Projects not working together because of fear of adding extra dependencies > Ops don't already have > * Reimplementing features, badly, over and over again in different projects > instead of standardizing something. > * More projects being created due to politics and not technical reasons. > * Less cross project communication > * Greater Operator pain at trying to assemble a more loose confederation of > projects into something useful. > * General pushing off more and more work to Operators/Users to deal with. > * Worse User experience as cross project issues aren't being addressed. > * Previously incubated projects such as Nova, Swift, etc getting a > disproportionate say in things as they have a greater current user base, and > its increasingly hard now for new projects to gain any traction. > * Much less community pressure on projects to work together to elevate > everyone. Architectural decisions are now made at individual project level > with little done at the OpenStack level. > * etc... The thing is, all of these problems pre-date the big tent. Arguably they were even worse before because so many projects were not only not blessed by the TC as hard dependencies, but officially weren't even part of OpenStack. Say what you want about the big tent, but at least we haven't been treated to another Zaqar graduation review farce. I think your complaint here is that the big tent hasn't done enough to solve these problems. That's a fair complaint. I think what you're suggesting is missing is some sort of TC imprimatur to say that it's acceptable to take a hard dependency on a particular project, which is effectively what graduation from incubation meant previously (e.g. distributors were effectively, though not actually, obliged to package it at this point if they weren't already). Of course the only thing that can really _force_ people to adopt a project is DefCore, and that comes with a major chicken-and-egg dilemma.
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Yeah. I'm not saying any project has done it out of malice. Everyone's just doing whats best for their project. But it does not seem like there is an overarching entity watching over the whole or (pushing, encouraging, enticing, whatever words are appropriate here) projects to work on the commons anymore. It use to be that incubating projects were pushed to help the other projects integrate with them by the incubating project being strongly encouraged to write the integrations themselves as part of the incubation process. Now it seems like each project just spawns and then hopes someone else will do the legwork. Using the carrot of incubation to further the commons is not an ideal solution, but it was at least something. Linux has an overarching entity, Linus for that task. He's there to make sure that someone is really paying attention to integration of the whole thing into a cohesive, usable whole. Sometimes he pushes back and makes sure commons work happens as part of features getting in to ensure commons work still gets done. I'm not advocating a benevolent dictator for OpenStack though. But what I thought what the TC's job was, was benevolent dictators, which each subproject (or subsystem in linux terms) are required to give up final say to, so that sometimes the projects have to sacrifice a bit so that the whole can flourish and those benevolent dictators are elected for a time, by the OpenStack community. (Actually, I think that kind of makes it a Democratic Republic... but I digress) Maybe I misunderstood what the TC's about. But I think we still do need some folks elected by the community to be involved in making sure OpenStack as a whole has a cohesive technical architecture that actually addresses user problems and that have some power to to stop the "this feature belongs in this project", "no, it belongs in that project", "no, lets spawn 3 new projects to cover that case" kinds of things, make the difficult decision, and ask a project to help the community out by going along with "a solution" and we all can move on. Critical features have been stuck in this space for years and OpenStacks competitors have had solutions for years. Interest in OpenStack as a whole has leveled off in about 2014, while interest in other clouds have continued to grow: https://www.google.com/trends/explore#q=%2Fm%2F0cm87w_%2C%20%2Fm%2F05nrgx%2C%20%2Fm%2F04y7lrx&cmpt=q&tz=Etc%2FGMT%2B7 If the trend of OpenStack as a whole is not rising, we need to figure out why and get the whole moving again. I think thats mostly the projects keeping their heads down and working on what they work on, and not much focus on the commons, or helping OpenStack as a whole get better, other then what just happens as a side effect of each project gaining features. We need to figure out how to fix this. Thanks, Kevin From: Zane Bitter [zbit...@redhat.com] Sent: Tuesday, July 19, 2016 1:08 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all) On 14/07/16 16:30, Fox, Kevin M wrote: > I'm going to go ahead and ask the difficult question now as the answer is > relevant to the attached proposal... > > Should we reconsider whether the big tent is the right approach going forward? > > There have been some major downsides I think to the Big Tent approach, such > as: > * Projects not working together because of fear of adding extra dependencies > Ops don't already have > * Reimplementing features, badly, over and over again in different projects > instead of standardizing something. > * More projects being created due to politics and not technical reasons. > * Less cross project communication > * Greater Operator pain at trying to assemble a more loose confederation of > projects into something useful. > * General pushing off more and more work to Operators/Users to deal with. > * Worse User experience as cross project issues aren't being addressed. > * Previously incubated projects such as Nova, Swift, etc getting a > disproportionate say in things as they have a greater current user base, and > its increasingly hard now for new projects to gain any traction. > * Much less community pressure on projects to work together to elevate > everyone. Architectural decisions are now made at individual project level > with little done at the OpenStack level. > * etc... The thing is, all of these problems pre-date the big tent. Arguably they were even worse before because so many projects were not only not blessed by the TC as hard dependencies, but officially weren't even part of OpenStack. Say what you want about the big tent, but at least we haven't been treated to another Zaqar graduation review farce. I think your compl
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Excerpts from Fox, Kevin M's message of 2016-07-19 21:59:29 +: > Yeah. I'm not saying any project has done it out of malice. Everyone's just > doing whats best for their project. But it does not seem like there is an > overarching entity watching over the whole or (pushing, encouraging, > enticing, whatever words are appropriate here) projects to work on the > commons anymore. It use to be that incubating projects were pushed to help > the other projects integrate with them by the incubating project being > strongly encouraged to write the integrations themselves as part of the > incubation process. Now it seems like each project just spawns and then hopes > someone else will do the legwork. Using the carrot of incubation to further > the commons is not an ideal solution, but it was at least something. > > Linux has an overarching entity, Linus for that task. He's there to make sure > that someone is really paying attention to integration of the whole thing > into a cohesive, usable whole. Sometimes he pushes back and makes sure > commons work happens as part of features getting in to ensure commons work > still gets done. I'm not advocating a benevolent dictator for OpenStack > though. > > But what I thought what the TC's job was, was benevolent dictators, which > each subproject (or subsystem in linux terms) are required to give up final > say to, so that sometimes the projects have to sacrifice a bit so that the > whole can flourish and those benevolent dictators are elected for a time, by > the OpenStack community. (Actually, I think that kind of makes it a > Democratic Republic... but I digress) Maybe I misunderstood what the TC's > about. But I think we still do need some folks elected by the community to be > involved in making sure OpenStack as a whole has a cohesive technical > architecture that actually addresses user problems and that have some power > to to stop the "this feature belongs in this project", "no, it belongs in > that project", "no, lets spawn 3 new projects to cover that case" kinds of > things, make the difficult decision, and ask a project to help the community > out by going along with "a solution" and we all can move on. Critical > features have been stuck > in this space for years and OpenStacks competitors have had solutions for > years. You're right, this is the TC's job. However, the TC does it more by exception, rather than by default. So while Linus and the subsystem leaders in the kernel look after changes in general, the TC is there to catch things that bubble out of the processes in place. So, I think the TC needs contributors to bring _specific_ things that need to be handled, and they will. They're just not going to be able to stand at the gate and review every spec... this process only scales to the velocity and breadth that OpenStack has if we get contributors involved. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Clint Byrum wrote: [...] But what I thought what the TC's job was, was benevolent dictators, which each subproject (or subsystem in linux terms) are required to give up final say to, so that sometimes the projects have to sacrifice a bit so that the whole can flourish and those benevolent dictators are elected for a time, by the OpenStack community. (Actually, I think that kind of makes it a Democratic Republic... but I digress) Maybe I misunderstood what the TC's about. But I think we still do need some folks elected by the community to be involved in making sure OpenStack as a whole has a cohesive technical architecture that actually addresses user problems and that have some power to to stop the "this feature belongs in this project", "no, it belongs in that project", "no, lets spawn 3 new projects to cover that case" kinds of things, make the difficult decision, and ask a project to help the community out by going along with "a solution" and we all can move on. Critical features have been stu ck in this space for years and OpenStacks competitors have had solutions for years. You're right, this is the TC's job. However, the TC does it more by exception, rather than by default. So while Linus and the subsystem leaders in the kernel look after changes in general, the TC is there to catch things that bubble out of the processes in place. So, I think the TC needs contributors to bring _specific_ things that need to be handled, and they will. They're just not going to be able to stand at the gate and review every spec... this process only scales to the velocity and breadth that OpenStack has if we get contributors involved. It's always a balancing act -- we want the TC to keep an eye on the big picture, provide guidance, encourage convergence, and stop the bucket when needed. At the same time, developers don't want a set of people that do not have specific experience in a their project to constantly interfere with project development with random mandates. However, I'd agree that the pendulum currently is still in the "not enough" territory rather than in the "too much" territory. We have a number of initiatives brewing though (clarifying principles, defining release goals, release stewards, architecture WG, PTG event...) which I think will improve things in the right direction, let's see how those pan out. Please be patient while we are rolling those out. So in summary: yes there still are issues, but it is not a simple problem, and we are working on it. -- Thierry Carrez (ttx) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On Tue, Jul 19 2016, Clint Byrum wrote: > Perhaps if we form and start working together as a group, we can disect > why nothing happened, build consensus on the most important thing to do > next, and actually fix some architectural problems. The social structure > that teams have is a huge part of the deadlock we find ourselves in > with certain controversial changes. The idea is to unroll the dependency > loop and start _somewhere_ rather than where a lot of these efforts die: > starting _everywhere_. I agree with your analysis, but I fail to see how e.g. a group of people X stating that Y should work this way in Cinder is going to achieve any change if nobody from Cinder is in X from the beginning. This is basically what seems to happen in many [working] groups as far as I can see. -- Julien Danjou ;; Free Software hacker ;; https://julien.danjou.info signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On Wed, 2016-07-20 at 11:58 +0200, Julien Danjou wrote: > On Tue, Jul 19 2016, Clint Byrum wrote: > > > Perhaps if we form and start working together as a group, we can > > disect why nothing happened, build consensus on the most important > > thing to do next, and actually fix some architectural problems. Architecture is often blamed for lack of interlock but most people forget that the need for interlock often isn't appreciated until after the components are built. This is why a lot of people embrace agile methodology (an excuse for not seeing the problem a priori). Conversely, architects who try to foresee all interlocks often end up with a combinatorial explosion that makes it a huge burden simply to begin the project (and then often get sidelined as 'powerpoint engineers'). The trick is to do something in the middle: try to foresee and build the most common interlocks, but sidestep the combinatorial explosion by building something simple enough to adapt to any interlock requirement that arises after completion. > > The social structure that teams have is a huge part of the > > deadlock we find ourselves in with certain controversial changes. > > The idea is to unroll the dependency loop and start _somewhere_ > > rather than where a lot of these efforts die: starting > > _everywhere_. > > I agree with your analysis, but I fail to see how e.g. a group of > people X stating that Y should work this way in Cinder is going to > achieve any change if nobody from Cinder is in X from the beginning. > > This is basically what seems to happen in many [working] groups as > far as I can see. So this is where the Open Source method takes over. Change is produced by those people who most care about it because they're invested. To take your Cinder example, you're unlikely to find them within Cinder because any project has inertia that resists change. It takes the energy of the outside group X to force the change to Y, but this means that X often gets to propose, develop and even code Y. Essentially they become drive by coders for Cinder. This is where Open Source differs from Enterprise because you have the code and the access, you can do this. However, you have to remember the inertia problem and build what you're trying to do as incrementally as possible: the larger the change, the bigger the resistance to it. It's also a good test of the value of the change: if group X can't really be bothered to code it (and Cinder doesn't want it) then perhaps there's not really enough value in it anyway and it shouldn't happen. This latter observation is also an improvement over the enterprise methods because enterprise architects do often mandate interlocks that later turn out to be unnecessary (or at least of a lot less value than was initially thought). I suppose the executive summary of the above is that I don't think you're describing a bug, I think you're describing a feature. James signature.asc Description: This is a digitally signed message part __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
+1 to the finding of a middle ground. The problem I've seen with your suggested OpenSource solution is the current social monetary system of OpenStack makes it extremely difficult. Each project currently prints its own currency. Reviews. It takes quite a few Reviews (time/effort) on a project to gain enough currency that you get payed attention to. And, one project doesn't honor another projects currency. When these sorts of major cross project things need to be done though, you need to have folks with capital in many different projects and its very difficult to amass that much. There is no OpenStack level currency (other then being elected as a TC member) that gets one project to stop and take the time to carefully consider what someone is saying when bringing up cross project issues. Without some shared currency, then the problem becomes, the person invested in getting a solution, can propose and prototype and implement the feature all they want (often several times), but it never actually is accepted into the projects because a project: a) doesn't take the time to really understand the problem, feeling its trivial and just not accepting any reviews b) doesn't take the time to really understand the problem, so minimize it in their mind to a 'you should just use existing thing X...' or a heavy handily propose alternate solutions that really aren't solutions. c) they decide its better handled by another project and stall/block reviews, trying to push the implementation to go elsewhere. When multiple projects decide this, the ball just keeps getting bounced around without any solution for years. The status quo is not working here. The current governance structure doesn't support progress. Thanks, Kevin From: James Bottomley [james.bottom...@hansenpartnership.com] Sent: Wednesday, July 20, 2016 8:31 AM To: OpenStack Development Mailing List (not for usage questions); Clint Byrum Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all) On Wed, 2016-07-20 at 11:58 +0200, Julien Danjou wrote: > On Tue, Jul 19 2016, Clint Byrum wrote: > > > Perhaps if we form and start working together as a group, we can > > disect why nothing happened, build consensus on the most important > > thing to do next, and actually fix some architectural problems. Architecture is often blamed for lack of interlock but most people forget that the need for interlock often isn't appreciated until after the components are built. This is why a lot of people embrace agile methodology (an excuse for not seeing the problem a priori). Conversely, architects who try to foresee all interlocks often end up with a combinatorial explosion that makes it a huge burden simply to begin the project (and then often get sidelined as 'powerpoint engineers'). The trick is to do something in the middle: try to foresee and build the most common interlocks, but sidestep the combinatorial explosion by building something simple enough to adapt to any interlock requirement that arises after completion. > > The social structure that teams have is a huge part of the > > deadlock we find ourselves in with certain controversial changes. > > The idea is to unroll the dependency loop and start _somewhere_ > > rather than where a lot of these efforts die: starting > > _everywhere_. > > I agree with your analysis, but I fail to see how e.g. a group of > people X stating that Y should work this way in Cinder is going to > achieve any change if nobody from Cinder is in X from the beginning. > > This is basically what seems to happen in many [working] groups as > far as I can see. So this is where the Open Source method takes over. Change is produced by those people who most care about it because they're invested. To take your Cinder example, you're unlikely to find them within Cinder because any project has inertia that resists change. It takes the energy of the outside group X to force the change to Y, but this means that X often gets to propose, develop and even code Y. Essentially they become drive by coders for Cinder. This is where Open Source differs from Enterprise because you have the code and the access, you can do this. However, you have to remember the inertia problem and build what you're trying to do as incrementally as possible: the larger the change, the bigger the resistance to it. It's also a good test of the value of the change: if group X can't really be bothered to code it (and Cinder doesn't want it) then perhaps there's not really enough value in it anyway and it shouldn't happen. This latter observation is also an improvement over the enterprise methods because enterprise architects do often mandate interlocks that later turn out to be unnecessary (or at least of a lot less v
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On Wed, 2016-07-20 at 16:08 +, Fox, Kevin M wrote: > +1 to the finding of a middle ground. Thanks ... I have actually been an enterprise architect (I just keep very quiet about it when talking Open Source). > The problem I've seen with your suggested OpenSource solution is the > current social monetary system of OpenStack makes it extremely > difficult. > > Each project currently prints its own currency. Reviews. It takes > quite a few Reviews (time/effort) on a project to gain enough > currency that you get payed attention to. And, one project doesn't > honor another projects currency. OK, I accept your analogy, even though I would view currency as the will to create and push patches. The problem you describe: getting the recipients to listen and accept your patches, is also a common one. The first essential is simple minimal patches because they're hard to reject. Once you've overcome the reject barrier, there's the indifference one (no-one says no, but no-one says yes). Overcoming indifference involves partly knowing who to bother and when (In openstack, it's quite easy since you know who the core reviewers are) and also building a consensus for the patch; usually this involves finding other people who need the feature and getting them to pipe up (if you can't find other projects, then you can get potential users to do this) even better if they help you write the patches. Ideally, you build your consensus before you actually push the patch set. Sometimes building consensus involves looking beyond your particular need to a bigger one that would satisfy you but also pulls other people in. > When these sorts of major cross project things need to be done > though, you need to have folks with capital in many different > projects and its very difficult to amass that much. > > There is no OpenStack level currency (other then being elected as a > TC member) that gets one project to stop and take the time to > carefully consider what someone is saying when bringing up cross > project issues. > > Without some shared currency, then the problem becomes, the person > invested in getting a solution, can propose and prototype and > implement the feature all they want (often several times), but it > never actually is accepted into the projects because a project: > a) doesn't take the time to really understand the problem, feeling > its trivial and just not accepting any reviews > b) doesn't take the time to really understand the problem, so > minimize it in their mind to a 'you should just use existing thing > X...' or a heavy handily propose alternate solutions that really > aren't solutions. > c) they decide its better handled by another project and stall/block > reviews, trying to push the implementation to go elsewhere. When > multiple projects decide this, the ball just keeps getting bounced > around without any solution for years. > > The status quo is not working here. The current governance structure > doesn't support progress. What you'll find you've described above is a process that doesn't support drive by coders at all and, by extension therefore, doesn't welcome newcomers, which is a big problem, but one I thought OpenStack was tackling? The bounce problem is annoying but not insuperable. It mostly occurs where there's overlap. Often the best method for coping is to field the bounce: produce the patch for the other project. If it's smaller and neater, perhaps the bounce was correct. If it's bigger and uglier, get the other project to say so and you now have a solid reason to go back and say "we tried what you suggested and here's why it didn't work". Morally, you're now on higher ground because incorrect technical advice is a personal failure for whoever bounced you (make sure to capitalise on it if they try another bounce). James __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Excerpts from James Bottomley's message of 2016-07-20 08:31:34 -0700: > So this is where the Open Source method takes over. Change is produced > by those people who most care about it because they're invested. To > take your Cinder example, you're unlikely to find them within Cinder > because any project has inertia that resists change. It takes the > energy of the outside group X to force the change to Y, but this means > that X often gets to propose, develop and even code Y. Essentially > they become drive by coders for Cinder. This is where Open Source > differs from Enterprise because you have the code and the access, you > can do this. However, you have to remember the inertia problem and > build what you're trying to do as incrementally as possible: the larger > the change, the bigger the resistance to it. It's also a good test of > the value of the change: if group X can't really be bothered to code it > (and Cinder doesn't want it) then perhaps there's not really enough > value in it anyway and it shouldn't happen. > Thanks so much for stating this James. I couldn't agree more. A group that can actually _accomplish_ change, and not just suggest it, is exactly what we're working to start with the architecture WG. There are plenty of people with the will to change, and I feel strongly that if those people are given a structure and support, then they'll find the time and space to complete these objectives. I just want to make one nuance point about Cinder changes: the recent DLM work, done outside any architecture working group, did actually come from both inside and outside Cinder. The Cinder team realized something was happening that would perhaps affect everyone, and raised it to the cross-project level, which helped external individuals get involved. So these initiatives can come from either direction. The key is that enough momentum builds up to counter the natural inertia that you mentioned in your message. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
I wish it was so simple. Its not. There is a good coding practice: "The code is done, not when there is nothing more to add, but nothing more to remove" Some of the more mature projects are in this mode of thinking now. (which is mostly good, really). They don't want to add features unless they see it as a benefit to their users. This is the problem, there is a disconnect between the view of Project X's users, and greater view of OpenStack Users. Even accepting the smallest of patches to work towards the goal is unacceptable to projects if they believe it is not a helpful feature to their perceived user base, or helpful enough to them to justify adding more code to their project. Or the feeling that "just push it to a new project or a different one is better". This fractured view of OpenStack users at the project level is preventing progress on OpenStack as a whole. Only an overarching Architectural group with some power to define what a user is, or the TC can address those sorts of issues. Thanks, Kevin From: James Bottomley [james.bottom...@hansenpartnership.com] Sent: Wednesday, July 20, 2016 9:57 AM To: OpenStack Development Mailing List (not for usage questions); Clint Byrum Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all) On Wed, 2016-07-20 at 16:08 +, Fox, Kevin M wrote: > +1 to the finding of a middle ground. Thanks ... I have actually been an enterprise architect (I just keep very quiet about it when talking Open Source). > The problem I've seen with your suggested OpenSource solution is the > current social monetary system of OpenStack makes it extremely > difficult. > > Each project currently prints its own currency. Reviews. It takes > quite a few Reviews (time/effort) on a project to gain enough > currency that you get payed attention to. And, one project doesn't > honor another projects currency. OK, I accept your analogy, even though I would view currency as the will to create and push patches. The problem you describe: getting the recipients to listen and accept your patches, is also a common one. The first essential is simple minimal patches because they're hard to reject. Once you've overcome the reject barrier, there's the indifference one (no-one says no, but no-one says yes). Overcoming indifference involves partly knowing who to bother and when (In openstack, it's quite easy since you know who the core reviewers are) and also building a consensus for the patch; usually this involves finding other people who need the feature and getting them to pipe up (if you can't find other projects, then you can get potential users to do this) even better if they help you write the patches. Ideally, you build your consensus before you actually push the patch set. Sometimes building consensus involves looking beyond your particular need to a bigger one that would satisfy you but also pulls other people in. > When these sorts of major cross project things need to be done > though, you need to have folks with capital in many different > projects and its very difficult to amass that much. > > There is no OpenStack level currency (other then being elected as a > TC member) that gets one project to stop and take the time to > carefully consider what someone is saying when bringing up cross > project issues. > > Without some shared currency, then the problem becomes, the person > invested in getting a solution, can propose and prototype and > implement the feature all they want (often several times), but it > never actually is accepted into the projects because a project: > a) doesn't take the time to really understand the problem, feeling > its trivial and just not accepting any reviews > b) doesn't take the time to really understand the problem, so > minimize it in their mind to a 'you should just use existing thing > X...' or a heavy handily propose alternate solutions that really > aren't solutions. > c) they decide its better handled by another project and stall/block > reviews, trying to push the implementation to go elsewhere. When > multiple projects decide this, the ball just keeps getting bounced > around without any solution for years. > > The status quo is not working here. The current governance structure > doesn't support progress. What you'll find you've described above is a process that doesn't support drive by coders at all and, by extension therefore, doesn't welcome newcomers, which is a big problem, but one I thought OpenStack was tackling? The bounce problem is annoying but not insuperable. It mostly occurs where there's overlap. Often the best method for coping is to field the bounce: produce the patch for the other project. If it's smaller and neate
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On 20 July 2016 at 19:57, James Bottomley < james.bottom...@hansenpartnership.com> wrote: > > OK, I accept your analogy, even though I would view currency as the > will to create and push patches. > > The problem you describe: getting the recipients to listen and accept > your patches, is also a common one. The first essential is simple > minimal patches because they're hard to reject. > > Once you've overcome the reject barrier, there's the indifference one > (no-one says no, but no-one says yes). > > [snip] The trouble with drive-by architecture patches (or large feature patches of any kind) is that it is often better *not* to merge them if you don't think the contributor is going to stick around for a while. This changes are usually intrusive, and have repercussions that take time to discover. It's often difficult to keep a change clean when the original author isn't around to review the follow-on work. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Duncan Thomas wrote: On 20 July 2016 at 19:57, James Bottomley mailto:james.bottom...@hansenpartnership.com>> wrote: OK, I accept your analogy, even though I would view currency as the will to create and push patches. The problem you describe: getting the recipients to listen and accept your patches, is also a common one. The first essential is simple minimal patches because they're hard to reject. Once you've overcome the reject barrier, there's the indifference one (no-one says no, but no-one says yes). [snip] The trouble with drive-by architecture patches (or large feature patches of any kind) is that it is often better *not* to merge them if you don't think the contributor is going to stick around for a while. This changes are usually intrusive, and have repercussions that take time to discover. It's often difficult to keep a change clean when the original author isn't around to review the follow-on work. Agreed, and knowing where yahoo and HP(e) are at right now (with regards to openstack and ...) these kind of things are a little more prevalent (with regards to quota, tasks...) now-a-days (for better or worse); not how I want it to be but it's reality. Which I guess is why I'd be nice to have cross-project architecture 'standardization' (? for lack of better word) with a more long term strategic vision (vs localized tactical visions). Such things are obviously not hard to get going (and are equally hard to sustain). __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On Wed, 2016-07-20 at 21:24 +0300, Duncan Thomas wrote: > On 20 July 2016 at 19:57, James Bottomley < > james.bottom...@hansenpartnership.com> wrote: > > > > > OK, I accept your analogy, even though I would view currency as the > > will to create and push patches. > > > > The problem you describe: getting the recipients to listen and > > accept > > your patches, is also a common one. The first essential is simple > > minimal patches because they're hard to reject. > > > > Once you've overcome the reject barrier, there's the indifference > > one > > (no-one says no, but no-one says yes). > > > > [snip] > > The trouble with drive-by architecture patches (or large feature > patches of any kind) OK, there's an assumption here: that the patch is large. > is that it is often better *not* to merge them if you don't think > the contributor is going to stick around for a while. This changes > are usually intrusive, and have repercussions that take time to > discover. It's often difficult to keep a change clean when the > original author isn't around to review the follow-on work. I understand (and do agree with) the Maintenance problem. However, if you're trying to change architecture, and you do it by small patches, it looks easily reversible and is a lot easier to gain acceptance for. The key is to think small not big (the latter being the temptation for architecture) even if the end result will eventually be big. Let me give you an example from ancient history. The biggest architectural change I pushed into Linux was switching from bus specific to generic device APIs. I did it at the time because I had a SCSI device in the PA-RISC architecture that could sit on a variety of internal busses and I didn't want a huge driver with five sets of partially duplicated code for five different busses. Here's the first patch (both dated 21 Dec 2002) generic device DMA API 27 files changed, 750 insertions(+), 155 deletions(-) And the second allow pci primary busses to have parents in the device model 2 files changed, 14 insertions(+), 5 deletions(-) The first is large because I'm actually introducing yet another parallel DMA API (50% of it is documentation). It looks OK because all our other DMA APIs were bus specific. The second is the key change because it allows cascades of bus hierarchies of different types. These two patches are easy to gain acceptance for because they're just introducing new APIs and not altering existing stuff. They're small and easily revertible if I happened to disappear (or they proved not to work out in practice). They were, however, enough to allow me to convert the PA-RISC architecture to the new model and write the driver I wanted for the 53c700 chip. The end point we're at today, where practically every device API in Linux uses generic devices, is no longer revertible. If it were done as one patch, it would touch about 80% of all the files in Linux and be larger than the kernel itself. However, I didn't do that: I pushed the minimum viable patch set that would support what I wanted to do (plus the use cases for a couple of other interested parties I picked up along the way). James __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On Wed, 2016-07-20 at 18:18 +, Fox, Kevin M wrote: > I wish it was so simple. Its not. > > There is a good coding practice: > "The code is done, not when there is nothing more to add, but nothing > more to remove" > > Some of the more mature projects are in this mode of thinking now. > (which is mostly good, really). They don't want to add features > unless they see it as a benefit to their users. This is the problem, > there is a disconnect between the view of Project X's users, and > greater view of OpenStack Users. > > Even accepting the smallest of patches to work towards the goal is > unacceptable to projects if they believe it is not a helpful feature > to their perceived user base, or helpful enough to them to justify > adding more code to their project. Or the feeling that "just push it > to a new project or a different one is better". This fractured view > of OpenStack users at the project level is preventing progress on > OpenStack as a whole. > > Only an overarching Architectural group with some power to define > what a user is, or the TC can address those sorts of issues. I'll concede this requirement if you can point out to me who this group is for the Linux Kernel. If you're tempted to say "Linus", it's most certainly not: while he does care about some architectural decisions, he emphatically avoids most of them, which leaves the subsystem maintainers (some equivalence to openstack projects/PTLs) doing this on a case by case basis. James > Thanks, > Kevin > > From: James Bottomley [james.bottom...@hansenpartnership.com] > Sent: Wednesday, July 20, 2016 9:57 AM > To: OpenStack Development Mailing List (not for usage questions); > Clint Byrum > Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins > for all) > > On Wed, 2016-07-20 at 16:08 +, Fox, Kevin M wrote: > > +1 to the finding of a middle ground. > > Thanks ... I have actually been an enterprise architect (I just keep > very quiet about it when talking Open Source). > > > The problem I've seen with your suggested OpenSource solution is > > the > > current social monetary system of OpenStack makes it extremely > > difficult. > > > > Each project currently prints its own currency. Reviews. It takes > > quite a few Reviews (time/effort) on a project to gain enough > > currency that you get payed attention to. And, one project doesn't > > honor another projects currency. > > OK, I accept your analogy, even though I would view currency as the > will to create and push patches. > > The problem you describe: getting the recipients to listen and accept > your patches, is also a common one. The first essential is simple > minimal patches because they're hard to reject. > > Once you've overcome the reject barrier, there's the indifference one > (no-one says no, but no-one says yes). > > Overcoming indifference involves partly knowing who to bother and > when > (In openstack, it's quite easy since you know who the core reviewers > are) and also building a consensus for the patch; usually this > involves > finding other people who need the feature and getting them to pipe up > (if you can't find other projects, then you can get potential users > to > do this) even better if they help you write the patches. Ideally, > you > build your consensus before you actually push the patch set. > Sometimes > building consensus involves looking beyond your particular need to a > bigger one that would satisfy you but also pulls other people in. > > > When these sorts of major cross project things need to be done > > though, you need to have folks with capital in many different > > projects and its very difficult to amass that much. > > > > There is no OpenStack level currency (other then being elected as a > > TC member) that gets one project to stop and take the time to > > carefully consider what someone is saying when bringing up cross > > project issues. > > > > Without some shared currency, then the problem becomes, the person > > invested in getting a solution, can propose and prototype and > > implement the feature all they want (often several times), but it > > never actually is accepted into the projects because a project: > > a) doesn't take the time to really understand the problem, feeling > > its trivial and just not accepting any reviews > > b) doesn't take the time to really understand the problem, so > > minimize it in their mind to a 'you should just use existing thing > > X...' or a heavy hand
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
I would argue Linus, yes. As he's constantly stepping up when a subsistem tries and breaks something for a user or creates a user facing mess and says, no, subsystem, no. breaking userspace is unacceptable, or, we're not adding support for an api we have to support forever thats very poorly designed. Yes, he defers a lot to subsystem maintainers, as they have generally gotten the message of paying close attention to that kind of thing over time, and he hasn't had to speak up so much anymore. The rest really is best left up to the subsystems. But someone has to keep an eye on the big picture. The users of the whole thing. Users care about the linux kernel as a whole, and less so about any given subsystem. Thanks, Kevin From: James Bottomley [james.bottom...@hansenpartnership.com] Sent: Wednesday, July 20, 2016 12:42 PM To: OpenStack Development Mailing List (not for usage questions); Clint Byrum Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all) On Wed, 2016-07-20 at 18:18 +, Fox, Kevin M wrote: > I wish it was so simple. Its not. > > There is a good coding practice: > "The code is done, not when there is nothing more to add, but nothing > more to remove" > > Some of the more mature projects are in this mode of thinking now. > (which is mostly good, really). They don't want to add features > unless they see it as a benefit to their users. This is the problem, > there is a disconnect between the view of Project X's users, and > greater view of OpenStack Users. > > Even accepting the smallest of patches to work towards the goal is > unacceptable to projects if they believe it is not a helpful feature > to their perceived user base, or helpful enough to them to justify > adding more code to their project. Or the feeling that "just push it > to a new project or a different one is better". This fractured view > of OpenStack users at the project level is preventing progress on > OpenStack as a whole. > > Only an overarching Architectural group with some power to define > what a user is, or the TC can address those sorts of issues. I'll concede this requirement if you can point out to me who this group is for the Linux Kernel. If you're tempted to say "Linus", it's most certainly not: while he does care about some architectural decisions, he emphatically avoids most of them, which leaves the subsystem maintainers (some equivalence to openstack projects/PTLs) doing this on a case by case basis. James > Thanks, > Kevin > > From: James Bottomley [james.bottom...@hansenpartnership.com] > Sent: Wednesday, July 20, 2016 9:57 AM > To: OpenStack Development Mailing List (not for usage questions); > Clint Byrum > Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins > for all) > > On Wed, 2016-07-20 at 16:08 +, Fox, Kevin M wrote: > > +1 to the finding of a middle ground. > > Thanks ... I have actually been an enterprise architect (I just keep > very quiet about it when talking Open Source). > > > The problem I've seen with your suggested OpenSource solution is > > the > > current social monetary system of OpenStack makes it extremely > > difficult. > > > > Each project currently prints its own currency. Reviews. It takes > > quite a few Reviews (time/effort) on a project to gain enough > > currency that you get payed attention to. And, one project doesn't > > honor another projects currency. > > OK, I accept your analogy, even though I would view currency as the > will to create and push patches. > > The problem you describe: getting the recipients to listen and accept > your patches, is also a common one. The first essential is simple > minimal patches because they're hard to reject. > > Once you've overcome the reject barrier, there's the indifference one > (no-one says no, but no-one says yes). > > Overcoming indifference involves partly knowing who to bother and > when > (In openstack, it's quite easy since you know who the core reviewers > are) and also building a consensus for the patch; usually this > involves > finding other people who need the feature and getting them to pipe up > (if you can't find other projects, then you can get potential users > to > do this) even better if they help you write the patches. Ideally, > you > build your consensus before you actually push the patch set. > Sometimes > building consensus involves looking beyond your particular need to a > bigger one that would satisfy you but also pulls other people in. > > > When these sorts of major cross project things need to be done > >
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
And maybe this raises an interesting defininition mismatch in the conversation. There is archetectural stuff like, do we support 7 different web frameworks, or do we standardize on flask... python vs go. Theres also the architectural stuff at the, what interactive surface do you expose to users/operators. How consistant is it. Does it have all the features, no matter where they are implemented to do work. They are somewhat related at the op level when debugging is involved. But Im much more concerned with the latter archetecture getting attention then the former. Thanks, Kevin From: James Bottomley Sent: Wednesday, July 20, 2016 12:42:27 PM To: OpenStack Development Mailing List (not for usage questions); Clint Byrum Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all) On Wed, 2016-07-20 at 18:18 +, Fox, Kevin M wrote: > I wish it was so simple. Its not. > > There is a good coding practice: > "The code is done, not when there is nothing more to add, but nothing > more to remove" > > Some of the more mature projects are in this mode of thinking now. > (which is mostly good, really). They don't want to add features > unless they see it as a benefit to their users. This is the problem, > there is a disconnect between the view of Project X's users, and > greater view of OpenStack Users. > > Even accepting the smallest of patches to work towards the goal is > unacceptable to projects if they believe it is not a helpful feature > to their perceived user base, or helpful enough to them to justify > adding more code to their project. Or the feeling that "just push it > to a new project or a different one is better". This fractured view > of OpenStack users at the project level is preventing progress on > OpenStack as a whole. > > Only an overarching Architectural group with some power to define > what a user is, or the TC can address those sorts of issues. I'll concede this requirement if you can point out to me who this group is for the Linux Kernel. If you're tempted to say "Linus", it's most certainly not: while he does care about some architectural decisions, he emphatically avoids most of them, which leaves the subsystem maintainers (some equivalence to openstack projects/PTLs) doing this on a case by case basis. James > Thanks, > Kevin > > From: James Bottomley [james.bottom...@hansenpartnership.com] > Sent: Wednesday, July 20, 2016 9:57 AM > To: OpenStack Development Mailing List (not for usage questions); > Clint Byrum > Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins > for all) > > On Wed, 2016-07-20 at 16:08 +, Fox, Kevin M wrote: > > +1 to the finding of a middle ground. > > Thanks ... I have actually been an enterprise architect (I just keep > very quiet about it when talking Open Source). > > > The problem I've seen with your suggested OpenSource solution is > > the > > current social monetary system of OpenStack makes it extremely > > difficult. > > > > Each project currently prints its own currency. Reviews. It takes > > quite a few Reviews (time/effort) on a project to gain enough > > currency that you get payed attention to. And, one project doesn't > > honor another projects currency. > > OK, I accept your analogy, even though I would view currency as the > will to create and push patches. > > The problem you describe: getting the recipients to listen and accept > your patches, is also a common one. The first essential is simple > minimal patches because they're hard to reject. > > Once you've overcome the reject barrier, there's the indifference one > (no-one says no, but no-one says yes). > > Overcoming indifference involves partly knowing who to bother and > when > (In openstack, it's quite easy since you know who the core reviewers > are) and also building a consensus for the patch; usually this > involves > finding other people who need the feature and getting them to pipe up > (if you can't find other projects, then you can get potential users > to > do this) even better if they help you write the patches. Ideally, > you > build your consensus before you actually push the patch set. > Sometimes > building consensus involves looking beyond your particular need to a > bigger one that would satisfy you but also pulls other people in. > > > When these sorts of major cross project things need to be done > > though, you need to have folks with capital in many different > > projects and its very difficult to amass that much. > > > > There is no OpenStack level currency (other then being
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On Wed, 2016-07-20 at 20:01 +, Fox, Kevin M wrote: > I would argue Linus, yes. As he's constantly stepping up when a > subsistem tries and breaks something for a user or creates a user > facing mess and says, no, subsystem, no. breaking userspace is > unacceptable, or, we're not adding support for an api we have to > support forever thats very poorly designed. Those are all vetos. He doesn't compel one subsystem to accept the patches of another, for instance. > Yes, he defers a lot to subsystem maintainers, as they have > generally gotten the message of paying close attention to that kind > of thing over time, and he hasn't had to speak up so much anymore. > The rest really is best left up to the subsystems. But someone has to > keep an eye on the big picture. The users of the whole thing. Users > care about the linux kernel as a whole, and less so about any given > subsystem. He says "don't build this" (veto) he doesn't say "do build that" (compulsion). The problem I've heard a lot of people describing on this thread is the latter: difficulty of getting one group to pay attention to the needs of another. Your "overarching Architectural group with some power to define what a user is" is something like this. The only power in Linux is the power to say "no". The only way an individual or a group builds acceptance for their own patches is on their own. Architectural decisions in this model are driven locally not globally. James > Thanks, > Kevin > > From: James Bottomley [james.bottom...@hansenpartnership.com] > Sent: Wednesday, July 20, 2016 12:42 PM > To: OpenStack Development Mailing List (not for usage questions); > Clint Byrum > Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins > for all) > > On Wed, 2016-07-20 at 18:18 +, Fox, Kevin M wrote: > > I wish it was so simple. Its not. > > > > There is a good coding practice: > > "The code is done, not when there is nothing more to add, but > > nothing > > more to remove" > > > > Some of the more mature projects are in this mode of thinking now. > > (which is mostly good, really). They don't want to add features > > unless they see it as a benefit to their users. This is the > > problem, > > there is a disconnect between the view of Project X's users, and > > greater view of OpenStack Users. > > > > Even accepting the smallest of patches to work towards the goal is > > unacceptable to projects if they believe it is not a helpful > > feature > > to their perceived user base, or helpful enough to them to justify > > adding more code to their project. Or the feeling that "just push > > it > > to a new project or a different one is better". This fractured view > > of OpenStack users at the project level is preventing progress on > > OpenStack as a whole. > > > > Only an overarching Architectural group with some power to define > > what a user is, or the TC can address those sorts of issues. > > I'll concede this requirement if you can point out to me who this > group > is for the Linux Kernel. If you're tempted to say "Linus", it's most > certainly not: while he does care about some architectural decisions, > he emphatically avoids most of them, which leaves the subsystem > maintainers (some equivalence to openstack projects/PTLs) doing this > on > a case by case basis. > > James > > > Thanks, > > Kevin > > > > From: James Bottomley [james.bottom...@hansenpartnership.com] > > Sent: Wednesday, July 20, 2016 9:57 AM > > To: OpenStack Development Mailing List (not for usage questions); > > Clint Byrum > > Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to > > Plugins > > for all) > > > > On Wed, 2016-07-20 at 16:08 +, Fox, Kevin M wrote: > > > +1 to the finding of a middle ground. > > > > Thanks ... I have actually been an enterprise architect (I just > > keep > > very quiet about it when talking Open Source). > > > > > The problem I've seen with your suggested OpenSource solution is > > > the > > > current social monetary system of OpenStack makes it extremely > > > difficult. > > > > > > Each project currently prints its own currency. Reviews. It takes > > > quite a few Reviews (time/effort) on a project to gain enough > > > currency that you get payed attention to. And, one project > > > doesn't > > > honor another projects
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Excerpts from Fox, Kevin M's message of 2016-07-20 20:12:48 +: > And maybe this raises an interesting defininition mismatch in the > conversation. > > There is archetectural stuff like, do we support 7 different web frameworks, > or do we standardize on flask... python vs go. > Yeah meh, that's developer centric implementation details and I think not very interesting. To me the architectural questions are deeper. "How do we do locking?", "How should we enable inter-process and inter-host communication?", "How do we handle distributed transactions?" and "What concurrency model should we use?". > Theres also the architectural stuff at the, what interactive surface do you > expose to users/operators. How consistant is it. Does it have all the > features, no matter where they are implemented to do work. I believe this is the goal of the API-WG. But again, they're not there to compel, they're there to advise, assist, and work. Ultimately, if an API is created and is poor, like Linus, the community can definitely say "No" and refuse to use that API. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Excerpts from Fox, Kevin M's message of 2016-07-19 21:59:29 +: > Yeah. I'm not saying any project has done it out of malice. Everyone's just > doing whats best for their project. But it does not seem like there is an > overarching entity watching over the whole or (pushing, encouraging, > enticing, whatever words are appropriate here) projects to work on the > commons anymore. It use to be that incubating projects were pushed to help > the other projects integrate with them by the incubating project being > strongly encouraged to write the integrations themselves as part of the > incubation process. Now it seems like each project just spawns and then hopes > someone else will do the legwork. Using the carrot of incubation to further > the commons is not an ideal solution, but it was at least something. > > Linux has an overarching entity, Linus for that task. He's there to make sure > that someone is really paying attention to integration of the whole thing > into a cohesive, usable whole. Sometimes he pushes back and makes sure > commons work happens as part of features getting in to ensure commons work > still gets done. I'm not advocating a benevolent dictator for OpenStack > though. > > But what I thought what the TC's job was, was benevolent dictators, which each subproject (or subsystem in linux terms) are required to give up final say to, so that sometimes the projects have to sacrifice a bit so that the whole can flourish and those benevolent dictators are elected for a time, by the OpenStack community. (Actually, I think that kind of makes it a Democratic Republic... but I digress) Maybe I misunderstood what the TC's about. But I think we still do need some folks elected by the community to be involved in making sure OpenStack as a whole has a cohesive technical architecture that actually addresses user problems and that have some power to to stop the "this feature belongs in this project", "no, it belongs in that project", "no, lets spawn 3 new projects to cover that case" kinds of things, make the difficult decision, and ask a project to help the community out by going along with "a solution" and we all can move on. Critical features have been stuck in this space for years and OpenStacks competitors have had solutions for years. > > Interest in OpenStack as a whole has leveled off in about: > > ____ > From: Zane Bitter [zbit...@redhat.com] > Sent: Tuesday, July 19, 2016 1:08 PM > To: openstack-dev@lists.openstack.org > Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all) > > On 14/07/16 16:30, Fox, Kevin M wrote: > > I'm going to go ahead and ask the difficult question now as the answer is > > relevant to the attached proposal... > > > > Should we reconsider whether the big tent is the right approach going > > forward? > > > > There have been some major downsides I think to the Big Tent approach, such > > as: > > * Projects not working together because of fear of adding extra > > dependencies Ops don't already have > > * Reimplementing features, badly, over and over again in different > > projects instead of standardizing something. > > * More projects being created due to politics and not technical reasons. > > * Less cross project communication > > * Greater Operator pain at trying to assemble a more loose confederation > > of projects into something useful. > > * General pushing off more and more work to Operators/Users to deal with. > > * Worse User experience as cross project issues aren't being addressed. > > * Previously incubated projects such as Nova, Swift, etc getting a > > disproportionate say in things as they have a greater current user base, > > and its increasingly hard now for new projects to gain any traction. > > * Much less community pressure on projects to work together to elevate > > everyone. Architectural decisions are now made at individual project level > > with little done at the OpenStack level. > > * etc... > > The thing is, all of these problems pre-date the big tent. Arguably they > were even worse before because so many projects were not only not > blessed by the TC as hard dependencies, but officially weren't even part > of OpenStack. Say what you want about the big tent, but at least we > haven't been treated to another Zaqar graduation review farce. > > I think your complaint here is that the big tent hasn't done enough to > solve these problems. That's a fair complaint. > > I think what you're suggesting is missing is some sort of TC imprimatur > to say that it's
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On 20/07/16 18:41, Clint Byrum wrote: Excerpts from Fox, Kevin M's message of 2016-07-20 20:12:48 +: And maybe this raises an interesting defininition mismatch in the conversation. There is archetectural stuff like, do we support 7 different web frameworks, or do we standardize on flask... python vs go. Yeah meh, that's developer centric implementation details and I think not very interesting. To me the architectural questions are deeper. "How do we do locking?", "How should we enable inter-process and inter-host communication?", "How do we handle distributed transactions?" and "What concurrency model should we use?". Theres also the architectural stuff at the, what interactive surface do you expose to users/operators. How consistant is it. Does it have all the features, no matter where they are implemented to do work. I believe this is the goal of the API-WG. But again, they're not there to compel, they're there to advise, assist, and work. Ultimately, if an API is created and is poor, like Linus, the community can definitely say "No" and refuse to use that API. No, the API-WG is pretty much just about coming up with standards for how individual REST APIs look. Kevin (IIUC) is talking about something at least as different from that as 'which web framework?' is from your list of architecture questions. It's questions like: How can applications running in the cloud authenticate themselves to the cloud? How can the user limit their authorisation to a minimal surface? How can the cloud notify applications of events? How can the user configure the cloud to respond to events without having to write their own service to process them? How should guest agents communicate with the cloud? If we're not to end up with 20 different answers to the those questions, we'll need some cross-project co-ordination and part of that will involve depending on various projects for functionality instead of implementing multiple different one-off solutions. Pick an asynchronous message transport (Zaqar). Pick an event source (Ceilometer? Searchlight?). Maybe pick an event sink (just Mistral? or lots of sinks?). So it's architecture, but it is in a sense "user-space" architecture, figuring out how services fit together into a cohesive whole, as opposed to the questions you're talking about which are more engineering-focused. I'd be very interested to know if you consider this stuff in scope for your architecture group or if you think it should have its own separate working group. cheers, Zane. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Zane, Thanks. You have managed to articulate my concern where I've failed so far. +1 :) Kevin From: Zane Bitter [zbit...@redhat.com] Sent: Thursday, July 21, 2016 3:04 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all) On 20/07/16 18:41, Clint Byrum wrote: > Excerpts from Fox, Kevin M's message of 2016-07-20 20:12:48 +: >> And maybe this raises an interesting defininition mismatch in the >> conversation. >> >> There is archetectural stuff like, do we support 7 different web frameworks, >> or do we standardize on flask... python vs go. >> > > Yeah meh, that's developer centric implementation details and I think > not very interesting. To me the architectural questions are deeper. "How > do we do locking?", "How should we enable inter-process and inter-host > communication?", "How do we handle distributed transactions?" and "What > concurrency model should we use?". > >> Theres also the architectural stuff at the, what interactive surface do you >> expose to users/operators. How consistant is it. Does it have all the >> features, no matter where they are implemented to do work. > > I believe this is the goal of the API-WG. But again, they're not there > to compel, they're there to advise, assist, and work. Ultimately, if an > API is created and is poor, like Linus, the community can definitely say > "No" and refuse to use that API. No, the API-WG is pretty much just about coming up with standards for how individual REST APIs look. Kevin (IIUC) is talking about something at least as different from that as 'which web framework?' is from your list of architecture questions. It's questions like: How can applications running in the cloud authenticate themselves to the cloud? How can the user limit their authorisation to a minimal surface? How can the cloud notify applications of events? How can the user configure the cloud to respond to events without having to write their own service to process them? How should guest agents communicate with the cloud? If we're not to end up with 20 different answers to the those questions, we'll need some cross-project co-ordination and part of that will involve depending on various projects for functionality instead of implementing multiple different one-off solutions. Pick an asynchronous message transport (Zaqar). Pick an event source (Ceilometer? Searchlight?). Maybe pick an event sink (just Mistral? or lots of sinks?). So it's architecture, but it is in a sense "user-space" architecture, figuring out how services fit together into a cohesive whole, as opposed to the questions you're talking about which are more engineering-focused. I'd be very interested to know if you consider this stuff in scope for your architecture group or if you think it should have its own separate working group. cheers, Zane. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
Thank you for making it more clear, Zane. I totally agree with you at here. IMHO, as a cloud computing platform, it should be an ecosystem just like the others. That said, not only looking from bottom to top, but also looking from top to bottom. It should looks like an unified platform. In other words, we need to understand what are the app developer need. I'm sure except a good VM creation, they also need something else. To avoid it turns to be another chicken/egg problem, we could/should to eat our dog food, instead of just "waiting" a service get matured by itself, we could provide some support/help. Therefore, I think plugins for all could be a good start. My 0.02 On 22/07/16 10:04, Zane Bitter wrote: On 20/07/16 18:41, Clint Byrum wrote: Excerpts from Fox, Kevin M's message of 2016-07-20 20:12:48 +: And maybe this raises an interesting defininition mismatch in the conversation. There is archetectural stuff like, do we support 7 different web frameworks, or do we standardize on flask... python vs go. Yeah meh, that's developer centric implementation details and I think not very interesting. To me the architectural questions are deeper. "How do we do locking?", "How should we enable inter-process and inter-host communication?", "How do we handle distributed transactions?" and "What concurrency model should we use?". Theres also the architectural stuff at the, what interactive surface do you expose to users/operators. How consistant is it. Does it have all the features, no matter where they are implemented to do work. I believe this is the goal of the API-WG. But again, they're not there to compel, they're there to advise, assist, and work. Ultimately, if an API is created and is poor, like Linus, the community can definitely say "No" and refuse to use that API. No, the API-WG is pretty much just about coming up with standards for how individual REST APIs look. Kevin (IIUC) is talking about something at least as different from that as 'which web framework?' is from your list of architecture questions. It's questions like: How can applications running in the cloud authenticate themselves to the cloud? How can the user limit their authorisation to a minimal surface? How can the cloud notify applications of events? How can the user configure the cloud to respond to events without having to write their own service to process them? How should guest agents communicate with the cloud? If we're not to end up with 20 different answers to the those questions, we'll need some cross-project co-ordination and part of that will involve depending on various projects for functionality instead of implementing multiple different one-off solutions. Pick an asynchronous message transport (Zaqar). Pick an event source (Ceilometer? Searchlight?). Maybe pick an event sink (just Mistral? or lots of sinks?). So it's architecture, but it is in a sense "user-space" architecture, figuring out how services fit together into a cohesive whole, as opposed to the questions you're talking about which are more engineering-focused. I'd be very interested to know if you consider this stuff in scope for your architecture group or if you think it should have its own separate working group. cheers, Zane. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Cheers & Best regards, Fei Long Wang (王飞龙) -- Senior Cloud Software Engineer Tel: +64-48032246 Email: flw...@catalyst.net.nz Catalyst IT Limited Level 6, Catalyst House, 150 Willis Street, Wellington -- __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
On 22/07/16 10:04, Zane Bitter wrote: > If we're not to end up with 20 different answers to the those > questions, we'll need some cross-project co-ordination and part of > that will involve depending on various projects for functionality > instead of implementing multiple different one-off solutions. Pick an > asynchronous message transport (Zaqar). Pick an event source > (Ceilometer? Searchlight?). Maybe pick an event sink (just Mistral? or > lots of sinks?). > > So it's architecture, but it is in a sense "user-space" architecture, > figuring out how services fit together into a cohesive whole, as > opposed to the questions you're talking about which are more > engineering-focused. I'd be very interested to know if you consider > this stuff in scope for your architecture group or if you think it > should have its own separate working group. > Well said, Zane. I 100% have felt and continue to feel the pain that Kevin originally mentioned with simply being blocked on progress because the projects are often silo’d and there is no high level architecture group essentially saying it is okay for project X to have a dependency on project Y. I know at least on Horizon, that summit after summit, cross cutting features that could really enhance the end user experience are rejected out of fear of adding any additional dependency (even if optional). I probably can’t even begin to count the number of times people have asked to add additional dynamic user preference customizations but been blocked because Horizon doesn’t want to add any kind of a dependency on any kind of a persistence mechanism. Another problem I see is that project A has a high priority need to land a relatively simple patch in project B, but due to the extremely limited amount of time to actually get any reviews for non-priority features in project B, that only a few weeks into the release cycle you get blocked with -2 and a “better luck next time” message. Which effectively means you are barely a month or two into the current release and now are faced with the reality that it will be *at least* 9 months before you can hope to see your patch be released in the next “alphabet letter of your choice” release. In the meantime, the people paying the salaries of the developers working on OpenStack decide that progress is too slow and pull their people off all together. But don’t worry, the community elitists will just declare these to be drive by contributions and are happy to essentially pat themselves on the back for preventing those people from contributing code. I am a core reviewer on a couple projects and I can honestly say that I believe the core reviewer teams are also a huge part of the bottleneck. This certainly isn’t due to cores being lazy, quite the contrary, but that perhaps it is too much to expect of core reviewers to essentially perform the product management role, the project management role, the architecture role, the evangelist role, the developer roles, and finally the quality assurance role (you know, the actual code review thing). One of the most beautiful things about OpenStack is that the developers are empowered in a way that they simply are not within the confines of large proprietary enterprise software shops. However, the inability to make progress from release to release whenever it comes to cross project integration and dependencies is an Achilles heel that I fear may be the downfall of OpenStack. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev