[openstack-dev] [all][tc] Equal Chances for all projects (was Plugins for all)
Top posting - this is a recap of what has been said, and some clarifications I realise that I was not very clear at the beginning of this process, so here is my effort to clarify things, from the ML thread, and the review. > ... does this also include plugins within projects, like storage > backends in cinder and hypervisor drivers in nova? No - this was not clear enough. This change is aimed at projects that are points of significant cross project interaction. While, in the future there may come a point where Nova Compute Drivers are developed out of tree (though I doubt it), that is not happening today. As a result, there is no projects in the list of projects that would need to integrate with Nova. > Could you please clarify: do you advocate for a generic plugin interface for > every project, or that each project should expose a plugin interface that > allows plugin to behave as in-tree components? Because the latter is what > happens with Tempest, and I see the former a bit complicated. For every project that has cross project interaction - tempest is a good example. For these projects, they should allow all projects in tree (like Nova, Neutron, Cinder etc are today), or they should have a plugin interface (like they currently do), but all projects *must* use it, and not use parts of tempest that are not exposed in that interface. This would mean that tempest would move the nova, neutron, etc tests to use the plugin interface. Now, that plugin could be kept in the tempest repo, and still maintained by the QA team, but should use the same interface as the other plugins that are not in that repository. Of course, it is not just tempest - an incomplete list looks like: * Tempest * Devstack * Grende * Horizon * OpenStack Client * OpenStack SDK * Searchlight * Heat * Mistral * Celiometer * Rally * Documentation And I am sure I have missed some obvious ones. (if you see a project missing let me know on the review) > I think I disagree here. The root cause is being addressed: external tests can > use the Tempest plugin interface, and use the API, which is being stabilized. > The fact that the Tempest API is partially unstable is a temporary things, due > to the origin of the project and the way the scope was redefined, but again > it's temporary. This seems to be the core of a lot of the disagreement - this is only temporary, it will all be fixed in the future, and it should stay this way. Unfortunately the discrepancy between projects is not temporary. The specific problems I have highlighted in the thread for one of the projects is temporary, but I beleive the only long-term solution is to remove the difference between projects. > Before we start making lots of specific rules about how teams > coordinate, I would like to understand the problem those rules are meant > to solve, so thank you for providing that example. > ... > It's not clear yet whether there needs to be a new policy to change the > existing intent, or if a discussion just hasn't happened, or if someone > simply needs to edit some code. Unfortunately there is a big push back on editing code to help plugins from some of the projects. Again, having the differing access between projects will continue to exacerbate the problem. > "Change the name of the resolution" That was done in the last patchset. I think the Level Playing Field title bounced around my head from the other resolution that was titled Level Playing Field. It may have been confusing alright. I feel like I have been using tempest as an example a little too much, as it captures the current issues perfectly, and a large number of the community have some knowledge of it, and how it works. There is other areas across OpenStack where plugins need attention as well: Horizon --- Horizon privileged projects have access to much more panels than plugins (service status, quotas, overviews etc). Plugins have to rely on tarballs of horizon OpenStack Client OpenStack CLI privileged projects have access to more commands, as plugins cannot hook in to them (e.g. quotas) Grenade --- Plugins may or may not have tempest tests ran (I think that patch merged), they have to use parts of tempest I was told explicitly plugins should not use to get the tests to run at that point. Docs We can now add install guides and hook into the API Reference, and API guides. This is great - and I am really happy about it. We still have issues trying to integrate with other areas in docs, and most non docs privileged projects end up with massive amounts of users docs in docs.openstack.org/developer/ , which is not ideal. Thanks, Graham On 14/07/2016 20:25, Hayes, Graham wrote: > I just proposed a review to openstack/governance repo [0] that aims > to have everything across OpenStack be plugin based for all cross > project interaction, or allow all projects access to the same internal >
Re: [openstack-dev] [all][tc] Equal Chances for all projects (was Plugins for all)
Hi, I am reading through the thread, and it puzzles me that I see a lot of right words about goals but not enough hints on who is going to implement that. We all love public and stable interfaces, everyone needs it, and I don’t think anyone works against that goal. For one, tempest folks work on defining a plugin interface, and they work with interested projects too, projects like neutron already switched to plugin interface. The very fact that the interface is in progress shows that the tempest project think about the issue and works with interested parties on covering their use cases. Yes, projects like neutron still rely on some internal functions, but it’s not because we have some special status, but because we had adopted in tree tempest tests long before the plugin interface became available. And yes, we still work to switch our in tree code to public interfaces, but it takes time and coordination with tempest folks. We have other things to do, so maybe we don’t put it as the most critical priority, but we are aware of the issue. In the meantime, neutron in tree tests are sometimes broken by tempest changes that refactor their guts. It happens to neutron as well as for other projects, not more often but neither less. So it’s a level playing field already. If you don’t like to be broken from time to time, spend time on tempest to expose missing public interfaces for your needs. Don’t expect that tempest folks will just show up and immediately cover all your use cases. Ihar Graham wrote: Top posting - this is a recap of what has been said, and some clarifications I realise that I was not very clear at the beginning of this process, so here is my effort to clarify things, from the ML thread, and the review. ... does this also include plugins within projects, like storage backends in cinder and hypervisor drivers in nova? No - this was not clear enough. This change is aimed at projects that are points of significant cross project interaction. While, in the future there may come a point where Nova Compute Drivers are developed out of tree (though I doubt it), that is not happening today. As a result, there is no projects in the list of projects that would need to integrate with Nova. Could you please clarify: do you advocate for a generic plugin interface for every project, or that each project should expose a plugin interface that allows plugin to behave as in-tree components? Because the latter is what happens with Tempest, and I see the former a bit complicated. For every project that has cross project interaction - tempest is a good example. For these projects, they should allow all projects in tree (like Nova, Neutron, Cinder etc are today), or they should have a plugin interface (like they currently do), but all projects *must* use it, and not use parts of tempest that are not exposed in that interface. This would mean that tempest would move the nova, neutron, etc tests to use the plugin interface. Now, that plugin could be kept in the tempest repo, and still maintained by the QA team, but should use the same interface as the other plugins that are not in that repository. Of course, it is not just tempest - an incomplete list looks like: * Tempest * Devstack * Grende * Horizon * OpenStack Client * OpenStack SDK * Searchlight * Heat * Mistral * Celiometer * Rally * Documentation And I am sure I have missed some obvious ones. (if you see a project missing let me know on the review) I think I disagree here. The root cause is being addressed: external tests can use the Tempest plugin interface, and use the API, which is being stabilized. The fact that the Tempest API is partially unstable is a temporary things, due to the origin of the project and the way the scope was redefined, but again it's temporary. This seems to be the core of a lot of the disagreement - this is only temporary, it will all be fixed in the future, and it should stay this way. Unfortunately the discrepancy between projects is not temporary. The specific problems I have highlighted in the thread for one of the projects is temporary, but I beleive the only long-term solution is to remove the difference between projects. Before we start making lots of specific rules about how teams coordinate, I would like to understand the problem those rules are meant to solve, so thank you for providing that example. ... It's not clear yet whether there needs to be a new policy to change the existing intent, or if a discussion just hasn't happened, or if someone simply needs to edit some code. Unfortunately there is a big push back on editing code to help plugins from some of the projects. Again, having the differing access between projects will continue to exacerbate the problem. "Change the name of the resolution" That was done in the last patchset. I think the Level Playing Field title bounced around my head from the ot
Re: [openstack-dev] [all][tc] Equal Chances for all projects (was Plugins for all)
On Mon, Jul 25, 2016 at 10:36 PM Hayes, Graham wrote: > Top posting - this is a recap of what has been said, and some > clarifications > > I realise that I was not very clear at the beginning of this process, so > here is my effort to clarify things, from the ML thread, and the review. > > > ... does this also include plugins within projects, like storage > > backends in cinder and hypervisor drivers in nova? > > No - this was not clear enough. This change is aimed at projects that are > points of significant cross project interaction. While, in the future > there may > come a point where Nova Compute Drivers are developed out of tree (though > I doubt it), that is not happening today. As a result, there is no > projects in > the list of projects that would need to integrate with Nova. > > > Could you please clarify: do you advocate for a generic plugin > interface for > > every project, or that each project should expose a plugin > interface that > > allows plugin to behave as in-tree components? Because the latter > is what > > happens with Tempest, and I see the former a bit complicated. > > For every project that has cross project interaction - tempest is a good > example. > > For these projects, they should allow all projects in tree (like Nova, > Neutron, Cinder etc are today), or they should have a plugin interface > (like they currently do), but all projects *must* use it, and not use > parts of tempest that are not exposed in that interface. > Defining a policy that *all* projects must use the plugin interface would not help. You could define plugins in tree in Tempest but that would not change much. The plugin interface is quite complete as it is today, I don't expect it will be extended much after [0] is merged. What still requires work in Tempest is the stable interface. Because plugins are not in the Tempest tree, the QA team recommend that they use only tempest stable interfaces. Increasing the surface of stable interfaces is what keeps a lot of the QA folks busy. There's a lot of code in tempest that was written under the assumption that all tests would always live in the tempest tree; evolving that code into a stable publicly consumable interface is simply a lot of work, which the QA team is prioritising based on the input from plugins. Tempest plugins are for all right now. Keystone, Cinder and Neutron already have a plugin today. We don't want tests which are not relevant for integration or defcore to be added to Tempest, and that is true for *all* services. > This would mean that tempest would move the nova, neutron, etc tests to > use the plugin interface. > Now, that plugin could be kept in the tempest repo, and still maintained > by the QA team, but should use the same interface as the other plugins > that are not in that repository. > All tests in tempest consume the stable interfaces in tempest.lib. The QA team is working continuously to stabilise and migrate more parts of tempest into the tempest.lib namespace. Since a lot of plugins are using tempest unstable interfaces, we're also trying as much as possible to maintain backward compatibility on unstable interfaces, until the stable version is available, but that is not always an option. > > Of course, it is not just tempest - an incomplete list looks like: > > * Tempest > * Devstack > * Grende > * Horizon > * OpenStack Client > * OpenStack SDK > * Searchlight > * Heat > * Mistral > * Celiometer > * Rally > * Documentation > > And I am sure I have missed some obvious ones. (if you see a project > missing > let me know on the review) > > > > I think I disagree here. The root cause is being addressed: > external tests can > > use the Tempest plugin interface, and use the API, which is being > stabilized. > > The fact that the Tempest API is partially unstable is a temporary > things, due > > to the origin of the project and the way the scope was redefined, > but again > > it's temporary. > > This seems to be the core of a lot of the disagreement - this is only > temporary, > it will all be fixed in the future, and it should stay this way. > > Unfortunately the discrepancy between projects is not temporary. The > specific > problems I have highlighted in the thread for one of the projects is > temporary, > but I beleive the only long-term solution is to remove the difference > between > projects. > > > Before we start making lots of specific rules about how teams > > coordinate, I would like to understand the problem those rules are > meant > > to solve, so thank you for providing that example. > > ... > > It's not clear yet whether there needs to be a new policy to change > the > > existing intent, or if a discussion just hasn't happened, or if > someone > > simply needs to edit some code. > > Unfortunately there is a big push back on editing code to help plugins from > some of the projects. Again, having the differing access between > projects will > continue to ex
Re: [openstack-dev] [all][tc] Equal Chances for all projects (was Plugins for all)
Excerpts from Andrea Frittoli's message of 2016-07-26 10:24:07 +: > On Mon, Jul 25, 2016 at 10:36 PM Hayes, Graham wrote: > > > Top posting - this is a recap of what has been said, and some > > clarifications > > > > I realise that I was not very clear at the beginning of this process, so > > here is my effort to clarify things, from the ML thread, and the review. > > > > > ... does this also include plugins within projects, like storage > > > backends in cinder and hypervisor drivers in nova? > > > > No - this was not clear enough. This change is aimed at projects that are > > points of significant cross project interaction. While, in the future > > there may > > come a point where Nova Compute Drivers are developed out of tree (though > > I doubt it), that is not happening today. As a result, there is no > > projects in > > the list of projects that would need to integrate with Nova. > > > > > Could you please clarify: do you advocate for a generic plugin > > interface for > > > every project, or that each project should expose a plugin > > interface that > > > allows plugin to behave as in-tree components? Because the latter > > is what > > > happens with Tempest, and I see the former a bit complicated. > > > > For every project that has cross project interaction - tempest is a good > > example. > > > > For these projects, they should allow all projects in tree (like Nova, > > Neutron, Cinder etc are today), or they should have a plugin interface > > (like they currently do), but all projects *must* use it, and not use > > parts of tempest that are not exposed in that interface. > > > > Defining a policy that *all* projects must use the plugin interface would > not help. > You could define plugins in tree in Tempest but that would not change much. > The plugin interface is quite complete as it is today, I don't expect it > will be extended much after [0] is merged. > > What still requires work in Tempest is the stable interface. Because > plugins are not in the Tempest tree, the QA team recommend that they use > only tempest stable interfaces. Increasing the surface of stable interfaces > is what keeps a lot of the QA folks busy. > > There's a lot of code in tempest that was written under the assumption that > all tests would always live in the tempest tree; evolving that code into a > stable publicly consumable interface is simply a lot of work, which the QA > team is prioritising based on the input from plugins. > > Tempest plugins are for all right now. Keystone, Cinder and Neutron already > have a plugin today. We don't want tests which are not relevant for > integration or defcore to be added to Tempest, and that is true for *all* > services. Thank you for those details, and for confirming that the situation is more or less what I expected (it's in progress, creating a stable API takes work, etc.). Where would someone who wanted to contribute look for details? Are there specs or an etherpad with a task list or something like that? And just to satisfy my own curiosity, how does someone looking at the internals of tempest know what's on the stable API and what's not considered stable? Are the parts of the API documented separately somehow or is there a different part of the code tree to look at? Doug > > > This would mean that tempest would move the nova, neutron, etc tests to > > use the plugin interface. > > > Now, that plugin could be kept in the tempest repo, and still maintained > > by the QA team, but should use the same interface as the other plugins > > that are not in that repository. > > > > All tests in tempest consume the stable interfaces in tempest.lib. > The QA team is working continuously to stabilise and migrate more parts of > tempest into the tempest.lib namespace. > > Since a lot of plugins are using tempest unstable interfaces, we're also > trying as much as possible to maintain backward compatibility on unstable > interfaces, until the stable version is available, but that is not always > an option. > > > > > Of course, it is not just tempest - an incomplete list looks like: > > > > * Tempest > > * Devstack > > * Grende > > * Horizon > > * OpenStack Client > > * OpenStack SDK > > * Searchlight > > * Heat > > * Mistral > > * Celiometer > > * Rally > > * Documentation > > > > And I am sure I have missed some obvious ones. (if you see a project > > missing > > let me know on the review) > > > > > > > I think I disagree here. The root cause is being addressed: > > external tests can > > > use the Tempest plugin interface, and use the API, which is being > > stabilized. > > > The fact that the Tempest API is partially unstable is a temporary > > things, due > > > to the origin of the project and the way the scope was redefined, > > but again > > > it's temporary. > > > > This seems to be the core of a lot of the disagreement - this is only > > temporary, > > it will all be fixed in the future, and it should stay this way. > > > > Un
Re: [openstack-dev] [all][tc] Equal Chances for all projects (was Plugins for all)
Excerpts from Hayes, Graham's message of 2016-07-25 21:36:29 +: > Top posting - this is a recap of what has been said, and some > clarifications > > I realise that I was not very clear at the beginning of this process, so > here is my effort to clarify things, from the ML thread, and the review. > > > ... does this also include plugins within projects, like storage > > backends in cinder and hypervisor drivers in nova? > > No - this was not clear enough. This change is aimed at projects that are > points of significant cross project interaction. While, in the future > there may > come a point where Nova Compute Drivers are developed out of tree (though > I doubt it), that is not happening today. As a result, there is no > projects in > the list of projects that would need to integrate with Nova. > > > Could you please clarify: do you advocate for a generic plugin > interface for > > every project, or that each project should expose a plugin > interface that > > allows plugin to behave as in-tree components? Because the latter > is what > > happens with Tempest, and I see the former a bit complicated. > > For every project that has cross project interaction - tempest is a good > example. > > For these projects, they should allow all projects in tree (like Nova, > Neutron, Cinder etc are today), or they should have a plugin interface > (like they currently do), but all projects *must* use it, and not use > parts of tempest that are not exposed in that interface. > > This would mean that tempest would move the nova, neutron, etc tests to > use the plugin interface. > > Now, that plugin could be kept in the tempest repo, and still maintained > by the QA team, but should use the same interface as the other plugins > that are not in that repository. > > Of course, it is not just tempest - an incomplete list looks like: > > * Tempest > * Devstack > * Grende > * Horizon > * OpenStack Client > * OpenStack SDK > * Searchlight > * Heat > * Mistral > * Celiometer > * Rally > * Documentation > > And I am sure I have missed some obvious ones. (if you see a project missing > let me know on the review) > > > I think I disagree here. The root cause is being addressed: > external tests can > > use the Tempest plugin interface, and use the API, which is being > stabilized. > > The fact that the Tempest API is partially unstable is a temporary > things, due > > to the origin of the project and the way the scope was redefined, > but again > > it's temporary. > > This seems to be the core of a lot of the disagreement - this is only > temporary, > it will all be fixed in the future, and it should stay this way. > > Unfortunately the discrepancy between projects is not temporary. The > specific > problems I have highlighted in the thread for one of the projects is > temporary, > but I beleive the only long-term solution is to remove the difference > between > projects. > > > Before we start making lots of specific rules about how teams > > coordinate, I would like to understand the problem those rules are > meant > > to solve, so thank you for providing that example. > > ... > > It's not clear yet whether there needs to be a new policy to change the > > existing intent, or if a discussion just hasn't happened, or if someone > > simply needs to edit some code. > > Unfortunately there is a big push back on editing code to help plugins from > some of the projects. Again, having the differing access between > projects will > continue to exacerbate the problem. > > > "Change the name of the resolution" > > That was done in the last patchset. I think the Level Playing Field title > bounced around my head from the other resolution that was titled Level > Playing > Field. It may have been confusing alright. > > I feel like I have been using tempest as an example a little too much, > as it captures > the current issues perfectly, and a large number of the community have some > knowledge of it, and how it works. > > There is other areas across OpenStack where plugins need attention as well: > > Horizon > --- > > Horizon privileged projects have access to much more panels than > plugins (service status, quotas, overviews etc). Is this by intent, lack of foresight, or changing requirements? > Plugins have to rely on tarballs of horizon It sounds like we need to figure out how to tag intermediate releases of horizon and publish them to PyPI so projects can use them in tests. I'd be happy to work with the Horizon team on that. > > OpenStack Client > > > OpenStack CLI privileged projects have access to more commands, as > plugins cannot hook in to them (e.g. quotas) I think Steve addressed this point elsewhere on the thread (it's not by design, but due to lack of contributor resources). > > Grenade > --- > > Plugins may or may not have tempest tests ran (I think that patch > merged), they have to use parts of te
Re: [openstack-dev] [all][tc] Equal Chances for all projects (was Plugins for all)
On Tuesday, 26 July 2016 08:34:27 CEST Doug Hellmann wrote: > Excerpts from Andrea Frittoli's message of 2016-07-26 10:24:07 +: > > > > What still requires work in Tempest is the stable interface. Because > > plugins are not in the Tempest tree, the QA team recommend that they use > > only tempest stable interfaces. Increasing the surface of stable > > interfaces > > is what keeps a lot of the QA folks busy. > > > > There's a lot of code in tempest that was written under the assumption > > that > > all tests would always live in the tempest tree; evolving that code into a > > stable publicly consumable interface is simply a lot of work, which the QA > > team is prioritising based on the input from plugins. > > > > Tempest plugins are for all right now. Keystone, Cinder and Neutron > > already > > have a plugin today. We don't want tests which are not relevant for > > integration or defcore to be added to Tempest, and that is true for *all* > > services. > > Thank you for those details, and for confirming that the situation > is more or less what I expected (it's in progress, creating a stable > API takes work, etc.). Where would someone who wanted to contribute > look for details? Are there specs or an etherpad with a task list > or something like that? Andrea can share more details as he is driving the Client Manager refactors, but for example: http://specs.openstack.org/openstack/qa-specs/specs/tempest/client-manager-refactor.html https://etherpad.openstack.org/p/newton-tempest-service-clients > > And just to satisfy my own curiosity, how does someone looking at the > internals of tempest know what's on the stable API and what's not > considered stable? Are the parts of the API documented separately > somehow or is there a different part of the code tree to look at? tempest.lib is the stable part (previously split in a separate tempest-lib repository, which is now deprecated as the code was put back into the main tempest repository): http://docs.openstack.org/developer/tempest/overview.html#library Ciao -- Luigi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] Equal Chances for all projects (was Plugins for all)
On 07/26/2016 08:51 AM, Doug Hellmann wrote: > > Given the amount of in-progress work to address the issue you've > raised, I'm not convinced we need a global rule or policy. All of > the teams mentioned are working toward the goal of providing stable > APIs already, and no one seems to be arguing against that goal. The > teams doing the work are not dragging their feet, and a rule or > policy wouldn't make the work go any faster. > > The directions for cross-project teams were given when the bit tent > change went into effect were to "support all official teams" and > definitely not "do the work for all official teams." There was also > no requirement that the support be exactly equal, and it would be > a mistake to try to require that because the issue is almost always > lack of resources and not lack of desire. Volunteers to contribute > to the work that's needed will do more to help than a one-size-fits-all > policy. Yes, exactly. Like Ihar said earlier, we get all kinds of breaks across out system all the time. It's a complex system. If you look hard what you'll notice is that project interactions where there are team members in common break a bit less. For instance Grenade breaks Nova less than other projects. oslo.versionedobjects breaks Nova less than oslo.messaging does. Why? Because even with stable interfaces and tests, a lot of behavior remains in flux given the patch rate on all projects. And when people understand both sides of a problem, they are more likely to anticipate a problem during review that no tests caught and didn't seem to change an interface. This isn't a thing that gets fixed with policy. It gets fixed with people. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] Equal Chances for all projects (was Plugins for all)
Excerpts from Luigi Toscano's message of 2016-07-26 15:02:28 +0200: > On Tuesday, 26 July 2016 08:34:27 CEST Doug Hellmann wrote: > > Excerpts from Andrea Frittoli's message of 2016-07-26 10:24:07 +: > > > > > > What still requires work in Tempest is the stable interface. Because > > > plugins are not in the Tempest tree, the QA team recommend that they use > > > only tempest stable interfaces. Increasing the surface of stable > > > interfaces > > > is what keeps a lot of the QA folks busy. > > > > > > There's a lot of code in tempest that was written under the assumption > > > that > > > all tests would always live in the tempest tree; evolving that code into a > > > stable publicly consumable interface is simply a lot of work, which the QA > > > team is prioritising based on the input from plugins. > > > > > > Tempest plugins are for all right now. Keystone, Cinder and Neutron > > > already > > > have a plugin today. We don't want tests which are not relevant for > > > integration or defcore to be added to Tempest, and that is true for *all* > > > services. > > > > Thank you for those details, and for confirming that the situation > > is more or less what I expected (it's in progress, creating a stable > > API takes work, etc.). Where would someone who wanted to contribute > > look for details? Are there specs or an etherpad with a task list > > or something like that? > Andrea can share more details as he is driving the Client Manager refactors, > but for example: > > http://specs.openstack.org/openstack/qa-specs/specs/tempest/client-manager-refactor.html > https://etherpad.openstack.org/p/newton-tempest-service-clients Good. > > > > > And just to satisfy my own curiosity, how does someone looking at the > > internals of tempest know what's on the stable API and what's not > > considered stable? Are the parts of the API documented separately > > somehow or is there a different part of the code tree to look at? > > tempest.lib is the stable part (previously split in a separate tempest-lib > repository, which is now deprecated as the code was put back into the main > tempest repository): > http://docs.openstack.org/developer/tempest/overview.html#library That seems quite clear, thanks. > > Ciao __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] Equal Chances for all projects (was Plugins for all)
On 26/07/2016 14:18, Doug Hellmann wrote: > Excerpts from Luigi Toscano's message of 2016-07-26 15:02:28 +0200: >> On Tuesday, 26 July 2016 08:34:27 CEST Doug Hellmann wrote: >>> Excerpts from Andrea Frittoli's message of 2016-07-26 10:24:07 +: What still requires work in Tempest is the stable interface. Because plugins are not in the Tempest tree, the QA team recommend that they use only tempest stable interfaces. Increasing the surface of stable interfaces is what keeps a lot of the QA folks busy. There's a lot of code in tempest that was written under the assumption that all tests would always live in the tempest tree; evolving that code into a stable publicly consumable interface is simply a lot of work, which the QA team is prioritising based on the input from plugins. Tempest plugins are for all right now. Keystone, Cinder and Neutron already have a plugin today. We don't want tests which are not relevant for integration or defcore to be added to Tempest, and that is true for *all* services. >>> >>> Thank you for those details, and for confirming that the situation >>> is more or less what I expected (it's in progress, creating a stable >>> API takes work, etc.). Where would someone who wanted to contribute >>> look for details? Are there specs or an etherpad with a task list >>> or something like that? >> Andrea can share more details as he is driving the Client Manager refactors, >> but for example: >> >> http://specs.openstack.org/openstack/qa-specs/specs/tempest/client-manager-refactor.html >> https://etherpad.openstack.org/p/newton-tempest-service-clients > > Good. > >> >>> >>> And just to satisfy my own curiosity, how does someone looking at the >>> internals of tempest know what's on the stable API and what's not >>> considered stable? Are the parts of the API documented separately >>> somehow or is there a different part of the code tree to look at? >> >> tempest.lib is the stable part (previously split in a separate tempest-lib >> repository, which is now deprecated as the code was put back into the main >> tempest repository): >> http://docs.openstack.org/developer/tempest/overview.html#library > > That seems quite clear, thanks. it is actually a bit more - http://docs.openstack.org/developer/tempest/plugin.html >> >> Ciao > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] Equal Chances for all projects (was Plugins for all)
On Tuesday, 26 July 2016 13:23:05 CEST Hayes, Graham wrote: > On 26/07/2016 14:18, Doug Hellmann wrote: > >>> And just to satisfy my own curiosity, how does someone looking at the > >>> internals of tempest know what's on the stable API and what's not > >>> considered stable? Are the parts of the API documented separately > >>> somehow or is there a different part of the code tree to look at? > >> > >> tempest.lib is the stable part (previously split in a separate > >> tempest-lib > >> repository, which is now deprecated as the code was put back into the > >> main > >> tempest repository): > >> http://docs.openstack.org/developer/tempest/overview.html#library > > > > That seems quite clear, thanks. > > it is actually a bit more - > > http://docs.openstack.org/developer/tempest/plugin.html This is the documentation on how plugins should be implemented, i.e. how the plugin code can consume the stable API described above. It's really a bit of boilerplate about the custom configuration options and the registration of the classes. -- Luigi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] Equal Chances for all projects (was Plugins for all)
On 26/07/2016 14:15, Sean Dague wrote: > On 07/26/2016 08:51 AM, Doug Hellmann wrote: > >> >> Given the amount of in-progress work to address the issue you've >> raised, I'm not convinced we need a global rule or policy. All of >> the teams mentioned are working toward the goal of providing stable >> APIs already, and no one seems to be arguing against that goal. The >> teams doing the work are not dragging their feet, and a rule or >> policy wouldn't make the work go any faster. >> >> The directions for cross-project teams were given when the bit tent >> change went into effect were to "support all official teams" and >> definitely not "do the work for all official teams." There was also >> no requirement that the support be exactly equal, and it would be >> a mistake to try to require that because the issue is almost always >> lack of resources and not lack of desire. Volunteers to contribute >> to the work that's needed will do more to help than a one-size-fits-all >> policy. > > Yes, exactly. > > Like Ihar said earlier, we get all kinds of breaks across out system all > the time. It's a complex system. If you look hard what you'll notice is > that project interactions where there are team members in common break a > bit less. For instance Grenade breaks Nova less than other projects. > oslo.versionedobjects breaks Nova less than oslo.messaging does. Why? > Because even with stable interfaces and tests, a lot of behavior remains > in flux given the patch rate on all projects. And when people understand > both sides of a problem, they are more likely to anticipate a problem > during review that no tests caught and didn't seem to change an interface. > > This isn't a thing that gets fixed with policy. It gets fixed with people. > > -Sean > If this is where all these projects are going, is a policy not a good way to indicate that this is how we want to work in the future? So when the next project starts, or a team wants to move in a different direction we have a piece of policy that shows we feel that this is the way we work as a community? I don't understand the objection to stating "this is how we should work" when teams are moving that way anyway. If to do that we need to dilute the policy, so be it. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] Equal Chances for all projects (was Plugins for all)
Excerpts from Hayes, Graham's message of 2016-07-26 13:48:35 +: > On 26/07/2016 14:15, Sean Dague wrote: > > On 07/26/2016 08:51 AM, Doug Hellmann wrote: > > > >> > >> Given the amount of in-progress work to address the issue you've > >> raised, I'm not convinced we need a global rule or policy. All of > >> the teams mentioned are working toward the goal of providing stable > >> APIs already, and no one seems to be arguing against that goal. The > >> teams doing the work are not dragging their feet, and a rule or > >> policy wouldn't make the work go any faster. > >> > >> The directions for cross-project teams were given when the bit tent > >> change went into effect were to "support all official teams" and > >> definitely not "do the work for all official teams." There was also > >> no requirement that the support be exactly equal, and it would be > >> a mistake to try to require that because the issue is almost always > >> lack of resources and not lack of desire. Volunteers to contribute > >> to the work that's needed will do more to help than a one-size-fits-all > >> policy. > > > > Yes, exactly. > > > > Like Ihar said earlier, we get all kinds of breaks across out system all > > the time. It's a complex system. If you look hard what you'll notice is > > that project interactions where there are team members in common break a > > bit less. For instance Grenade breaks Nova less than other projects. > > oslo.versionedobjects breaks Nova less than oslo.messaging does. Why? > > Because even with stable interfaces and tests, a lot of behavior remains > > in flux given the patch rate on all projects. And when people understand > > both sides of a problem, they are more likely to anticipate a problem > > during review that no tests caught and didn't seem to change an interface. > > > > This isn't a thing that gets fixed with policy. It gets fixed with people. > > > > -Sean > > > > If this is where all these projects are going, is a policy not a good > way to indicate that this is how we want to work in the future? > > So when the next project starts, or a team wants to move in a different > direction we have a piece of policy that shows we feel that this is the > way we work as a community? > > I don't understand the objection to stating "this is how we should work" > when teams are moving that way anyway. > > If to do that we need to dilute the policy, so be it. > I do agree that writing down our expectations is a good thing. In this case, I think the policy that the TC has agreed to is accurately and sufficiently documented in the existing resolution under the "Impact for horizontal teams" section [1]. The stewardship working group is actively working on a set of general principles for the TC to consider, and it may make sense to incorporate the existing policy into that list of principles to make it easier to find. I personally don't agree with changing the policy, or making it "stronger" in some way, right now because I'm not convinced that we need to treat all projects exactly the same in all cases. It seems like a relatively benign assumption to make, but until we actually have an issue that would be solved by formalizing it in policy I would prefer to avoid introducing unintended consequences in implementations of projects brought on by layers of rules. We don't want any projects discriminated against, but I don't think that's what is happening in these cases, and I think the existing policy addresses that case. Doug [1] http://governance.openstack.org/resolutions/20141202-project-structure-reform-spec.html#impact-for-horizontal-teams __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] Equal Chances for all projects (was Plugins for all)
On 26/07/2016 18:47, Doug Hellmann wrote: > Excerpts from Hayes, Graham's message of 2016-07-26 13:48:35 +: >> On 26/07/2016 14:15, Sean Dague wrote: >>> On 07/26/2016 08:51 AM, Doug Hellmann wrote: >>> Given the amount of in-progress work to address the issue you've raised, I'm not convinced we need a global rule or policy. All of the teams mentioned are working toward the goal of providing stable APIs already, and no one seems to be arguing against that goal. The teams doing the work are not dragging their feet, and a rule or policy wouldn't make the work go any faster. The directions for cross-project teams were given when the bit tent change went into effect were to "support all official teams" and definitely not "do the work for all official teams." There was also no requirement that the support be exactly equal, and it would be a mistake to try to require that because the issue is almost always lack of resources and not lack of desire. Volunteers to contribute to the work that's needed will do more to help than a one-size-fits-all policy. >>> >>> Yes, exactly. >>> >>> Like Ihar said earlier, we get all kinds of breaks across out system all >>> the time. It's a complex system. If you look hard what you'll notice is >>> that project interactions where there are team members in common break a >>> bit less. For instance Grenade breaks Nova less than other projects. >>> oslo.versionedobjects breaks Nova less than oslo.messaging does. Why? >>> Because even with stable interfaces and tests, a lot of behavior remains >>> in flux given the patch rate on all projects. And when people understand >>> both sides of a problem, they are more likely to anticipate a problem >>> during review that no tests caught and didn't seem to change an interface. >>> >>> This isn't a thing that gets fixed with policy. It gets fixed with people. >>> >>> -Sean >>> >> >> If this is where all these projects are going, is a policy not a good >> way to indicate that this is how we want to work in the future? >> >> So when the next project starts, or a team wants to move in a different >> direction we have a piece of policy that shows we feel that this is the >> way we work as a community? >> >> I don't understand the objection to stating "this is how we should work" >> when teams are moving that way anyway. >> >> If to do that we need to dilute the policy, so be it. >> > > I do agree that writing down our expectations is a good thing. In > this case, I think the policy that the TC has agreed to is accurately > and sufficiently documented in the existing resolution under the > "Impact for horizontal teams" section [1]. The stewardship working > group is actively working on a set of general principles for the > TC to consider, and it may make sense to incorporate the existing > policy into that list of principles to make it easier to find. Sure, I know the work they are doing - this has been bouncing around my PC for a bit before I had any knowledge of what the plans were there. > I personally don't agree with changing the policy, or making it > "stronger" in some way, right now because I'm not convinced that > we need to treat all projects exactly the same in all cases. It This is the crux of the debate. Are all projects equal? If they are not, I will be disappointed, but we should then move to define this in-equality, and state who is in, and who is out. In some ways we are back to the stackforge vs integrated situation, but unlike then we have no frame of reference for how a project would become more equal. If projects have difficulty getting into top level docs (for example) it is more difficult to grow adoption, which would let them into top level docs, which would help adoption ... > seems like a relatively benign assumption to make, but until we > actually have an issue that would be solved by formalizing it in I think that we have one. I could be wrong (it happens), but I think this is a problem, and waiting for something bigger to happen, then having the existential debate about what OpenStack is (again) causes issues that could be dealt with to grow. (See Golang discussion). > policy I would prefer to avoid introducing unintended consequences > in implementations of projects brought on by layers of rules. We > don't want any projects discriminated against, but I don't think > that's what is happening in these cases, and I think the existing > policy addresses that case. And I disagree - I think we need it to be more explicit. > Doug > > [1] > http://governance.openstack.org/resolutions/20141202-project-structure-reform-spec.html#impact-for-horizontal-teams > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listin
Re: [openstack-dev] [all][tc] Equal Chances for all projects (was Plugins for all)
Excerpts from Hayes, Graham's message of 2016-07-26 18:16:16 +: > On 26/07/2016 18:47, Doug Hellmann wrote: > > Excerpts from Hayes, Graham's message of 2016-07-26 13:48:35 +: > >> On 26/07/2016 14:15, Sean Dague wrote: > >>> On 07/26/2016 08:51 AM, Doug Hellmann wrote: > >>> > > Given the amount of in-progress work to address the issue you've > raised, I'm not convinced we need a global rule or policy. All of > the teams mentioned are working toward the goal of providing stable > APIs already, and no one seems to be arguing against that goal. The > teams doing the work are not dragging their feet, and a rule or > policy wouldn't make the work go any faster. > > The directions for cross-project teams were given when the bit tent > change went into effect were to "support all official teams" and > definitely not "do the work for all official teams." There was also > no requirement that the support be exactly equal, and it would be > a mistake to try to require that because the issue is almost always > lack of resources and not lack of desire. Volunteers to contribute > to the work that's needed will do more to help than a one-size-fits-all > policy. > >>> > >>> Yes, exactly. > >>> > >>> Like Ihar said earlier, we get all kinds of breaks across out system all > >>> the time. It's a complex system. If you look hard what you'll notice is > >>> that project interactions where there are team members in common break a > >>> bit less. For instance Grenade breaks Nova less than other projects. > >>> oslo.versionedobjects breaks Nova less than oslo.messaging does. Why? > >>> Because even with stable interfaces and tests, a lot of behavior remains > >>> in flux given the patch rate on all projects. And when people understand > >>> both sides of a problem, they are more likely to anticipate a problem > >>> during review that no tests caught and didn't seem to change an interface. > >>> > >>> This isn't a thing that gets fixed with policy. It gets fixed with people. > >>> > >>> -Sean > >>> > >> > >> If this is where all these projects are going, is a policy not a good > >> way to indicate that this is how we want to work in the future? > >> > >> So when the next project starts, or a team wants to move in a different > >> direction we have a piece of policy that shows we feel that this is the > >> way we work as a community? > >> > >> I don't understand the objection to stating "this is how we should work" > >> when teams are moving that way anyway. > >> > >> If to do that we need to dilute the policy, so be it. > >> > > > > I do agree that writing down our expectations is a good thing. In > > this case, I think the policy that the TC has agreed to is accurately > > and sufficiently documented in the existing resolution under the > > "Impact for horizontal teams" section [1]. The stewardship working > > group is actively working on a set of general principles for the > > TC to consider, and it may make sense to incorporate the existing > > policy into that list of principles to make it easier to find. > > Sure, I know the work they are doing - this has been bouncing around > my PC for a bit before I had any knowledge of what the plans were there. > > > I personally don't agree with changing the policy, or making it > > "stronger" in some way, right now because I'm not convinced that > > we need to treat all projects exactly the same in all cases. It > > This is the crux of the debate. > > Are all projects equal? According to our current policy, all projects are equal in the sense that cross-project teams must support them. They are not equal in that the level of support given, and criteria for choosing that level of support, is left up to the cross-project teams. > > If they are not, I will be disappointed, but we should then move to > define this in-equality, and state who is in, and who is out. This is not an in-or-out distinction. It's a matter of resources available to support teams directly, or not. A few examples: * The documentation team has drawn a line for which projects are included in the tutorial-driven installation guide they are writing. Other teams are able to write their own installation guides, or even collaborate to create joint guides. * The release team formerly had a special tag that we used to indicate which project teams we were managing. With the increasing automation, we have expanded that list of projects significantly. * The TC has directed the QA team to take tests into the Tempest tree when those tests are needed for DefCore. Any other decisions about what tests live in-tree or out is up to the QA team. > In some ways we are back to the stackforge vs integrated situation, > but unlike then we have no frame of reference for how a project would > become more equal. > > If projects have difficulty getting into top level docs (for example) > it is more difficult to grow ad
