Re: [openstack-dev] [qa] New upgrade test tool proposal.
On 06/27/2017 03:19 PM, Dean Troyer wrote: > On Tue, Jun 27, 2017 at 1:37 PM, Jay Pipeswrote: >> Hi Castulo, sorry for the delayed response on this. Has your team moved >> forward on any of this? > > IIRC this work was impacted by the OSIC shutdown, I believe it is not > currently on anyone's radar. > >> What about the Grenade testing framework that uses devstack as its >> deployment system was not useful or usable for you? > > I can take at least part of the blame in encouraging them to not > attempt to leverage Grenade directly. Grenade needs to be replaced as > it has far out-lived its expected life[0]. > > Grenade was built to do static in-place upgrades and the fact that it > has been pushed as far as it has is a happy surprise to me. However, > it is fundamentally limited in its abilities as a test orchestrator, > implementing robust multi-node capabilities and the granularity that > is required to properly do upgrade testing really needs a reboot. In > a well-funded world that would include replacing DevStack too, which > while nice is not strictly necessary to achieve the testing goals they > had. > > The thing that Grenade and DevStack have going for them besides > inertia is that they are not otherwise tied to any deployment > strategy. Starting over from scratch really is not an option at this > point, something existing really does need to be leveraged even though > it may hurt some feelings along the way for the project(s) not chosen. > > dt > > > [0] Seriously, I never expected Grenade (or DevStack for that matter) > to have survived this long, but they have mostly because they were/are > just barely good enough that nobody wants to fund replacing them. Well, we also lengthened their life and usefulness by adding an external plugin interface, that let people leverage what we had without having to wait on review queues. But, I also agree that grenade is largely pushed to its limits. That also includes the fact that I'm more or less the only "active" reviewer at this point. The only way this is even vaguely tenable is by the complexity being capped so it's straight forward enough to think through implications of patches. The complicated upgrade orchestration including multiple things executing at the same time, breaks that. It's cool if someone wants to do it, however it definitely needs more dedicated folks on the review side to be successful. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] New upgrade test tool proposal.
On Tue, Jun 27, 2017 at 1:37 PM, Jay Pipeswrote: > Hi Castulo, sorry for the delayed response on this. Has your team moved > forward on any of this? IIRC this work was impacted by the OSIC shutdown, I believe it is not currently on anyone's radar. > What about the Grenade testing framework that uses devstack as its > deployment system was not useful or usable for you? I can take at least part of the blame in encouraging them to not attempt to leverage Grenade directly. Grenade needs to be replaced as it has far out-lived its expected life[0]. Grenade was built to do static in-place upgrades and the fact that it has been pushed as far as it has is a happy surprise to me. However, it is fundamentally limited in its abilities as a test orchestrator, implementing robust multi-node capabilities and the granularity that is required to properly do upgrade testing really needs a reboot. In a well-funded world that would include replacing DevStack too, which while nice is not strictly necessary to achieve the testing goals they had. The thing that Grenade and DevStack have going for them besides inertia is that they are not otherwise tied to any deployment strategy. Starting over from scratch really is not an option at this point, something existing really does need to be leveraged even though it may hurt some feelings along the way for the project(s) not chosen. dt [0] Seriously, I never expected Grenade (or DevStack for that matter) to have survived this long, but they have mostly because they were/are just barely good enough that nobody wants to fund replacing them. -- Dean Troyer dtro...@gmail.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] New upgrade test tool proposal.
On 04/05/2017 04:22 PM, Martinez, Castulo wrote: Hi, As you might know, the TC introduced new tags [1] for upgrade processes quite some time ago, which reflect the level of maturity of an upgrade. Therefore there is the growing need of test tools which help to exercise and validate the upgrade approaches defined by the community. During the PTG at Atlanta there were discussions around options for covering this gap in testing tools, one proposal was to add this functionality to Grenade, but this idea was dismissed since it was determined that Grenade was originally designed for a different purpose and is already pushing its limits, so the consensus was towards creating a new tool to fill this gap. We are proposing a new toolset for testing an OpenStack environment before, during, and after an upgrade process. The toolset answers the question ³how does OpenStack behave across upgrades from one release N to a release N+1, or from the latest official release to master?². It also provides information that can be used to assess if an upgrade complies with the requirements for being recognized as a rolling upgrade, a zero downtime upgrade or a zero impact upgrade. Before starting this effort, we would like to hear feedback from everybody, are there any concerns with this approach? Are we missing something? You can find details of this proposal here: https://review.openstack.org/#/c/449295/3 Hi Castulo, sorry for the delayed response on this. Has your team moved forward on any of this? From first glance at the spec, it does seem that you are biting off a lot -- deploying, healthchecking, upgrading, monitoring, data/report storage, and configuration management are all parts of this tool. What about the Grenade testing framework that uses devstack as its deployment system was not useful or usable for you? Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [qa] New upgrade test tool proposal.
Hi, As you might know, the TC introduced new tags [1] for upgrade processes quite some time ago, which reflect the level of maturity of an upgrade. Therefore there is the growing need of test tools which help to exercise and validate the upgrade approaches defined by the community. During the PTG at Atlanta there were discussions around options for covering this gap in testing tools, one proposal was to add this functionality to Grenade, but this idea was dismissed since it was determined that Grenade was originally designed for a different purpose and is already pushing its limits, so the consensus was towards creating a new tool to fill this gap. We are proposing a new toolset for testing an OpenStack environment before, during, and after an upgrade process. The toolset answers the question ³how does OpenStack behave across upgrades from one release N to a release N+1, or from the latest official release to master?². It also provides information that can be used to assess if an upgrade complies with the requirements for being recognized as a rolling upgrade, a zero downtime upgrade or a zero impact upgrade. Before starting this effort, we would like to hear feedback from everybody, are there any concerns with this approach? Are we missing something? You can find details of this proposal here: https://review.openstack.org/#/c/449295/3 Thanks -- Castulo J. Martinez [1] https://governance.openstack.org/tc/reference/tags/assert_supports-rolling- upgrade.html https://governance.openstack.org/tc/reference/tags/assert_supports-zero-dow ntime-upgrade.html https://governance.openstack.org/tc/reference/tags/assert_supports-zero-imp act-upgrade.html __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev