Re: [openstack-dev] [tripleo] Managing no-mergepy template duplication
On Wed, 2014-12-03 at 14:42 +0100, Tomas Sedovic wrote:
On 12/03/2014 11:11 AM, Steven Hardy wrote:
Hi all,
Lately I've been spending more time looking at tripleo and doing some
reviews. I'm particularly interested in helping the no-mergepy and
subsequent puppet-software-config implementations mature (as well as
improving overcloud updates via heat).
Since Tomas's patch landed[1] to enable --no-mergepy in
tripleo-heat-templates, it's become apparent that frequently patches are
submitted which only update overcloud-source.yaml, so I've been trying to
catch these and ask for a corresponding change to e.g controller.yaml.
You beat me to this. Thanks for writing it up!
This raises the following questions:
1. Is it reasonable to -1 a patch and ask folks to update in both places?
I'm in favour.
2. How are we going to handle this duplication and divergence?
To follow this up we are getting in really bad shape with divergence
already. I found 3 missing sets of Rabbit, Keystone, and Neutron DVR
parameters which due to the merge window were properly ported into
overcloud-without-mergepy.yaml yet.
https://review.openstack.org/#/c/139649/ (missing Rabbit parameters)
https://review.openstack.org/#/c/139656/ (missing Keystone parameters)
https://review.openstack.org/#/c/139671/ (missing Neutron DVR
parameters)
We need to be very careful at this point not to continue merging things
into overcloud-source.yaml which don't have the equivalent bits for
overcloud-without-mergepy.yaml.
Dan
I'm not sure we can. get_file doesn't handle structured data and I don't
know what else we can do. Maybe we could split out all SoftwareConfig
resources to separate files (like Dan did in [nova config])? But the
SoftwareDeployments, nova servers, etc. have a different structure.
[nova config] https://review.openstack.org/#/c/130303/
3. What's the status of getting gating CI on the --no-mergepy templates?
Derek, can we add a job that's identical to
check-tripleo-ironic-overcloud-{f20,precise}-nonha except it passes
--no-mergepy to devtest.sh?
4. What barriers exist (now that I've implemented[2] the eliding
functionality
requested[3] for ResourceGroup) to moving to the --no-mergepy
implementation by default?
I'm about to post a patch that moves us from ResourceGroup to
AutoScalingGroup (for rolling updates), which is going to complicate
this a bit.
Barring that, I think you've identified all the requirements: CI job,
parity between the merge/non-merge templates and a process that
maintains it going forward (or puts the old ones in a maintanence-only
mode).
Anyone have anything else that's missing?
Thanks for any clarification you can provide! :)
Steve
[1] https://review.openstack.org/#/c/123100/
[2] https://review.openstack.org/#/c/128365/
[3] https://review.openstack.org/#/c/123713/
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] Managing no-mergepy template duplication
Excerpts from Steven Hardy's message of 2014-12-04 01:09:18 -0800: On Wed, Dec 03, 2014 at 06:54:48PM -0800, Clint Byrum wrote: Excerpts from Dan Prince's message of 2014-12-03 18:35:15 -0800: On Wed, 2014-12-03 at 10:11 +, Steven Hardy wrote: Hi all, Lately I've been spending more time looking at tripleo and doing some reviews. I'm particularly interested in helping the no-mergepy and subsequent puppet-software-config implementations mature (as well as improving overcloud updates via heat). Since Tomas's patch landed[1] to enable --no-mergepy in tripleo-heat-templates, it's become apparent that frequently patches are submitted which only update overcloud-source.yaml, so I've been trying to catch these and ask for a corresponding change to e.g controller.yaml. This raises the following questions: 1. Is it reasonable to -1 a patch and ask folks to update in both places? Yes! In fact until we abandon merge.py we shouldn't land anything that doesn't make the change in both places. Probably more important to make sure things go into the new (no-mergepy) templates though. 2. How are we going to handle this duplication and divergence? Move as quickly as possible to the new without-mergepy varients? That is my vote anyways. 3. What's the status of getting gating CI on the --no-mergepy templates? Devtest already supports it by simply setting an option (which sets an ENV variable). Just need to update tripleo-ci to do that and then make the switch. 4. What barriers exist (now that I've implemented[2] the eliding functionality requested[3] for ResourceGroup) to moving to the --no-mergepy implementation by default? None that I know of. I concur with Dan. Elide was the last reason not to use this. That's great news! :) One thing to consider is that there is no actual upgrade path from non-autoscaling-group based clouds, to auto-scaling-group based templates. We should consider how we'll do that before making it the default. So, I suggest we discuss possible upgrade paths and then move forward with switching one of the CI jobs to using the new templates. This is probably going to be really hard :( The sort of pattern which might work is: 1. Abandon mergepy based stack 2. Have helper script to reformat abandon data into nomergepy based adopt data 3. Adopt stack Unforunately there are several abandon/adopt bugs we'll have to fix if we decide this is the way to go (original author hasn't maintained it, but we can pick up the slack if it's on the critical path for TripleO). An alternative could be the external resource feature Angus is looking at: https://review.openstack.org/#/c/134848/ This would be more limited (we just reference rather than manage the existing resources), but potentially safer. The main risk here is import (or subsequent update) operations becoming destructive and replacing things, but I guess to some extent this is a risk with any change to tripleo-heat-templates. So you and I talked on IRC, but I want to socialize what we talked about more. The abandon/adopt pipeline is a bit broken in Heat and hasn't proven to be as useful as I'd hoped when it was first specced out. It seems too broad, and relies on any tools understanding how to morph a whole new format (the abandon json). With external_reference, the external upgrade process just needs to know how to morph the template. So if we're combining 8 existing servers into an autoscaling group, we just need to know how to make an autoscaling group with 8 servers as the external reference ids. This is, I think, the shortest path to a working solution, as I feel the external reference work in Heat is relatively straight forward and the spec has widescale agreement. There was another approach I mentioned, which is that we can teach Heat how to morph resources. So we could teach Heat that servers can be made into autoscaling groups, and vice-versa. This is a whole new feature though, and IMO, something that should be tackled _after_ we make it work with the external_reference feature, as this is basically a superset of what we'll do externally. Has any thought been given to upgrade CI testing? I'm thinking grenade or grenade-style testing here where we test maintaing a deployed overcloud over an upgrade of (some subset of) changes. I know the upgrade testing thing will be hard, but to me it's a key requirement to mature heat-driven updates vs those driven by external tooling. Upgrade testing is vital to the future of the project IMO. We really haven't validated the image based update method upstream yet. In Helion, we're using tripleo-ansible for updates, and that works great, but we need to get that or something similar into the pipeline for the gate, or every user who adopts will be left with a ton of work if they want to
Re: [openstack-dev] [tripleo] Managing no-mergepy template duplication
On Wed, Dec 03, 2014 at 06:54:48PM -0800, Clint Byrum wrote: Excerpts from Dan Prince's message of 2014-12-03 18:35:15 -0800: On Wed, 2014-12-03 at 10:11 +, Steven Hardy wrote: Hi all, Lately I've been spending more time looking at tripleo and doing some reviews. I'm particularly interested in helping the no-mergepy and subsequent puppet-software-config implementations mature (as well as improving overcloud updates via heat). Since Tomas's patch landed[1] to enable --no-mergepy in tripleo-heat-templates, it's become apparent that frequently patches are submitted which only update overcloud-source.yaml, so I've been trying to catch these and ask for a corresponding change to e.g controller.yaml. This raises the following questions: 1. Is it reasonable to -1 a patch and ask folks to update in both places? Yes! In fact until we abandon merge.py we shouldn't land anything that doesn't make the change in both places. Probably more important to make sure things go into the new (no-mergepy) templates though. 2. How are we going to handle this duplication and divergence? Move as quickly as possible to the new without-mergepy varients? That is my vote anyways. 3. What's the status of getting gating CI on the --no-mergepy templates? Devtest already supports it by simply setting an option (which sets an ENV variable). Just need to update tripleo-ci to do that and then make the switch. 4. What barriers exist (now that I've implemented[2] the eliding functionality requested[3] for ResourceGroup) to moving to the --no-mergepy implementation by default? None that I know of. I concur with Dan. Elide was the last reason not to use this. That's great news! :) One thing to consider is that there is no actual upgrade path from non-autoscaling-group based clouds, to auto-scaling-group based templates. We should consider how we'll do that before making it the default. So, I suggest we discuss possible upgrade paths and then move forward with switching one of the CI jobs to using the new templates. This is probably going to be really hard :( The sort of pattern which might work is: 1. Abandon mergepy based stack 2. Have helper script to reformat abandon data into nomergepy based adopt data 3. Adopt stack Unforunately there are several abandon/adopt bugs we'll have to fix if we decide this is the way to go (original author hasn't maintained it, but we can pick up the slack if it's on the critical path for TripleO). An alternative could be the external resource feature Angus is looking at: https://review.openstack.org/#/c/134848/ This would be more limited (we just reference rather than manage the existing resources), but potentially safer. The main risk here is import (or subsequent update) operations becoming destructive and replacing things, but I guess to some extent this is a risk with any change to tripleo-heat-templates. Has any thought been given to upgrade CI testing? I'm thinking grenade or grenade-style testing here where we test maintaing a deployed overcloud over an upgrade of (some subset of) changes. I know the upgrade testing thing will be hard, but to me it's a key requirement to mature heat-driven updates vs those driven by external tooling. Steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [tripleo] Managing no-mergepy template duplication
Hi all, Lately I've been spending more time looking at tripleo and doing some reviews. I'm particularly interested in helping the no-mergepy and subsequent puppet-software-config implementations mature (as well as improving overcloud updates via heat). Since Tomas's patch landed[1] to enable --no-mergepy in tripleo-heat-templates, it's become apparent that frequently patches are submitted which only update overcloud-source.yaml, so I've been trying to catch these and ask for a corresponding change to e.g controller.yaml. This raises the following questions: 1. Is it reasonable to -1 a patch and ask folks to update in both places? 2. How are we going to handle this duplication and divergence? 3. What's the status of getting gating CI on the --no-mergepy templates? 4. What barriers exist (now that I've implemented[2] the eliding functionality requested[3] for ResourceGroup) to moving to the --no-mergepy implementation by default? Thanks for any clarification you can provide! :) Steve [1] https://review.openstack.org/#/c/123100/ [2] https://review.openstack.org/#/c/128365/ [3] https://review.openstack.org/#/c/123713/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] Managing no-mergepy template duplication
On 12/03/2014 11:11 AM, Steven Hardy wrote:
Hi all,
Lately I've been spending more time looking at tripleo and doing some
reviews. I'm particularly interested in helping the no-mergepy and
subsequent puppet-software-config implementations mature (as well as
improving overcloud updates via heat).
Since Tomas's patch landed[1] to enable --no-mergepy in
tripleo-heat-templates, it's become apparent that frequently patches are
submitted which only update overcloud-source.yaml, so I've been trying to
catch these and ask for a corresponding change to e.g controller.yaml.
You beat me to this. Thanks for writing it up!
This raises the following questions:
1. Is it reasonable to -1 a patch and ask folks to update in both places?
I'm in favour.
2. How are we going to handle this duplication and divergence?
I'm not sure we can. get_file doesn't handle structured data and I don't
know what else we can do. Maybe we could split out all SoftwareConfig
resources to separate files (like Dan did in [nova config])? But the
SoftwareDeployments, nova servers, etc. have a different structure.
[nova config] https://review.openstack.org/#/c/130303/
3. What's the status of getting gating CI on the --no-mergepy templates?
Derek, can we add a job that's identical to
check-tripleo-ironic-overcloud-{f20,precise}-nonha except it passes
--no-mergepy to devtest.sh?
4. What barriers exist (now that I've implemented[2] the eliding functionality
requested[3] for ResourceGroup) to moving to the --no-mergepy
implementation by default?
I'm about to post a patch that moves us from ResourceGroup to
AutoScalingGroup (for rolling updates), which is going to complicate
this a bit.
Barring that, I think you've identified all the requirements: CI job,
parity between the merge/non-merge templates and a process that
maintains it going forward (or puts the old ones in a maintanence-only
mode).
Anyone have anything else that's missing?
Thanks for any clarification you can provide! :)
Steve
[1] https://review.openstack.org/#/c/123100/
[2] https://review.openstack.org/#/c/128365/
[3] https://review.openstack.org/#/c/123713/
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] Managing no-mergepy template duplication
On Wed, 2014-12-03 at 10:11 +, Steven Hardy wrote: Hi all, Lately I've been spending more time looking at tripleo and doing some reviews. I'm particularly interested in helping the no-mergepy and subsequent puppet-software-config implementations mature (as well as improving overcloud updates via heat). Since Tomas's patch landed[1] to enable --no-mergepy in tripleo-heat-templates, it's become apparent that frequently patches are submitted which only update overcloud-source.yaml, so I've been trying to catch these and ask for a corresponding change to e.g controller.yaml. This raises the following questions: 1. Is it reasonable to -1 a patch and ask folks to update in both places? Yes! In fact until we abandon merge.py we shouldn't land anything that doesn't make the change in both places. Probably more important to make sure things go into the new (no-mergepy) templates though. 2. How are we going to handle this duplication and divergence? Move as quickly as possible to the new without-mergepy varients? That is my vote anyways. 3. What's the status of getting gating CI on the --no-mergepy templates? Devtest already supports it by simply setting an option (which sets an ENV variable). Just need to update tripleo-ci to do that and then make the switch. 4. What barriers exist (now that I've implemented[2] the eliding functionality requested[3] for ResourceGroup) to moving to the --no-mergepy implementation by default? None that I know of. Thanks for any clarification you can provide! :) Steve [1] https://review.openstack.org/#/c/123100/ [2] https://review.openstack.org/#/c/128365/ [3] https://review.openstack.org/#/c/123713/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] Managing no-mergepy template duplication
Excerpts from Dan Prince's message of 2014-12-03 18:35:15 -0800: On Wed, 2014-12-03 at 10:11 +, Steven Hardy wrote: Hi all, Lately I've been spending more time looking at tripleo and doing some reviews. I'm particularly interested in helping the no-mergepy and subsequent puppet-software-config implementations mature (as well as improving overcloud updates via heat). Since Tomas's patch landed[1] to enable --no-mergepy in tripleo-heat-templates, it's become apparent that frequently patches are submitted which only update overcloud-source.yaml, so I've been trying to catch these and ask for a corresponding change to e.g controller.yaml. This raises the following questions: 1. Is it reasonable to -1 a patch and ask folks to update in both places? Yes! In fact until we abandon merge.py we shouldn't land anything that doesn't make the change in both places. Probably more important to make sure things go into the new (no-mergepy) templates though. 2. How are we going to handle this duplication and divergence? Move as quickly as possible to the new without-mergepy varients? That is my vote anyways. 3. What's the status of getting gating CI on the --no-mergepy templates? Devtest already supports it by simply setting an option (which sets an ENV variable). Just need to update tripleo-ci to do that and then make the switch. 4. What barriers exist (now that I've implemented[2] the eliding functionality requested[3] for ResourceGroup) to moving to the --no-mergepy implementation by default? None that I know of. I concur with Dan. Elide was the last reason not to use this. One thing to consider is that there is no actual upgrade path from non-autoscaling-group based clouds, to auto-scaling-group based templates. We should consider how we'll do that before making it the default. So, I suggest we discuss possible upgrade paths and then move forward with switching one of the CI jobs to using the new templates. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
