Re: [openstack-dev] [TripleO] An experiment with Ansible

[James Slagle]

On Mon, Jul 24, 2017 at 3:12 AM, Marios Andreou  wrote:
>
>
> On Fri, Jul 21, 2017 at 1:21 AM, James Slagle 
> wrote:
>>
>> Following up on the previous thread:
>> http://lists.openstack.org/pipermail/openstack-dev/2017-July/119405.html
>>
>> I wanted to share some work I did around the prototype I mentioned
>> there. I spent a couple days exploring this idea. I came up with a
>> Python script that when run against an in progress Heat stack, will
>> pull all the server and deployment metadata out of Heat and generate
>> ansible playbooks/tasks from the deployments.
>>
>> Here's the code:
>> https://github.com/slagle/pump
>>
>> And an example of what gets generated:
>> https://gist.github.com/slagle/433ea1bdca7e026ce8ab2c46f4d716a8
>>
>> If you're interested in any more detail, let me know.
>>
>> It signals the stack to completion with a dummy "ok" signal so that
>> the stack will complete. You can then use ansible-playbook to apply
>> the actual deloyments (in the expected order, respecting the steps
>> across all roles, and in parallel across all the roles).
>>
>> Effectively, this treats Heat as nothing but a yaml cruncher. When
>> using it with deployed-server, Heat doesn't actually change anything
>> on an overcloud node, you're only using it to generate ansible.
>
>
>
> Hi James,
>
> FYI this actually describes the current plan for Pike minor update [1] - the
> idea is to use the "openstack overcloud config download " (matbu++) to write
> the playbooks for each node from the deployed stack outputs. The minor
> update playbook(s) itself will be generated from new 'update_tasks' added to
> each of the service manifests (akin to the current upgrade_tasks). The plan
> is to disable the actual service config deployment steps so that we just get
> the stack outputs for the playbook generation.
>
> The effort is lead by shardy and he has posted reviews/comments on the
> etherpad @ [1] FYI (I know he is away this week so may not respond ++ I was
> struck by the similarity between what you described above and the consensus
> we seemed to reach towards the end of the week about the minor update plan,
> so I thought you and others may be interested to hear it).

Yes, I've been looking at that work as well. I'm not entirely sure
what the longer term goals are, although I like the approach we are
taking with updates. Looking at the patches that have been posted so
far, I'm not sure if they are meant to be Docker/container specific
only, or if they would work with the puppet services as well or any
SoftwareConfig group type.

I've pulled all the patches locally and was actually testing with a
puppet only stack for initial deployment (no stack update to
Containers), and the generated config/playbooks are not correct
(trying to do something with containers when they shouldn't).

I'm not sure if that is intended to work or if there is a bug. I can
check with shardy when he returns what the goals and further context
around that approach are.

I think one of the primary differences between that approach and what
I prototyped was that my goal was to completely eliminate the
os-collect-config -> Heat metadata Deployment "transport" for any
SoftwareConfig group type (puppet, script, hiera, ansible). IME, that
has been one of the most difficult aspects of TripleO for users and
operators to reason about, reproduce, troubleshoot, and understand.

An additional goal is to see if it would be possible to do that
entirely external to Heat and/or tripleo-heat-templates. Just
considering all the reviews that are currently in progress for the
"config download" approach, there is a lot of refactoring, output
changes, and yaql churning in tripleo-heat-templates.

Certainly the approaches are similar, and could even co-exist,
although they are tackling the problem from different angles.

>
> Your review @ /#/c/485303/ is slightly different in that it doesn't disable
> the deployment/postdeploy steps but signals completion to Heat. Haven't
> checked that review in  detail but first concern is can/do you catch it in
> time... I mean you start the heat stack update and have to immediately call
> the openstack overcloud signal , if I understood correctly?

Yes, you'd have to signal the deployments before they time out on the Heat side.

You could also configure the signal_transport to NO_SIGNAL, in which
case Heat would just create the stack to completion without waiting
(and thus possibly timing out) for any signals.





-- 
-- James Slagle
--

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] An experiment with Ansible

[Marios Andreou]

On Fri, Jul 21, 2017 at 1:21 AM, James Slagle 
wrote:

> Following up on the previous thread:
> http://lists.openstack.org/pipermail/openstack-dev/2017-July/119405.html
>
> I wanted to share some work I did around the prototype I mentioned
> there. I spent a couple days exploring this idea. I came up with a
> Python script that when run against an in progress Heat stack, will
> pull all the server and deployment metadata out of Heat and generate
> ansible playbooks/tasks from the deployments.
>
> Here's the code:
> https://github.com/slagle/pump
>
> And an example of what gets generated:
> https://gist.github.com/slagle/433ea1bdca7e026ce8ab2c46f4d716a8
>
> If you're interested in any more detail, let me know.
>
> It signals the stack to completion with a dummy "ok" signal so that
> the stack will complete. You can then use ansible-playbook to apply
> the actual deloyments (in the expected order, respecting the steps
> across all roles, and in parallel across all the roles).
>
> Effectively, this treats Heat as nothing but a yaml cruncher. When
> using it with deployed-server, Heat doesn't actually change anything
> on an overcloud node, you're only using it to generate ansible.
>


Hi James,

FYI this actually describes the current plan for Pike minor update [1] -
the idea is to use the "openstack overcloud config download " (matbu++) to
write the playbooks for each node from the deployed stack outputs. The
minor update playbook(s) itself will be generated from new 'update_tasks'
added to each of the service manifests (akin to the current upgrade_tasks).
The plan is to disable the actual service config deployment steps so that
we just get the stack outputs for the playbook generation.

The effort is lead by shardy and he has posted reviews/comments on the
etherpad @ [1] FYI (I know he is away this week so may not respond ++ I was
struck by the similarity between what you described above and the consensus
we seemed to reach towards the end of the week about the minor update plan,
so I thought you and others may be interested to hear it).

Your review @ /#/c/485303/ is slightly different in that it doesn't disable
the deployment/postdeploy steps but signals completion to Heat. Haven't
checked that review in  detail but first concern is can/do you catch it in
time... I mean you start the heat stack update and have to immediately call
the openstack overcloud signal , if I understood correctly?

thanks, marios

[1] https://etherpad.openstack.org/p/tripleo-pike-updates-upgrades



>
> Honestly, I think I will prefer the longer term approach of using
> stack outputs. Although, I am not sure of the end goal of that work
> and if it is the same as this prototype.
>
> And some of what I've done may be useful with that approach as well:
> https://review.openstack.org/#/c/485303/
>
> However, I found this prototype interesting and worth exploring for a
> couple of reasons:
>
> Regardless of the approach we take, I wanted to explore what an end
> result might look like. Personally, this illustrates what I kind of
> had in mind for an "end goal".
>
> I also wanted to see if this was at all feasible. I envisioned some
> hurdles, such as deployments depending on output values of previous
> deployments, but we actually only do that in 1 place in
> tripleo-heat-templates, and I was able to workaround that. In the end
> I used it to deploy an all in one overcloud equivalent to our
> multinode CI job, so I believe it's feasible.
>
> It meets most of the requirements we're looking to get out of ansible.
> You can (re)apply just a single deployment, or a given deployment
> across all ResourceGroup members, or all deployments for a given
> server(s), it's easy to see what failed and for what servers, etc.
>
> FInally, It's something we could deliver  without much (any?) change
> in tripleo-heat-templates. Although I'm not trying to say it'd be a
> small amount of work to even do that, as this is a very rough
> prototype.
>
> --
> -- James Slagle
> --
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] An experiment with Ansible

[Paul Belanger]

On Thu, Jul 20, 2017 at 10:06:00PM -0400, James Slagle wrote:
> On Thu, Jul 20, 2017 at 9:52 PM, James Slagle  wrote:
> > On Thu, Jul 20, 2017 at 9:04 PM, Paul Belanger  
> > wrote:
> >> On Thu, Jul 20, 2017 at 06:21:22PM -0400, James Slagle wrote:
> >>> Following up on the previous thread:
> >>> http://lists.openstack.org/pipermail/openstack-dev/2017-July/119405.html
> >>>
> >>> I wanted to share some work I did around the prototype I mentioned
> >>> there. I spent a couple days exploring this idea. I came up with a
> >>> Python script that when run against an in progress Heat stack, will
> >>> pull all the server and deployment metadata out of Heat and generate
> >>> ansible playbooks/tasks from the deployments.
> >>>
> >>> Here's the code:
> >>> https://github.com/slagle/pump
> >>>
> >>> And an example of what gets generated:
> >>> https://gist.github.com/slagle/433ea1bdca7e026ce8ab2c46f4d716a8
> >>>
> >>> If you're interested in any more detail, let me know.
> >>>
> >>> It signals the stack to completion with a dummy "ok" signal so that
> >>> the stack will complete. You can then use ansible-playbook to apply
> >>> the actual deloyments (in the expected order, respecting the steps
> >>> across all roles, and in parallel across all the roles).
> >>>
> >>> Effectively, this treats Heat as nothing but a yaml cruncher. When
> >>> using it with deployed-server, Heat doesn't actually change anything
> >>> on an overcloud node, you're only using it to generate ansible.
> >>>
> >>> Honestly, I think I will prefer the longer term approach of using
> >>> stack outputs. Although, I am not sure of the end goal of that work
> >>> and if it is the same as this prototype.
> >>>
> >> Sorry if this hasn't been asked before but why don't you removed all of 
> >> your
> >> ansible-playbook logic out of heat and write them directly as native 
> >> playbooks /
> >> roles? Then instead of having a tool that reads heat to then generate the
> >> playbooks / roles, you update heat just to directly call the playbooks? Any
> >> dynamic information about be stored in the inventory or using the 
> >> --extra-vars
> >> on the CLI?
> >
> > We must maintain backwards compatibility with our existing Heat based
> > interfaces (cli, api, templates). While that could probably be done
> > with the approach you mention, it feels like it would be much more
> > difficult to do so in that you'd need to effectively add back on the
> > compatibility layer once the new pristine native ansible
> > playbooks/roles were written. And it seems like it would be quite a
> > lot of heat template work to translate existing interfaces to call
> > into the new playbooks.
> >
> > Even then, any new playbooks written from scratch would have to be
> > flexible enough to accommodate the old interfaces. On the surface, it
> > feels like you may end up sacrificing a lot of your goals in your
> > playbooks so you can maintain backwards compatibility anyways.
> >
> > The existing interface must be the first class citizen. We can't break
> > those contracts, so we need ways to quickly iterate towards ansible.
> > Writing all new native playbooks sounds like just writing a new
> > OpenStack installer to me, and then making Heat call that so that it's
> > backwards compatible.
> >
> > The focus on the interface flips that around so that you use existing
> > systems and iterate them towards the end goal. Just my POV.
> >
> > FYI, there are other ongoing solutions as well such as existing
> > ansible tasks directly in the templates today. These are much easier
> > to reason about when it comes to generating the roles and playbooks,
> > because it is direct Ansible syntax in the templates, so it's easier
> > to see the origin of tasks and make changes.
> 
> I also wanted to mention that the Ansible tasks in the templates today
> could be included with Heat's get_file function. In which case, as a
> template developer you basically are writing a native Ansible tasks
> file that could be included in an Ansible role.
> 
Right, that's what I was trying to say. I believe you have some ansible logic
within your heat templates today. As I understand it, dynamically generated. I
would think first moving that logic outside of heat into stand alone playbooks
would be a good first step. Then you wouldn't need to write the tool from above
right?

I'm not too familiar with all the lays in tripleo, so it is possible I am wrong.

> The generation would still come into play when combining the tasks
> into the role/playbook that is actually applied to a given server,
> since that is all dynamic based on user input.
> 
Are you running a single ansible-playbook command for each task or multiple
ansible-playbook runs?

> -- 
> -- James Slagle
> --
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 

Re: [openstack-dev] [TripleO] An experiment with Ansible

[Giulio Fidente]

On 07/21/2017 03:04 AM, Paul Belanger wrote:
> On Thu, Jul 20, 2017 at 06:21:22PM -0400, James Slagle wrote:

[...]

> Sorry if this hasn't been asked before but why don't you removed all of your
> ansible-playbook logic out of heat and write them directly as native 
> playbooks /
> roles? Then instead of having a tool that reads heat to then generate the
> playbooks / roles, you update heat just to directly call the playbooks? Any
> dynamic information about be stored in the inventory or using the --extra-vars
> on the CLI?
this is actually what the heat > mistral triggering mechansim implements

Heat provides the extra_vars and triggers the native ceph-ansible
playbook at the right time during the deployment

I believe Steven's goal in [1] is to prepare the data to build the
ansible inventory group vars from the role data which seems to me to go
in the same direction that James is proposing of driving the deployment
steps without the Heat software deployments

If from a service template we could point to a playbook, instead of
describing one in Heat, this might replace the Heat > Mistral calling too

Yet, there is another question we probably want to discuss: is this just
so we use ansible to drive the deployment steps or do we want ansible to
replace puppet for the overcloud configuration?

1. https://review.openstack.org/#/c/483929/
-- 
Giulio Fidente
GPG KEY: 08D733BA

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] An experiment with Ansible

[James Slagle]

On Thu, Jul 20, 2017 at 9:52 PM, James Slagle  wrote:
> On Thu, Jul 20, 2017 at 9:04 PM, Paul Belanger  wrote:
>> On Thu, Jul 20, 2017 at 06:21:22PM -0400, James Slagle wrote:
>>> Following up on the previous thread:
>>> http://lists.openstack.org/pipermail/openstack-dev/2017-July/119405.html
>>>
>>> I wanted to share some work I did around the prototype I mentioned
>>> there. I spent a couple days exploring this idea. I came up with a
>>> Python script that when run against an in progress Heat stack, will
>>> pull all the server and deployment metadata out of Heat and generate
>>> ansible playbooks/tasks from the deployments.
>>>
>>> Here's the code:
>>> https://github.com/slagle/pump
>>>
>>> And an example of what gets generated:
>>> https://gist.github.com/slagle/433ea1bdca7e026ce8ab2c46f4d716a8
>>>
>>> If you're interested in any more detail, let me know.
>>>
>>> It signals the stack to completion with a dummy "ok" signal so that
>>> the stack will complete. You can then use ansible-playbook to apply
>>> the actual deloyments (in the expected order, respecting the steps
>>> across all roles, and in parallel across all the roles).
>>>
>>> Effectively, this treats Heat as nothing but a yaml cruncher. When
>>> using it with deployed-server, Heat doesn't actually change anything
>>> on an overcloud node, you're only using it to generate ansible.
>>>
>>> Honestly, I think I will prefer the longer term approach of using
>>> stack outputs. Although, I am not sure of the end goal of that work
>>> and if it is the same as this prototype.
>>>
>> Sorry if this hasn't been asked before but why don't you removed all of your
>> ansible-playbook logic out of heat and write them directly as native 
>> playbooks /
>> roles? Then instead of having a tool that reads heat to then generate the
>> playbooks / roles, you update heat just to directly call the playbooks? Any
>> dynamic information about be stored in the inventory or using the 
>> --extra-vars
>> on the CLI?
>
> We must maintain backwards compatibility with our existing Heat based
> interfaces (cli, api, templates). While that could probably be done
> with the approach you mention, it feels like it would be much more
> difficult to do so in that you'd need to effectively add back on the
> compatibility layer once the new pristine native ansible
> playbooks/roles were written. And it seems like it would be quite a
> lot of heat template work to translate existing interfaces to call
> into the new playbooks.
>
> Even then, any new playbooks written from scratch would have to be
> flexible enough to accommodate the old interfaces. On the surface, it
> feels like you may end up sacrificing a lot of your goals in your
> playbooks so you can maintain backwards compatibility anyways.
>
> The existing interface must be the first class citizen. We can't break
> those contracts, so we need ways to quickly iterate towards ansible.
> Writing all new native playbooks sounds like just writing a new
> OpenStack installer to me, and then making Heat call that so that it's
> backwards compatible.
>
> The focus on the interface flips that around so that you use existing
> systems and iterate them towards the end goal. Just my POV.
>
> FYI, there are other ongoing solutions as well such as existing
> ansible tasks directly in the templates today. These are much easier
> to reason about when it comes to generating the roles and playbooks,
> because it is direct Ansible syntax in the templates, so it's easier
> to see the origin of tasks and make changes.

I also wanted to mention that the Ansible tasks in the templates today
could be included with Heat's get_file function. In which case, as a
template developer you basically are writing a native Ansible tasks
file that could be included in an Ansible role.

The generation would still come into play when combining the tasks
into the role/playbook that is actually applied to a given server,
since that is all dynamic based on user input.

-- 
-- James Slagle
--

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] An experiment with Ansible

[James Slagle]

On Thu, Jul 20, 2017 at 9:04 PM, Paul Belanger  wrote:
> On Thu, Jul 20, 2017 at 06:21:22PM -0400, James Slagle wrote:
>> Following up on the previous thread:
>> http://lists.openstack.org/pipermail/openstack-dev/2017-July/119405.html
>>
>> I wanted to share some work I did around the prototype I mentioned
>> there. I spent a couple days exploring this idea. I came up with a
>> Python script that when run against an in progress Heat stack, will
>> pull all the server and deployment metadata out of Heat and generate
>> ansible playbooks/tasks from the deployments.
>>
>> Here's the code:
>> https://github.com/slagle/pump
>>
>> And an example of what gets generated:
>> https://gist.github.com/slagle/433ea1bdca7e026ce8ab2c46f4d716a8
>>
>> If you're interested in any more detail, let me know.
>>
>> It signals the stack to completion with a dummy "ok" signal so that
>> the stack will complete. You can then use ansible-playbook to apply
>> the actual deloyments (in the expected order, respecting the steps
>> across all roles, and in parallel across all the roles).
>>
>> Effectively, this treats Heat as nothing but a yaml cruncher. When
>> using it with deployed-server, Heat doesn't actually change anything
>> on an overcloud node, you're only using it to generate ansible.
>>
>> Honestly, I think I will prefer the longer term approach of using
>> stack outputs. Although, I am not sure of the end goal of that work
>> and if it is the same as this prototype.
>>
> Sorry if this hasn't been asked before but why don't you removed all of your
> ansible-playbook logic out of heat and write them directly as native 
> playbooks /
> roles? Then instead of having a tool that reads heat to then generate the
> playbooks / roles, you update heat just to directly call the playbooks? Any
> dynamic information about be stored in the inventory or using the --extra-vars
> on the CLI?

We must maintain backwards compatibility with our existing Heat based
interfaces (cli, api, templates). While that could probably be done
with the approach you mention, it feels like it would be much more
difficult to do so in that you'd need to effectively add back on the
compatibility layer once the new pristine native ansible
playbooks/roles were written. And it seems like it would be quite a
lot of heat template work to translate existing interfaces to call
into the new playbooks.

Even then, any new playbooks written from scratch would have to be
flexible enough to accommodate the old interfaces. On the surface, it
feels like you may end up sacrificing a lot of your goals in your
playbooks so you can maintain backwards compatibility anyways.

The existing interface must be the first class citizen. We can't break
those contracts, so we need ways to quickly iterate towards ansible.
Writing all new native playbooks sounds like just writing a new
OpenStack installer to me, and then making Heat call that so that it's
backwards compatible.

The focus on the interface flips that around so that you use existing
systems and iterate them towards the end goal. Just my POV.

FYI, there are other ongoing solutions as well such as existing
ansible tasks directly in the templates today. These are much easier
to reason about when it comes to generating the roles and playbooks,
because it is direct Ansible syntax in the templates, so it's easier
to see the origin of tasks and make changes.

As we move forward on these approaches if we end up with users
gravitating towards certain usage patterns, I think we'd consider
deprecating interfaces that are no longer seen as useful.

>
> Basically, we do this for zuulv2.5 today in openstack-infra (dynamically
> generate playbooks at run-time) and it is a large amount of work to debug
> issues.  In our case, we did it to quickly migrate from jenkins to ansible
> (since zuulv3 completely fixes this with native playbooks) and I wouldn't
> recommend it to operators to do.  Not fun.

I'm not familiar with the technical reasoning there, but on the
surface it sounds similar to what some of our goals may be. We want to
quickly add some Ansible features and move in that direction.

We don't want to write all new roles and playbooks in Ansible, at
least I don't :) We can't even say definitively right now that native
Ansible is the interface we want long term. So I think it would be
premature to approach the problem from the angle of writing new native
playbooks.

Whether or not it ever becomes a full migration is as I said TBD, and
not even something we have to decide now. The decision would be more
driven by how people end up using it.

-- 
-- James Slagle
--

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] An experiment with Ansible

[Paul Belanger]

On Thu, Jul 20, 2017 at 06:21:22PM -0400, James Slagle wrote:
> Following up on the previous thread:
> http://lists.openstack.org/pipermail/openstack-dev/2017-July/119405.html
> 
> I wanted to share some work I did around the prototype I mentioned
> there. I spent a couple days exploring this idea. I came up with a
> Python script that when run against an in progress Heat stack, will
> pull all the server and deployment metadata out of Heat and generate
> ansible playbooks/tasks from the deployments.
> 
> Here's the code:
> https://github.com/slagle/pump
> 
> And an example of what gets generated:
> https://gist.github.com/slagle/433ea1bdca7e026ce8ab2c46f4d716a8
> 
> If you're interested in any more detail, let me know.
> 
> It signals the stack to completion with a dummy "ok" signal so that
> the stack will complete. You can then use ansible-playbook to apply
> the actual deloyments (in the expected order, respecting the steps
> across all roles, and in parallel across all the roles).
> 
> Effectively, this treats Heat as nothing but a yaml cruncher. When
> using it with deployed-server, Heat doesn't actually change anything
> on an overcloud node, you're only using it to generate ansible.
> 
> Honestly, I think I will prefer the longer term approach of using
> stack outputs. Although, I am not sure of the end goal of that work
> and if it is the same as this prototype.
> 
Sorry if this hasn't been asked before but why don't you removed all of your
ansible-playbook logic out of heat and write them directly as native playbooks /
roles? Then instead of having a tool that reads heat to then generate the
playbooks / roles, you update heat just to directly call the playbooks? Any
dynamic information about be stored in the inventory or using the --extra-vars
on the CLI?

Basically, we do this for zuulv2.5 today in openstack-infra (dynamically
generate playbooks at run-time) and it is a large amount of work to debug
issues.  In our case, we did it to quickly migrate from jenkins to ansible
(since zuulv3 completely fixes this with native playbooks) and I wouldn't
recommend it to operators to do.  Not fun.

> And some of what I've done may be useful with that approach as well:
> https://review.openstack.org/#/c/485303/
> 
> However, I found this prototype interesting and worth exploring for a
> couple of reasons:
> 
> Regardless of the approach we take, I wanted to explore what an end
> result might look like. Personally, this illustrates what I kind of
> had in mind for an "end goal".
> 
> I also wanted to see if this was at all feasible. I envisioned some
> hurdles, such as deployments depending on output values of previous
> deployments, but we actually only do that in 1 place in
> tripleo-heat-templates, and I was able to workaround that. In the end
> I used it to deploy an all in one overcloud equivalent to our
> multinode CI job, so I believe it's feasible.
> 
> It meets most of the requirements we're looking to get out of ansible.
> You can (re)apply just a single deployment, or a given deployment
> across all ResourceGroup members, or all deployments for a given
> server(s), it's easy to see what failed and for what servers, etc.
> 
> FInally, It's something we could deliver  without much (any?) change
> in tripleo-heat-templates. Although I'm not trying to say it'd be a
> small amount of work to even do that, as this is a very rough
> prototype.
> 
> -- 
> -- James Slagle
> --
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [TripleO] An experiment with Ansible

[James Slagle]

Following up on the previous thread:
http://lists.openstack.org/pipermail/openstack-dev/2017-July/119405.html

I wanted to share some work I did around the prototype I mentioned
there. I spent a couple days exploring this idea. I came up with a
Python script that when run against an in progress Heat stack, will
pull all the server and deployment metadata out of Heat and generate
ansible playbooks/tasks from the deployments.

Here's the code:
https://github.com/slagle/pump

And an example of what gets generated:
https://gist.github.com/slagle/433ea1bdca7e026ce8ab2c46f4d716a8

If you're interested in any more detail, let me know.

It signals the stack to completion with a dummy "ok" signal so that
the stack will complete. You can then use ansible-playbook to apply
the actual deloyments (in the expected order, respecting the steps
across all roles, and in parallel across all the roles).

Effectively, this treats Heat as nothing but a yaml cruncher. When
using it with deployed-server, Heat doesn't actually change anything
on an overcloud node, you're only using it to generate ansible.

Honestly, I think I will prefer the longer term approach of using
stack outputs. Although, I am not sure of the end goal of that work
and if it is the same as this prototype.

And some of what I've done may be useful with that approach as well:
https://review.openstack.org/#/c/485303/

However, I found this prototype interesting and worth exploring for a
couple of reasons:

Regardless of the approach we take, I wanted to explore what an end
result might look like. Personally, this illustrates what I kind of
had in mind for an "end goal".

I also wanted to see if this was at all feasible. I envisioned some
hurdles, such as deployments depending on output values of previous
deployments, but we actually only do that in 1 place in
tripleo-heat-templates, and I was able to workaround that. In the end
I used it to deploy an all in one overcloud equivalent to our
multinode CI job, so I believe it's feasible.

It meets most of the requirements we're looking to get out of ansible.
You can (re)apply just a single deployment, or a given deployment
across all ResourceGroup members, or all deployments for a given
server(s), it's easy to see what failed and for what servers, etc.

FInally, It's something we could deliver  without much (any?) change
in tripleo-heat-templates. Although I'm not trying to say it'd be a
small amount of work to even do that, as this is a very rough
prototype.

-- 
-- James Slagle
--

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev