Re: [openstack-dev] [kolla] better solution for the non-ini format configure file

[Steven Dake (stdake)]

I looked more at this solution and am struggling to come up with a solution for:
<https://github.com/openstack/kolla/blob/master/ansible/roles/nova/tasks/config.yml#L69-L85>

https://github.com/openstack/kolla/blob/master/ansible/roles/nova/tasks/config.yml#L69-L85


That doesn't involve creating a task per file.


Any ideas?


Regards

-steve

From: Steven Dake mailto:std...@cisco.com>>
Reply-To: 
"openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>" 
mailto:openstack-dev@lists.openstack.org>>
Date: Thursday, June 23, 2016 at 6:07 AM
To: 
"openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>" 
mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [kolla] better solution for the non-ini format 
configure file

Looks like a really cool feature, and possibly a way to non-ini files, such as 
policy.json files which we just want to copy rather then override.

Kolla already has ini merging for ini files though and the example you provide 
is an ini file.  I like Kolla's ini merging, and it is sort of an external 
interface, since operators have been using it, so to remove it would mean 
following the deprecation cycle.  I do agree this would be fantastic for 
straight copies of non-ini configuration files.

Regards
-steve

From: OpenStack Mailing List Archive mailto:cor...@gmail.com>>
Reply-To: 
"openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>" 
mailto:openstack-dev@lists.openstack.org>>
Date: Wednesday, June 22, 2016 at 8:47 PM
To: 
"openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>" 
mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [kolla] better solution for the non-ini format 
configure file

Link: https://openstack.nimeyo.com/83165/?show=88496#a88496
From: AndrewLiu mailto:andrew@gmail.com>>


Recently, we find this feature of ansible:
http://docs.ansible.com/ansible/playbooks_loops.html#finding-first-matched-files<https://openstack.nimeyo.com/tag/finding-first-matched-files>

A specific path of template file can be add in the ansible task.

If a user want to customize an non-ini template file, the user can copy the 
template file to the customization directory, and modify the template file as 
the user wants.

An example of how to modify the ansible task:

change from:

- name: Copying over horizon.conf
  template:
  src: "{{ item }}.conf.j2"
  dest: "{{ node_config_directory }}/{{ item }}/{{ item }}.conf"
  with_items:
  - "horizon"


to:

- name: Copying over horizon.conf
  template:
  src: "{{ item }}"
  dest: "{{ node_config_directory }}/horizon/horizon.conf"
  with_first_found:
  - "{{ node_custom_config }}/horizon.conf.j2"
  - "horizon.conf.j2"


But a convention of how to organize the directory structure of customization 
template files should be addressed.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] better solution for the non-ini format configure file

[Steven Dake (stdake)]

Looks like a really cool feature, and possibly a way to non-ini files, such as 
policy.json files which we just want to copy rather then override.

Kolla already has ini merging for ini files though and the example you provide 
is an ini file.  I like Kolla's ini merging, and it is sort of an external 
interface, since operators have been using it, so to remove it would mean 
following the deprecation cycle.  I do agree this would be fantastic for 
straight copies of non-ini configuration files.

Regards
-steve

From: OpenStack Mailing List Archive mailto:cor...@gmail.com>>
Reply-To: 
"openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>" 
mailto:openstack-dev@lists.openstack.org>>
Date: Wednesday, June 22, 2016 at 8:47 PM
To: 
"openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>" 
mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [kolla] better solution for the non-ini format 
configure file

Link: https://openstack.nimeyo.com/83165/?show=88496#a88496
From: AndrewLiu mailto:andrew@gmail.com>>


Recently, we find this feature of ansible:
http://docs.ansible.com/ansible/playbooks_loops.html#finding-first-matched-files<https://openstack.nimeyo.com/tag/finding-first-matched-files>

A specific path of template file can be add in the ansible task.

If a user want to customize an non-ini template file, the user can copy the 
template file to the customization directory, and modify the template file as 
the user wants.

An example of how to modify the ansible task:

change from:

- name: Copying over horizon.conf
  template:
  src: "{{ item }}.conf.j2"
  dest: "{{ node_config_directory }}/{{ item }}/{{ item }}.conf"
  with_items:
  - "horizon"


to:

- name: Copying over horizon.conf
  template:
  src: "{{ item }}"
  dest: "{{ node_config_directory }}/horizon/horizon.conf"
  with_first_found:
  - "{{ node_custom_config }}/horizon.conf.j2"
  - "horizon.conf.j2"


But a convention of how to organize the directory structure of customization 
template files should be addressed.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] better solution for the non-ini format configure file

[OpenStack Mailing List Archive]

Link: https://openstack.nimeyo.com/83165/?show=88496#a88496
From: AndrewLiu 

Recently, we find this feature of ansible:http://docs.ansible.com/ansible/playbooks_loops.html#finding-first-matched-files

A specific path of template file can be add in the ansible task.

If a user want to customize an  non-ini template file, the user can copy the template file to the customization directory,  and modify the template file as the user wants.

An example of how to modify the ansible task:

change from:

- name: Copying over horizon.conf
  template:
  src: "{{ item }}.conf.j2"
  dest: "{{ node_config_directory }}/{{ item }}/{{ item }}.conf"
  with_items:
  - "horizon"


to:

- name: Copying over horizon.conf
  template:
  src: "{{ item }}"
  dest: "{{ node_config_directory }}/horizon/horizon.conf"
  with_first_found:
  - "{{ node_custom_config }}/horizon.conf.j2"
  - "horizon.conf.j2"


But a convention of how to organize the directory structure of customization template files should be addressed.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] better solution for the non-ini format configure file

[Steven Dake (stdake)]

On 5/5/16, 4:52 AM, "Paul Bourke"  wrote:

>TL;DR keep globals.yml to a minimum, customise configs via
>host_vars/group_vars
>
>It seems right now the "best" approach may be to tokenise variables as
>required. This is the approach we currently use in Oracle. There are two
>other approaches I can think of available to us:
>
>1) The overwrite mechanism as Jeffrey mentioned
>2) Make the merge_configs script modular so it can handle more formats
>than just ini
>
>The problem with 1) is that it is heavy weight for the Operator who
>wants to just customise one or two variables such as WEBROOT. Now they
>have to copy and maintain the entire config file.
>
>The problem with 2) is that I feel it's a burden on us to write and
>maintain code that can merge many different file formats. It could be
>done, though may potentially be outside the scope of the project.

I'm a fan of this option.  It is not outside the scope of our project to
maintain fundamental building blocks to execute various parts of our
software stack (in this case reconfigure).

Regards
-steve
>
>The tokenisation approach is unfortunately against what is described in
>"Kolla¹s Deployment Philosophy"[0] though the reality may be that this
>approach is the most Operator friendly. In regards to concern of
>globals.yml growing unmanageable, I feel globals.yml is overused and
>should only store the bare minimum. Service specific variables should be
>kept within their own role files (e.g. ansible/roles/horizon/defaults),
>and then documented which are available for tweaking via top level
>host_vars/group_vars. This is standard practice in other Ansible roles
>I've come across.
>
>-Paul
>
>[0] http://docs.openstack.org/developer/kolla/deployment-philosophy.html
>
>On 04/05/16 16:28, Mauricio Lima wrote:
>> I agree with your approach Jeffrey, although this is not ideal, this is
>> an approach already used in kolla.
>>
>> 2016-05-04 12:01 GMT-03:00 Jeffrey Zhang > >:
>>
>> Recently, Jack Ning pushed a PS[0], which export the `WEBROOT` to
>> the globals.yml file.
>> Because there is no chance to change the horizon/apache configure
>> file now.
>>
>> The root cause is that: Kolla do not support non-ini format
>> configure file. for the
>> ini-format file, we use a merge_config module[1] to merge all the
>> found file. But it
>> will be not work for configure file for apache, rabbitmq and so on.
>>
>> I would like to the current merge_config implementation. It is
>> directly and easy to use.
>> Not like the puppet, we have to remember the variable name defined
>> in the module. we have
>> no chance to add some user-defined variable.
>>
>> Export the variable to global is very bad and ugly. It will became a
>> disaster when more
>> and more variables is exported.
>>
>> So we should catch up a better solution to handle the configure
>>file.
>>
>> One solution I have is use overwrite mechanism. for example when
>> there is a file in
>> /etc/kolla/config/apache.conf, it will overwrite the templates in
>> the roles. But this
>> is still not ideal.
>>
>> Any body has better solution?
>>
>> [0] https://review.openstack.org/306928
>> [1]
>> 
>>http://git.openstack.org/cgit/openstack/kolla/tree/ansible/action_plugins
>>/merge_configs.py
>>
>> --
>> Regards,
>> Jeffrey Zhang
>> Blog: http://xcodest.me 
>>
>> 
>>_
>>_
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> 
>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>> 
>>_
>>_
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: 
>>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] better solution for the non-ini format configure file

[Paul Bourke]

TL;DR keep globals.yml to a minimum, customise configs via 
host_vars/group_vars


It seems right now the "best" approach may be to tokenise variables as 
required. This is the approach we currently use in Oracle. There are two 
other approaches I can think of available to us:


1) The overwrite mechanism as Jeffrey mentioned
2) Make the merge_configs script modular so it can handle more formats 
than just ini


The problem with 1) is that it is heavy weight for the Operator who 
wants to just customise one or two variables such as WEBROOT. Now they 
have to copy and maintain the entire config file.


The problem with 2) is that I feel it's a burden on us to write and 
maintain code that can merge many different file formats. It could be 
done, though may potentially be outside the scope of the project.


The tokenisation approach is unfortunately against what is described in 
"Kolla’s Deployment Philosophy"[0] though the reality may be that this 
approach is the most Operator friendly. In regards to concern of 
globals.yml growing unmanageable, I feel globals.yml is overused and 
should only store the bare minimum. Service specific variables should be 
kept within their own role files (e.g. ansible/roles/horizon/defaults), 
and then documented which are available for tweaking via top level 
host_vars/group_vars. This is standard practice in other Ansible roles 
I've come across.


-Paul

[0] http://docs.openstack.org/developer/kolla/deployment-philosophy.html

On 04/05/16 16:28, Mauricio Lima wrote:

I agree with your approach Jeffrey, although this is not ideal, this is
an approach already used in kolla.

2016-05-04 12:01 GMT-03:00 Jeffrey Zhang mailto:zhang.lei@gmail.com>>:

Recently, Jack Ning pushed a PS[0], which export the `WEBROOT` to
the globals.yml file.
Because there is no chance to change the horizon/apache configure
file now.

The root cause is that: Kolla do not support non-ini format
configure file. for the
ini-format file, we use a merge_config module[1] to merge all the
found file. But it
will be not work for configure file for apache, rabbitmq and so on.

I would like to the current merge_config implementation. It is
directly and easy to use.
Not like the puppet, we have to remember the variable name defined
in the module. we have
no chance to add some user-defined variable.

Export the variable to global is very bad and ugly. It will became a
disaster when more
and more variables is exported.

So we should catch up a better solution to handle the configure file.

One solution I have is use overwrite mechanism. for example when
there is a file in
/etc/kolla/config/apache.conf, it will overwrite the templates in
the roles. But this
is still not ideal.

Any body has better solution?

[0] https://review.openstack.org/306928
[1]

http://git.openstack.org/cgit/openstack/kolla/tree/ansible/action_plugins/merge_configs.py

--
Regards,
Jeffrey Zhang
Blog: http://xcodest.me 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] better solution for the non-ini format configure file

[Mauricio Lima]

I agree with your approach Jeffrey, although this is not ideal, this is an
approach already used in kolla.

2016-05-04 12:01 GMT-03:00 Jeffrey Zhang :

> Recently, Jack Ning pushed a PS[0], which export the `WEBROOT` to the
> globals.yml file.
> Because there is no chance to change the horizon/apache configure file now.
>
> The root cause is that: Kolla do not support non-ini format configure
> file. for the
> ini-format file, we use a merge_config module[1] to merge all the found
> file. But it
> will be not work for configure file for apache, rabbitmq and so on.
>
> I would like to the current merge_config implementation. It is directly
> and easy to use.
> Not like the puppet, we have to remember the variable name defined in the
> module. we have
> no chance to add some user-defined variable.
>
> Export the variable to global is very bad and ugly. It will became a
> disaster when more
> and more variables is exported.
>
> So we should catch up a better solution to handle the configure file.
>
> One solution I have is use overwrite mechanism. for example when there is
> a file in
> /etc/kolla/config/apache.conf, it will overwrite the templates in the
> roles. But this
> is still not ideal.
>
> Any body has better solution?
>
> [0] https://review.openstack.org/306928
> [1]
> http://git.openstack.org/cgit/openstack/kolla/tree/ansible/action_plugins/merge_configs.py
>
> --
> Regards,
> Jeffrey Zhang
> Blog: http://xcodest.me
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [kolla] better solution for the non-ini format configure file

[Jeffrey Zhang]

Recently, Jack Ning pushed a PS[0], which export the `WEBROOT` to the
globals.yml file.
Because there is no chance to change the horizon/apache configure file now.

The root cause is that: Kolla do not support non-ini format configure file.
for the
ini-format file, we use a merge_config module[1] to merge all the found
file. But it
will be not work for configure file for apache, rabbitmq and so on.

I would like to the current merge_config implementation. It is directly and
easy to use.
Not like the puppet, we have to remember the variable name defined in the
module. we have
no chance to add some user-defined variable.

Export the variable to global is very bad and ugly. It will became a
disaster when more
and more variables is exported.

So we should catch up a better solution to handle the configure file.

One solution I have is use overwrite mechanism. for example when there is a
file in
/etc/kolla/config/apache.conf, it will overwrite the templates in the
roles. But this
is still not ideal.

Any body has better solution?

[0] https://review.openstack.org/306928
[1]
http://git.openstack.org/cgit/openstack/kolla/tree/ansible/action_plugins/merge_configs.py

-- 
Regards,
Jeffrey Zhang
Blog: http://xcodest.me
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev