Re: [openstack-dev] [puppet] Proposal to configure Oslo libraries

[Ben Nemec]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

I don't know much about the puppet project organization so I won't
comment on whether 1 or 2 is better, but a big +1 to having a common
way to configure Oslo opts.  Consistency of those options across all
services is one of the big reasons we pushed so hard for the libraries
to own their option definitions, so this would align well with the way
the projects are consumed.

- -Ben

On 05/07/2015 03:19 PM, Emilien Macchi wrote:
> Hi,
> 
> I think one of the biggest challenges working on Puppet OpenStack 
> modules is to keep code consistency across all our modules (~20). 
> If you've read the code, you'll see there is some differences
> between RabbitMQ configuration/parameters in some modules and this
> is because we did not have the right tools to make it properly. A
> lot of the duplicated code we have comes from Oslo libraries 
> configuration.
> 
> Now, I come up with an idea and two proposals.
> 
> Idea 
> 
> We could have some defined types to configure oslo sections in
> OpenStack configuration files.
> 
> Something like: define oslo::messaging::rabbitmq( $user, $password 
> ) { ensure_resource($name, 'oslo_messaging_rabbit/rabbit_userid',
> {'value' => $user}) ... }
> 
> Usage in puppet-nova: ::oslo::messaging::rabbitmq{'nova_config': 
> user => 'nova', password => 'secrete', }
> 
> And patch all our modules to consume these defines and finally
> have consistency at the way we configure Oslo projects (messaging,
> logging, etc).
> 
> Proposals =
> 
> #1 Creating puppet-oslo ... and having oslo::messaging::rabbitmq,
> oslo::messaging::qpid, ..., oslo::logging, etc. This module will be
> used only to configure actual Oslo libraries when we deploy
> OpenStack. To me, this solution is really consistent with how 
> OpenStack works today and is scalable as soon we contribute Oslo 
> configuration changes in this module.
> 
> #2 Using puppet-openstacklib ... and having
> openstacklib::oslo::messaging::(...) A good thing is our modules
> already use openstacklib. But openstacklib does not configure
> OpenStack now, it creates some common defines & classes that are
> consumed in other modules.
> 
> 
> I personally prefer #1 because: * it's consistent with OpenStack. *
> I don't want openstacklib the repo where we put everything common.
> We have to differentiate *common-in-OpenStack* and
> *common-in-our-modules*. I think openstacklib should continue to be
> used for common things in our modules, like providers, wrappers,
> database management, etc. But to configure common OpenStack bits
> (aka Oslo©), we might want to create puppet-oslo.
> 
> As usual, any thoughts are welcome,
> 
> Best,
> 
> 
> 
> __

>
> 
OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQEcBAEBCAAGBQJVS9HMAAoJEDehGd0Fy7uq24sH/j/ctaGaNbGdxyRCfBatIPbU
Vk810yyMYzNH67s4Ku8LsEKvqMAoToEtnq/84ZXiUGUH65PtwGm9e6Nb54tkIHTE
tPVjQSePC7omn97M5A4tb94b6h0TaLxWT+0oZjnto1Lk+/Q1tCYgCySClyF/CsmM
2CZvHRqRKWG1ytWhJuYrjymury4Xfgpcwt7MA69Nqun/7fwjSgFvvVdfVlln6VI+
2Nx4AIFDyXVafvN7ZBIGkyqrWRsmyht3elvJg5JtxSu8gQbf3LVgbkTLREUccHDA
07/edo00ouAHMhKyYdvFimmjqr6gom5OqmpQqiw8TsFqFUDEXunTVil/v5W1dL8=
=B85A
-END PGP SIGNATURE-

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] Proposal to configure Oslo libraries

[Doug Hellmann]

Excerpts from Ben Nemec's message of 2015-05-07 15:57:48 -0500:
> I don't know much about the puppet project organization so I won't
> comment on whether 1 or 2 is better, but a big +1 to having a common
> way to configure Oslo opts.  Consistency of those options across all
> services is one of the big reasons we pushed so hard for the libraries
> to own their option definitions, so this would align well with the way
> the projects are consumed.
> 
> - -Ben

Well said, Ben.

Doug

> 
> On 05/07/2015 03:19 PM, Emilien Macchi wrote:
> > Hi,
> > 
> > I think one of the biggest challenges working on Puppet OpenStack 
> > modules is to keep code consistency across all our modules (~20). 
> > If you've read the code, you'll see there is some differences
> > between RabbitMQ configuration/parameters in some modules and this
> > is because we did not have the right tools to make it properly. A
> > lot of the duplicated code we have comes from Oslo libraries 
> > configuration.
> > 
> > Now, I come up with an idea and two proposals.
> > 
> > Idea 
> > 
> > We could have some defined types to configure oslo sections in
> > OpenStack configuration files.
> > 
> > Something like: define oslo::messaging::rabbitmq( $user, $password 
> > ) { ensure_resource($name, 'oslo_messaging_rabbit/rabbit_userid',
> > {'value' => $user}) ... }
> > 
> > Usage in puppet-nova: ::oslo::messaging::rabbitmq{'nova_config': 
> > user => 'nova', password => 'secrete', }
> > 
> > And patch all our modules to consume these defines and finally
> > have consistency at the way we configure Oslo projects (messaging,
> > logging, etc).
> > 
> > Proposals =
> > 
> > #1 Creating puppet-oslo ... and having oslo::messaging::rabbitmq,
> > oslo::messaging::qpid, ..., oslo::logging, etc. This module will be
> > used only to configure actual Oslo libraries when we deploy
> > OpenStack. To me, this solution is really consistent with how 
> > OpenStack works today and is scalable as soon we contribute Oslo 
> > configuration changes in this module.
> > 
> > #2 Using puppet-openstacklib ... and having
> > openstacklib::oslo::messaging::(...) A good thing is our modules
> > already use openstacklib. But openstacklib does not configure
> > OpenStack now, it creates some common defines & classes that are
> > consumed in other modules.
> > 
> > 
> > I personally prefer #1 because: * it's consistent with OpenStack. *
> > I don't want openstacklib the repo where we put everything common.
> > We have to differentiate *common-in-OpenStack* and
> > *common-in-our-modules*. I think openstacklib should continue to be
> > used for common things in our modules, like providers, wrappers,
> > database management, etc. But to configure common OpenStack bits
> > (aka Oslo©), we might want to create puppet-oslo.
> > 
> > As usual, any thoughts are welcome,
> > 
> > Best,
> > 
> > 
> > 
> > __
> 
> >
> > 
> OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > 
> 
> -BEGIN PGP SIGNATURE-
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] Proposal to configure Oslo libraries

[Rich Megginson]

On 05/08/2015 07:17 AM, Doug Hellmann wrote:

Excerpts from Ben Nemec's message of 2015-05-07 15:57:48 -0500:

I don't know much about the puppet project organization so I won't
comment on whether 1 or 2 is better, but a big +1 to having a common
way to configure Oslo opts.  Consistency of those options across all
services is one of the big reasons we pushed so hard for the libraries
to own their option definitions, so this would align well with the way
the projects are consumed.

- -Ben

Well said, Ben.

Doug


On 05/07/2015 03:19 PM, Emilien Macchi wrote:

Hi,

I think one of the biggest challenges working on Puppet OpenStack
modules is to keep code consistency across all our modules (~20).
If you've read the code, you'll see there is some differences
between RabbitMQ configuration/parameters in some modules and this
is because we did not have the right tools to make it properly. A
lot of the duplicated code we have comes from Oslo libraries
configuration.

Now, I come up with an idea and two proposals.

Idea 

We could have some defined types to configure oslo sections in
OpenStack configuration files.

Something like: define oslo::messaging::rabbitmq( $user, $password
) { ensure_resource($name, 'oslo_messaging_rabbit/rabbit_userid',
{'value' => $user}) ... }

Usage in puppet-nova: ::oslo::messaging::rabbitmq{'nova_config':
user => 'nova', password => 'secrete', }

And patch all our modules to consume these defines and finally
have consistency at the way we configure Oslo projects (messaging,
logging, etc).

Proposals =

#1 Creating puppet-oslo ... and having oslo::messaging::rabbitmq,
oslo::messaging::qpid, ..., oslo::logging, etc. This module will be
used only to configure actual Oslo libraries when we deploy
OpenStack. To me, this solution is really consistent with how
OpenStack works today and is scalable as soon we contribute Oslo
configuration changes in this module.


+1 - For the Keystone authentication options, I think it is important to 
encapsulate this and hide the implementation from the other services as 
much as possible, to make it easier to use all of the different types of 
authentication supported by Keystone now and in the future.  I would 
think that something similar applies to the configuration of other 
OpenStack services.




#2 Using puppet-openstacklib ... and having
openstacklib::oslo::messaging::(...) A good thing is our modules
already use openstacklib. But openstacklib does not configure
OpenStack now, it creates some common defines & classes that are
consumed in other modules.


I personally prefer #1 because: * it's consistent with OpenStack. *
I don't want openstacklib the repo where we put everything common.
We have to differentiate *common-in-OpenStack* and
*common-in-our-modules*. I think openstacklib should continue to be
used for common things in our modules, like providers, wrappers,
database management, etc. But to configure common OpenStack bits
(aka Oslo©), we might want to create puppet-oslo.

As usual, any thoughts are welcome,

Best,



__





OpenStack Development Mailing List (not for usage questions)

Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-BEGIN PGP SIGNATURE-


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] Proposal to configure Oslo libraries

[Mike Dorman]

+1 I agree we should do this, etc., etc.

I don’t have a strong preference for #1 or #2, either.  But I do think #1 
is slightly more complicated from a deployer/operator perspective.  It’s 
another module I have to manage, pull in, etc.  Granted this is a trivial 
amount of incremental work.

I confess I am not super familiar with openstacklib, but I don’t 
understand why "We have to differentiate *common-in-OpenStack* and 
*common-in-our-modules*.”  To me, openstacklib is for _anything_ that’s 
common.  Maybe you could expand upon your thinking on this a little more, 
just so it’s a little more explicit?

Since others are not chomping at the bit to chime in here, I guess there 
is probably not many major preferences on this.  I would be happy with 
getting this done, regardless of how it’s implemented.

Thanks,
Mike






On 5/8/15, 7:50 AM, "Rich Megginson"  wrote:

>On 05/08/2015 07:17 AM, Doug Hellmann wrote:
>> Excerpts from Ben Nemec's message of 2015-05-07 15:57:48 -0500:
>>> I don't know much about the puppet project organization so I won't
>>> comment on whether 1 or 2 is better, but a big +1 to having a common
>>> way to configure Oslo opts.  Consistency of those options across all
>>> services is one of the big reasons we pushed so hard for the libraries
>>> to own their option definitions, so this would align well with the way
>>> the projects are consumed.
>>>
>>> - -Ben
>> Well said, Ben.
>>
>> Doug
>>
>>> On 05/07/2015 03:19 PM, Emilien Macchi wrote:
 Hi,

 I think one of the biggest challenges working on Puppet OpenStack
 modules is to keep code consistency across all our modules (~20).
 If you've read the code, you'll see there is some differences
 between RabbitMQ configuration/parameters in some modules and this
 is because we did not have the right tools to make it properly. A
 lot of the duplicated code we have comes from Oslo libraries
 configuration.

 Now, I come up with an idea and two proposals.

 Idea 

 We could have some defined types to configure oslo sections in
 OpenStack configuration files.

 Something like: define oslo::messaging::rabbitmq( $user, $password
 ) { ensure_resource($name, 'oslo_messaging_rabbit/rabbit_userid',
 {'value' => $user}) ... }

 Usage in puppet-nova: ::oslo::messaging::rabbitmq{'nova_config':
 user => 'nova', password => 'secrete', }

 And patch all our modules to consume these defines and finally
 have consistency at the way we configure Oslo projects (messaging,
 logging, etc).

 Proposals =

 #1 Creating puppet-oslo ... and having oslo::messaging::rabbitmq,
 oslo::messaging::qpid, ..., oslo::logging, etc. This module will be
 used only to configure actual Oslo libraries when we deploy
 OpenStack. To me, this solution is really consistent with how
 OpenStack works today and is scalable as soon we contribute Oslo
 configuration changes in this module.
>
>+1 - For the Keystone authentication options, I think it is important to 
>encapsulate this and hide the implementation from the other services as 
>much as possible, to make it easier to use all of the different types of 
>authentication supported by Keystone now and in the future.  I would 
>think that something similar applies to the configuration of other 
>OpenStack services.
>

 #2 Using puppet-openstacklib ... and having
 openstacklib::oslo::messaging::(...) A good thing is our modules
 already use openstacklib. But openstacklib does not configure
 OpenStack now, it creates some common defines & classes that are
 consumed in other modules.


 I personally prefer #1 because: * it's consistent with OpenStack. *
 I don't want openstacklib the repo where we put everything common.
 We have to differentiate *common-in-OpenStack* and
 *common-in-our-modules*. I think openstacklib should continue to be
 used for common things in our modules, like providers, wrappers,
 database management, etc. But to configure common OpenStack bits
 (aka Oslo©), we might want to create puppet-oslo.

 As usual, any thoughts are welcome,

 Best,



 __
>>> 

>>> OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

>>> -BEGIN PGP SIGNATURE-
>>>
>> 
>>_
>>_
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: 
>>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>__
>OpenSt

Re: [openstack-dev] [puppet] Proposal to configure Oslo libraries

[Colleen Murphy]

On Fri, May 8, 2015 at 9:47 AM, Mike Dorman  wrote:

> +1 I agree we should do this, etc., etc.
>
> I don’t have a strong preference for #1 or #2, either.  But I do think #1
> is slightly more complicated from a deployer/operator perspective.  It’s
> another module I have to manage, pull in, etc.  Granted this is a trivial
> amount of incremental work.
>
> I confess I am not super familiar with openstacklib, but I don’t
> understand why "We have to differentiate *common-in-OpenStack* and
> *common-in-our-modules*.”  To me, openstacklib is for _anything_ that’s
> common.  Maybe you could expand upon your thinking on this a little more,
> just so it’s a little more explicit?
>
> Since others are not chomping at the bit to chime in here, I guess there
> is probably not many major preferences on this.  I would be happy with
> getting this done, regardless of how it’s implemented.
>
> Thanks,
> Mike

I am strongly for #2. Adding another dependent module adds complexity for
both the operators who have to deploy it and the developers who have to
release it. puppet-openstacklib is already our dumping ground for shared
code, and I don't see why we should be shy of adding new things to it -
"common-in-OpenStack" IS "common-in-our-modules" and that's why
puppet-openstacklib was created.

Colleen
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] Proposal to configure Oslo libraries

[Mathieu Gagné]

On 2015-05-07 4:19 PM, Emilien Macchi wrote:
> 
> Proposals
> =
> 
> #1 Creating puppet-oslo
> ... and having oslo::messaging::rabbitmq, oslo::messaging::qpid, ...,
> oslo::logging, etc.
> This module will be used only to configure actual Oslo libraries when we
> deploy OpenStack. To me, this solution is really consistent with how
> OpenStack works today and is scalable as soon we contribute Oslo
> configuration changes in this module.
> 
> #2 Using puppet-openstacklib
> ... and having openstacklib::oslo::messaging::(...)
> A good thing is our modules already use openstacklib.
> But openstacklib does not configure OpenStack now, it creates some
> common defines & classes that are consumed in other modules.
> 

I prefer #1 due to oslo configs being specific to OpenStack versions.

The goal of openstacklib is to (hopefully) be OpenStack version agnostic
and be used only for code common across all *our* modules.

That's why I suggest going with solution #1, unless someone comes with a
solution to support multiple OpenStack versions in openstacklib without
the use of stable branches.

-- 
Mathieu

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] Proposal to configure Oslo libraries

[Colleen Murphy]

On Fri, May 8, 2015 at 12:50 PM, Mathieu Gagné  wrote:

> On 2015-05-07 4:19 PM, Emilien Macchi wrote:
> >
> > Proposals
> > =
> >
> > #1 Creating puppet-oslo
> > ... and having oslo::messaging::rabbitmq, oslo::messaging::qpid, ...,
> > oslo::logging, etc.
> http://git.openstack.org/cgit/stackforge/puppet-openstacklib/refs/
> > This module will be used only to configure actual Oslo libraries when we
> > deploy OpenStack. To me, this solution is really consistent with how
> > OpenStack works today and is scalable as soon we contribute Oslo
> > configuration changes in this module.
> >
> > #2 Using puppet-openstacklib
> > ... and having openstacklib::oslo::messaging::(...)
> > A good thing is our modules already use openstacklib.
> > But openstacklib does not configure OpenStack now, it creates some
> > common defines & classes that are consumed in other modules.
> >
>
> I prefer #1 due to oslo configs being specific to OpenStack versions.
>
> The goal of openstacklib is to (hopefully) be OpenStack version agnostic
> and be used only for code common across all *our* modules.
>
> That's why I suggest going with solution #1, unless someone comes with a
> solution to support multiple OpenStack versions in openstacklib without
> the use of stable branches.
>
puppet-openstacklib already has stable branches:
http://git.openstack.org/cgit/stackforge/puppet-openstacklib/refs/

I was not aware of any assumption that openstacklib would work for
different versions of our modules. I think this would be a difficult goal
to achieve. For example, the provider code in puppet-keystone is tightly
coupled with the code in puppet-openstacklib. If making puppet-openstacklib
version-agnostic is important (and I do think there would be value in it)
then maybe we should consider ripping out the provider backend code from
puppet-openstacklib and creating a puppet-openstackclient module.

Colleen

>
> --
> Mathieu
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] Proposal to configure Oslo libraries

[Emilien Macchi]

On 05/08/2015 04:09 PM, Colleen Murphy wrote:
> 
> 
> On Fri, May 8, 2015 at 12:50 PM, Mathieu Gagné  > wrote:
> 
> On 2015-05-07 4:19 PM, Emilien Macchi wrote:
> >
> > Proposals
> > =
> >
> > #1 Creating puppet-oslo
> > ... and having oslo::messaging::rabbitmq, oslo::messaging::qpid, ...,
> > oslo::logging, 
> etc.http://git.openstack.org/cgit/stackforge/puppet-openstacklib/refs/
> > This module will be used only to configure actual Oslo libraries when we
> > deploy OpenStack. To me, this solution is really consistent with how
> > OpenStack works today and is scalable as soon we contribute Oslo
> > configuration changes in this module.
> >
> > #2 Using puppet-openstacklib
> > ... and having openstacklib::oslo::messaging::(...)
> > A good thing is our modules already use openstacklib.
> > But openstacklib does not configure OpenStack now, it creates some
> > common defines & classes that are consumed in other modules.
> >
> 
> I prefer #1 due to oslo configs being specific to OpenStack versions.
> 
> The goal of openstacklib is to (hopefully) be OpenStack version agnostic
> and be used only for code common across all *our* modules.
> 
> That's why I suggest going with solution #1, unless someone comes with a
> solution to support multiple OpenStack versions in openstacklib without
> the use of stable branches.
> 
> puppet-openstacklib already has stable branches:
>  http://git.openstack.org/cgit/stackforge/puppet-openstacklib/refs/
> 
> I was not aware of any assumption that openstacklib would work for
> different versions of our modules. I think this would be a difficult
> goal to achieve. For example, the provider code in puppet-keystone is
> tightly coupled with the code in puppet-openstacklib. If making
> puppet-openstacklib version-agnostic is important (and I do think there
> would be value in it) then maybe we should consider ripping out the
> provider backend code from puppet-openstacklib and creating a
> puppet-openstackclient module.

Here is a first try for puppet-oslo:
https://github.com/enovance/puppet-oslo
And a patch in puppet-nova: https://review.openstack.org/182482

It's POC now, but any feedback is welcome...

> 
> Colleen
> 
> 
> --
> Mathieu
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

-- 
Emilien Macchi



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev