Re: [Puppet Users] Cross-module (package) dependencies
On Wed, 25 Jan 2012 09:59:10 -0500
Nan Liu wrote:
[...]
> > i.e. Say there were two unrelated modules which said slightly
> > different but *compatible* things:
> >
> > Module a:
> >
> > file { "/foo/bar":
> > ensure => 'present',
> > owner => 'root',
> > content => "blah blah",
> > }
> >
> >
> > Module b:
> >
> > file { "/foo/bar":
> > ensure => 'present',
> > mode => '0774',
> > }
> >
> >
> > Currently Puppet doesn't allow them to co-exist. It would be nice
> > if instead it could be told to check these definitions are
> > consistent, and then enforce the union of the two. The same
> > principle could apply to users, groups, packages, and presumably
> > any other resources.
>
> How would this be implemented in a sane way to deal with any
> attributes that are hash/array? Merge, merge+unique, fail? What if we
> add relationship (require/before) or other meta-parameters to the mix?
> If I use the puppet config_version feature to track what resource is
> changed by which line of puppet code for auditing purpose, how would I
> audit a single attribute that can be due to multiple line of code?
> Once I started thinking about define types (which behave like a
> resource), it's gets rather complex especially with conditional
> branching in the define type.
>
> Don't get me wrong, this clearly would be a useful feature, but I'm
> interested only if the rules of how this would behave can be clearly
> expressed and understood, otherwise this will be a maze of pain trying
> to figure out what part of the code broke something.
Getting into this level of detail is interesting, but quite beyond the
initial simple use case of the package type. As it has already been
pointed out in this thread, the initial problem would be solved by
simply allowing duplicates as long as all parameters are identical,
since it's typically just "package { 'foo': ensure => installed }".
That would also have a whole bunch of new implications, as I can imagine
people changing a single parameter in one place and getting confused as
to why the now get duplicate definition errors, but it would be a heck
of a lot simpler than trying to "merge" definitions together :-)
Matthias
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Cross-module (package) dependencies
On 01/25/2012 06:59 PM, Ashley Penney wrote: > After all, if they've been included on the host all of those attributes > should apply so I can't see a dangerous downside to this. Nan provided some killer arguments (imho) in the proper cross-dependency thread. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Cross-module (package) dependencies
On Wed, Jan 25, 2012 at 9:59 AM, Ashley Penney wrote:
> I don't have a solution but I do want to chime in and state that modules
> that
> exist just to collect together virtual resources is a horrible thing.
> We're just
> talking about working around the way Puppet works right now rather than
> defining what the end solution should be.
>
Absolutely right Ashley.
Please lets focus the conversation here on how things *should* be.
>
> I would be happiest with some kind of thing that can handle multiple
> versions
> of the same resource and work out a way to apply them as one entry. Like
> others suggested something that can merge all the unique attributes
> together
> for one entry. After all, if they've been included on the host all of
> those attributes
> should apply so I can't see a dangerous downside to this.
>
> It has to be better than trying to organize things so you only use a
> resource
> once and having all kinds of tricks and complicated workarounds. I don't
> want
> my modules to have to rely on some weird generic module full of virtual
> resources
> just to use a package without clashing into a module written by a
> coworker. I
> think modules should be capable, if possible, of standing alone, and this
> is ruining
> that basic concept.
>
> On Wed, Jan 25, 2012 at 12:12 PM, Dan Bode wrote:
>
>>
>>
>> On Tue, Jan 24, 2012 at 1:28 AM, Felix Frank <
>> felix.fr...@alumni.tu-berlin.de> wrote:
>>
>>> Hi,
>>>
>>> there was a discussion in the "can we deprecate defined() in Telly"
>>> thread about how we can even begin to design Forge modules without it.
>>>
>>> A recurring problem is that multiple modules rely on certain packages,
>>> and there is no good model (yet) to unite their resource declarations.
>>> Therefore it's a common (although imho disgusting) workaround to do
>>> things like
>>> if !defined(Package[foo]) { package { "foo": ensure => installed } }
>>>
>>> On 01/20/2012 11:34 PM, Cody wrote:
>>> > Defining all somewhat common packages in a central location becomes
>>> > unrealistic when you no longer "control" the code that is in every
>>> > module you use. If you obtain five modules from the forge and they
>>> > all require a specific package and so all define that package your not
>>> > going to convince, nor is it a good design to require everyone to move
>>> > the package definitions from that collection of modules. They need to
>>> > function as a collection out of the box.
>>>
>>> Agreed. How can this be accomplished?
>>>
>>> Perhaps there needs to be some kind of "Forge common" module that by
>>> policy can only ever declare virtual resources (packages are a prominent
>>>
>>
>> Until there is a way to distinguish between collection of regular versus
>> virtual resources, I hope that we don't do anything that advocates the
>> usage of virtual resources.
>>
>> The problem is that collections (like below) meant to
>> establish multiple dependencies unfortunately have the side effect of
>> realizing virtual resources.
>>
>> Class['apt'] -> Package<| |>
>>
>>
>>> example).
>>> A user who wishes to retain the capability of using modules from the
>>> Forge would be required to install this common module, and replace their
>>> own resource declarations with realizations of the common resources.
>>> For this to work, it's definitely a plus that you can override
>>> attributes in collections:
>>> Package<| title == "apache2": |> { ensure => "2.2.12" }
>>> ...although that does bear some caveats. Does this still work in recent
>>> versions?
>>>
>>> If we can take this for granted, all Forge modules can adhere to that
>>> same standard.
>>>
>>> This is a rough sketch of how things might possibly work, and surely has
>>> lots of wrinkles of its own. Still, I'm quite sure we need a proper way
>>> to rid ourselves of the horror that is the parse order dependent check
>>> for defined resources ;-)
>>>
>>> Cheers,
>>> Felix
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Puppet Users" group.
>>> To post to this group, send email to puppet-users@googlegroups.com.
>>> To unsubscribe from this group, send email to
>>> puppet-users+unsubscr...@googlegroups.com.
>>> For more options, visit this group at
>>> http://groups.google.com/group/puppet-users?hl=en.
>>>
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Puppet Users" group.
>> To post to this group, send email to puppet-users@googlegroups.com.
>> To unsubscribe from this group, send email to
>> puppet-users+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/puppet-users?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http
Re: [Puppet Users] Cross-module (package) dependencies
I don't have a solution but I do want to chime in and state that modules
that
exist just to collect together virtual resources is a horrible thing.
We're just
talking about working around the way Puppet works right now rather than
defining what the end solution should be.
I would be happiest with some kind of thing that can handle multiple
versions
of the same resource and work out a way to apply them as one entry. Like
others suggested something that can merge all the unique attributes together
for one entry. After all, if they've been included on the host all of
those attributes
should apply so I can't see a dangerous downside to this.
It has to be better than trying to organize things so you only use a
resource
once and having all kinds of tricks and complicated workarounds. I don't
want
my modules to have to rely on some weird generic module full of virtual
resources
just to use a package without clashing into a module written by a coworker.
I
think modules should be capable, if possible, of standing alone, and this
is ruining
that basic concept.
On Wed, Jan 25, 2012 at 12:12 PM, Dan Bode wrote:
>
>
> On Tue, Jan 24, 2012 at 1:28 AM, Felix Frank <
> felix.fr...@alumni.tu-berlin.de> wrote:
>
>> Hi,
>>
>> there was a discussion in the "can we deprecate defined() in Telly"
>> thread about how we can even begin to design Forge modules without it.
>>
>> A recurring problem is that multiple modules rely on certain packages,
>> and there is no good model (yet) to unite their resource declarations.
>> Therefore it's a common (although imho disgusting) workaround to do
>> things like
>> if !defined(Package[foo]) { package { "foo": ensure => installed } }
>>
>> On 01/20/2012 11:34 PM, Cody wrote:
>> > Defining all somewhat common packages in a central location becomes
>> > unrealistic when you no longer "control" the code that is in every
>> > module you use. If you obtain five modules from the forge and they
>> > all require a specific package and so all define that package your not
>> > going to convince, nor is it a good design to require everyone to move
>> > the package definitions from that collection of modules. They need to
>> > function as a collection out of the box.
>>
>> Agreed. How can this be accomplished?
>>
>> Perhaps there needs to be some kind of "Forge common" module that by
>> policy can only ever declare virtual resources (packages are a prominent
>>
>
> Until there is a way to distinguish between collection of regular versus
> virtual resources, I hope that we don't do anything that advocates the
> usage of virtual resources.
>
> The problem is that collections (like below) meant to
> establish multiple dependencies unfortunately have the side effect of
> realizing virtual resources.
>
> Class['apt'] -> Package<| |>
>
>
>> example).
>> A user who wishes to retain the capability of using modules from the
>> Forge would be required to install this common module, and replace their
>> own resource declarations with realizations of the common resources.
>> For this to work, it's definitely a plus that you can override
>> attributes in collections:
>> Package<| title == "apache2": |> { ensure => "2.2.12" }
>> ...although that does bear some caveats. Does this still work in recent
>> versions?
>>
>> If we can take this for granted, all Forge modules can adhere to that
>> same standard.
>>
>> This is a rough sketch of how things might possibly work, and surely has
>> lots of wrinkles of its own. Still, I'm quite sure we need a proper way
>> to rid ourselves of the horror that is the parse order dependent check
>> for defined resources ;-)
>>
>> Cheers,
>> Felix
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Puppet Users" group.
>> To post to this group, send email to puppet-users@googlegroups.com.
>> To unsubscribe from this group, send email to
>> puppet-users+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/puppet-users?hl=en.
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Cross-module (package) dependencies
On Tue, Jan 24, 2012 at 1:28 AM, Felix Frank <
felix.fr...@alumni.tu-berlin.de> wrote:
> Hi,
>
> there was a discussion in the "can we deprecate defined() in Telly"
> thread about how we can even begin to design Forge modules without it.
>
> A recurring problem is that multiple modules rely on certain packages,
> and there is no good model (yet) to unite their resource declarations.
> Therefore it's a common (although imho disgusting) workaround to do
> things like
> if !defined(Package[foo]) { package { "foo": ensure => installed } }
>
> On 01/20/2012 11:34 PM, Cody wrote:
> > Defining all somewhat common packages in a central location becomes
> > unrealistic when you no longer "control" the code that is in every
> > module you use. If you obtain five modules from the forge and they
> > all require a specific package and so all define that package your not
> > going to convince, nor is it a good design to require everyone to move
> > the package definitions from that collection of modules. They need to
> > function as a collection out of the box.
>
> Agreed. How can this be accomplished?
>
> Perhaps there needs to be some kind of "Forge common" module that by
> policy can only ever declare virtual resources (packages are a prominent
>
Until there is a way to distinguish between collection of regular versus
virtual resources, I hope that we don't do anything that advocates the
usage of virtual resources.
The problem is that collections (like below) meant to
establish multiple dependencies unfortunately have the side effect of
realizing virtual resources.
Class['apt'] -> Package<| |>
> example).
> A user who wishes to retain the capability of using modules from the
> Forge would be required to install this common module, and replace their
> own resource declarations with realizations of the common resources.
> For this to work, it's definitely a plus that you can override
> attributes in collections:
> Package<| title == "apache2": |> { ensure => "2.2.12" }
> ...although that does bear some caveats. Does this still work in recent
> versions?
>
> If we can take this for granted, all Forge modules can adhere to that
> same standard.
>
> This is a rough sketch of how things might possibly work, and surely has
> lots of wrinkles of its own. Still, I'm quite sure we need a proper way
> to rid ourselves of the horror that is the parse order dependent check
> for defined resources ;-)
>
> Cheers,
> Felix
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Cross-module (package) dependencies
On Wed, Jan 25, 2012 at 6:11 AM, Nick wrote:
> On 24/01/12 19:53, Nigel Kersten wrote:
>> What if you made two functions:
>>
>> 1. declare_capability("foo")
>> This would essentially just create an empty resource, Capability[foo]. If
>> multiple modules tried to declare the same capability, it would error, just
>> like
>> we do today with duplicate resources.
>>
>> 2. require_capability("foo")
>> This would just declare a requirement relationship to Capability[foo]. If
>> it
>> couldn't find that resource, compilation would fail.
>>
>>
>> Wouldn't something like this solve the general problem? We don't really want
>> to
>> just speak about packages, you generally want to be able to express
>> requirements
>> like:
>>
>> * This module requires a working SSH server
>> * This module requires a MySQL database
>
>
> More generally, I think it'd be nice to have a way to be able to define
> resources, in a way approximately like Puppet already does, but without also
> having to say "and if anyone else tries to define the same name resource in
> any
> way, fail".
>
> i.e. Say there were two unrelated modules which said slightly different but
> *compatible* things:
>
> Module a:
>
> file { "/foo/bar":
> ensure => 'present',
> owner => 'root',
> content => "blah blah",
> }
>
>
> Module b:
>
> file { "/foo/bar":
> ensure => 'present',
> mode => '0774',
> }
>
>
> Currently Puppet doesn't allow them to co-exist. It would be nice if instead
> it
> could be told to check these definitions are consistent, and then enforce the
> union of the two. The same principle could apply to users, groups, packages,
> and presumably any other resources.
How would this be implemented in a sane way to deal with any
attributes that are hash/array? Merge, merge+unique, fail? What if we
add relationship (require/before) or other meta-parameters to the mix?
If I use the puppet config_version feature to track what resource is
changed by which line of puppet code for auditing purpose, how would I
audit a single attribute that can be due to multiple line of code?
Once I started thinking about define types (which behave like a
resource), it's gets rather complex especially with conditional
branching in the define type.
Don't get me wrong, this clearly would be a useful feature, but I'm
interested only if the rules of how this would behave can be clearly
expressed and understood, otherwise this will be a maze of pain trying
to figure out what part of the code broke something.
Nan
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Cross-module (package) dependencies
On 24/01/12 19:53, Nigel Kersten wrote:
> What if you made two functions:
>
> 1. declare_capability("foo")
> This would essentially just create an empty resource, Capability[foo]. If
> multiple modules tried to declare the same capability, it would error, just
> like
> we do today with duplicate resources.
>
> 2. require_capability("foo")
> This would just declare a requirement relationship to Capability[foo]. If it
> couldn't find that resource, compilation would fail.
>
>
> Wouldn't something like this solve the general problem? We don't really want
> to
> just speak about packages, you generally want to be able to express
> requirements
> like:
>
> * This module requires a working SSH server
> * This module requires a MySQL database
More generally, I think it'd be nice to have a way to be able to define
resources, in a way approximately like Puppet already does, but without also
having to say "and if anyone else tries to define the same name resource in any
way, fail".
i.e. Say there were two unrelated modules which said slightly different but
*compatible* things:
Module a:
file { "/foo/bar":
ensure => 'present',
owner => 'root',
content => "blah blah",
}
Module b:
file { "/foo/bar":
ensure => 'present',
mode => '0774',
}
Currently Puppet doesn't allow them to co-exist. It would be nice if instead it
could be told to check these definitions are consistent, and then enforce the
union of the two. The same principle could apply to users, groups, packages,
and presumably any other resources.
See my related point in the thread "constraint checking".
N
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Cross-module (package) dependencies
On 01/25/2012 05:48 AM, Walter Heck wrote:
>>> We'd also need a way to test if a capability is being provided by the
>>> >> system as a whole. Ie, a mysql module needs to be able to check if it
>>> >> needs to include resources for a firewall, monitoring, etc.
>> > Something like defined()? ;-)
> Come on, I thought we all agreed that defined is Nasty and Evil. At
> least make it defined_new() or maybe even defined2() :P
Great, now we're regressing back to thread that Nigel kicked me out of
for this :P
Fine, I'll play.
On 01/19/2012 07:56 PM, Nick Fagerlund wrote:
> Defined() doesn't suck! It's a 100% reliable way to check what classes
> and defined types are available to the autoloader. I challenge anyone
> to find me an example of this usage that fails.
>
> But trying to use it to see whether a specific resource instance has
> been declared is pure madness and must be stopped. It's defined(), not
> declared(), and it shouldn't accept resource references as arguments
> at all.
So there. In that thread, Nick gave pretty much the very use case
example Nan has referred to earlier in this thread, so I hope we can
agree that this can be handled by the (newly introduced?) declared()
function (or stripped down version of defined() that won't allow
resource reference parameters).
On 01/19/2012 09:17 PM, Nick Fagerlund wrote:
> So, you can conditionally declare the rule if the defined type is
> available to the autoloader, and otherwise you don't attempt to manage
> the firewall and expect that the user has read the documentation and
> will make a hole for the service themselves.
>
> if defined(firewall::iptables::rule) {
> firewall::iptables::rule {'mysql_server':
> ...etc. etc.
> }
> }
>
> See? It's just a way to peek around at what the user has installed.
Cheers,
Felix
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Cross-module (package) dependencies
On Wed, Jan 25, 2012 at 05:51, Nigel Kersten wrote: > Yeah. I went for functions to try and hide the fact that this is a resource > underneath, but perhaps modeling this more transparently makes sense. > >> We'd also need a way to test if a capability is being provided by the >> system as a whole. Ie, a mysql module needs to be able to check if it >> needs to include resources for a firewall, monitoring, etc. > Something like defined()? ;-) Come on, I thought we all agreed that defined is Nasty and Evil. At least make it defined_new() or maybe even defined2() :P -- Walter Heck -- follow @walterheck on twitter to see what I'm up to! -- Check out my new startup: Server Monitoring as a Service @ http://tribily.com Follow @tribily on Twitter and/or 'Like' our Facebook page at http://www.facebook.com/tribily -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Cross-module (package) dependencies
On Tue, Jan 24, 2012 at 4:34 PM, Walter Heck wrote:
> > I had this thought the other day, but more focused around the higher
> level
> > problem of dependency specification and consumption than Packages in
> > particular.
> Yeah, since that is something that is currently kind of difficult to
> implement properly. I'd welcome a proper way to do this.
>
> > What if you made two functions:
> >
> > 1. declare_capability("foo")
> > This would essentially just create an empty resource, Capability[foo].
> If
> > multiple modules tried to declare the same capability, it would error,
> just
> > like we do today with duplicate resources.
> >
> > 2. require_capability("foo")
> > This would just declare a requirement relationship to Capability[foo].
> If
> > it couldn't find that resource, compilation would fail.
> >
> >
> > Wouldn't something like this solve the general problem? We don't really
> want
> > to just speak about packages, you generally want to be able to express
> > requirements like:
> >
> > * This module requires a working SSH server
> > * This module requires a MySQL database
> >
> > etc etc
> >
> > I've deliberately focused this on doing this with functions as the
> extension
> > point as I think it keeps us focused, even if we decide that in the
> future
> > this should be solved in the metadata layer of a module.
> Hmm, while I can see your point of using functions, I think a more
> "puppety" way of doing this would be to have
>
> require Capability['yadda']
>
> vs
>
> capability{'yadda':
> ensure => present
> }
>
Yeah. I went for functions to try and hide the fact that this is a resource
underneath, but perhaps modeling this more transparently makes sense.
>
> We'd also need a way to test if a capability is being provided by the
> system as a whole. Ie, a mysql module needs to be able to check if it
> needs to include resources for a firewall, monitoring, etc.
>
>
Something like defined()? ;-)
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Cross-module (package) dependencies
> I had this thought the other day, but more focused around the higher level
> problem of dependency specification and consumption than Packages in
> particular.
Yeah, since that is something that is currently kind of difficult to
implement properly. I'd welcome a proper way to do this.
> What if you made two functions:
>
> 1. declare_capability("foo")
> This would essentially just create an empty resource, Capability[foo]. If
> multiple modules tried to declare the same capability, it would error, just
> like we do today with duplicate resources.
>
> 2. require_capability("foo")
> This would just declare a requirement relationship to Capability[foo]. If
> it couldn't find that resource, compilation would fail.
>
>
> Wouldn't something like this solve the general problem? We don't really want
> to just speak about packages, you generally want to be able to express
> requirements like:
>
> * This module requires a working SSH server
> * This module requires a MySQL database
>
> etc etc
>
> I've deliberately focused this on doing this with functions as the extension
> point as I think it keeps us focused, even if we decide that in the future
> this should be solved in the metadata layer of a module.
Hmm, while I can see your point of using functions, I think a more
"puppety" way of doing this would be to have
require Capability['yadda']
vs
capability{'yadda':
ensure => present
}
We'd also need a way to test if a capability is being provided by the
system as a whole. Ie, a mysql module needs to be able to check if it
needs to include resources for a firewall, monitoring, etc.
cheers,
--
Walter Heck
--
follow @walterheck on twitter to see what I'm up to!
--
Check out my new startup: Server Monitoring as a Service @ http://tribily.com
Follow @tribily on Twitter and/or 'Like' our Facebook page at
http://www.facebook.com/tribily
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Cross-module (package) dependencies
On Tue, Jan 24, 2012 at 2:53 PM, Nigel Kersten wrote:
>
>
> On Tue, Jan 24, 2012 at 4:42 AM, Walter Heck wrote:
>>
>> What if there was some standardised way modules expose a sort of list
>> of requirements or an API of some sort?
>
>
> I had this thought the other day, but more focused around the higher level
> problem of dependency specification and consumption than Packages in
> particular.
>
> What if you made two functions:
>
> 1. declare_capability("foo")
> This would essentially just create an empty resource, Capability[foo]. If
> multiple modules tried to declare the same capability, it would error, just
> like we do today with duplicate resources.
>
> 2. require_capability("foo")
> This would just declare a requirement relationship to Capability[foo]. If
> it couldn't find that resource, compilation would fail.
I'm actually also interested in detect capability too. For example if
class firewall is included, add the following iptable rules, otherwise
don't include firewall rules in the catalog. It can be implemented as
parameter in the application class, but feels much nicer if I can
simply include firewall, and all the app added firewall rules, vs.
class { 'app1': firewall => true }... I'm aware this might be order
dependent and tricky to implement.
Thanks,
Nan
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Cross-module (package) dependencies
On Tue, Jan 24, 2012 at 4:42 AM, Walter Heck wrote:
> What if there was some standardised way modules expose a sort of list
> of requirements or an API of some sort?
I had this thought the other day, but more focused around the higher level
problem of dependency specification and consumption than Packages in
particular.
What if you made two functions:
1. declare_capability("foo")
This would essentially just create an empty resource, Capability[foo]. If
multiple modules tried to declare the same capability, it would error, just
like we do today with duplicate resources.
2. require_capability("foo")
This would just declare a requirement relationship to Capability[foo]. If
it couldn't find that resource, compilation would fail.
Wouldn't something like this solve the general problem? We don't really
want to just speak about packages, you generally want to be able to express
requirements like:
* This module requires a working SSH server
* This module requires a MySQL database
etc etc
I've deliberately focused this on doing this with functions as the
extension point as I think it keeps us focused, even if we decide that in
the future this should be solved in the metadata layer of a module.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Cross-module (package) dependencies
I was thinking more in terms of an exception handler:
package { "foo": ensure => installed, exceptDefined => skip}
Or something of that nature. This could also be used in other
situations where you want to bypass default behaviors. We've seen
situations where users want to apply a file if it exists in the module
but otherwise proceed without errors. This could be done like so:
file {"$foo": ensure => present, source => "${foo}.txt", exceptAbsent => skip}
On Tue, Jan 24, 2012 at 1:28 AM, Felix Frank
wrote:
> Hi,
>
> there was a discussion in the "can we deprecate defined() in Telly"
> thread about how we can even begin to design Forge modules without it.
>
> A recurring problem is that multiple modules rely on certain packages,
> and there is no good model (yet) to unite their resource declarations.
> Therefore it's a common (although imho disgusting) workaround to do
> things like
> if !defined(Package[foo]) { package { "foo": ensure => installed } }
>
> On 01/20/2012 11:34 PM, Cody wrote:
>> Defining all somewhat common packages in a central location becomes
>> unrealistic when you no longer "control" the code that is in every
>> module you use. If you obtain five modules from the forge and they
>> all require a specific package and so all define that package your not
>> going to convince, nor is it a good design to require everyone to move
>> the package definitions from that collection of modules. They need to
>> function as a collection out of the box.
>
> Agreed. How can this be accomplished?
>
> Perhaps there needs to be some kind of "Forge common" module that by
> policy can only ever declare virtual resources (packages are a prominent
> example).
> A user who wishes to retain the capability of using modules from the
> Forge would be required to install this common module, and replace their
> own resource declarations with realizations of the common resources.
> For this to work, it's definitely a plus that you can override
> attributes in collections:
> Package<| title == "apache2": |> { ensure => "2.2.12" }
> ...although that does bear some caveats. Does this still work in recent
> versions?
>
> If we can take this for granted, all Forge modules can adhere to that
> same standard.
>
> This is a rough sketch of how things might possibly work, and surely has
> lots of wrinkles of its own. Still, I'm quite sure we need a proper way
> to rid ourselves of the horror that is the parse order dependent check
> for defined resources ;-)
>
> Cheers,
> Felix
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Cross-module (package) dependencies
What if there was some standardised way modules expose a sort of list
of requirements or an API of some sort? I don't know how this would
work in practice since it just popped in my mind, but I imagine for
instance a monitoring module would expose some kind of hook that other
modules can check for, and if it is found then execute the code
pertaining to monitoring. It would mean a certain abstraction, so
instead of saying an external module needs to implement nagios-perl,
it would just ask for an external module to implement monitoring, and
pass certain parameters to it in order to get that monitoring
implemented. The nagios module would then be responsible for the
actual monitoring stuff.
Come to think of it, this wouldn't take care of the duplicate resource
stuff discussed here. Nevertheless I wanted to put out this thought
anyway as it seems to be a larger issue related to this.
As for the duplicate resource issue, what about a diff/merge strategy:
If duplicate resources are found, tehre are three options:
1. if the two resource were compared and found the same it would just
be allowed and logged to the user
2. if not found the same, merged and logged:
file{x: ensure => present} and
file {x: owner => root}
becomes
file {x: ensure => present, owner => root},
3. if merge conflicts are found (ensure => present and ensure =>
absent in the two definitions) we get an error.
Could that work?
Walter
On Tue, Jan 24, 2012 at 13:38, Tim Sharpe wrote:
> The only "clean" way to handle this that I can think of off the top of my
> head
> is for Puppet to start silently discarding duplicate definitions and just
> using
> the first one it comes across (with perhaps a message being logged at info
> level so that it's not completely invisible).
>
> This is far from ideal but so are the other proposed solutions so far :)
>
> On Tuesday, 24 January 2012 at 10:20 PM, Matthias Saou wrote:
>
> On Tue, 24 Jan 2012 10:32:31 +0100
> Felix Frank wrote:
>
> Perhaps there needs to be some kind of "Forge common" module that by
> policy can only ever declare virtual resources (packages are a
> prominent example).
>
>
> This only takes care of the Forge case. The problem of having the same
> package being required in multiple places isn't limited to Forge, it's
> one I run into locally all the time (and I'm sure I'm not the only one).
>
> On top of that, once you try to cover multiple distributions where
> package names diverge, it starts getting hard. And once the packaging
> itself is different, as in (non-existing) sub-packages... there is no
> end (1).
>
> I don't have the slightest idea as to how all this could be solved in
> a clean way, but "requiring a common bit of high level code" isn't the
> global solution I'm myself hoping for.
>
> Matthias
>
> (1) Example where a nagios plugin requires the package for the nagios
> perl bindings. It's "nagios-perl" for most RPMs, but not split out and
> in the main "nagios-plugins" on Gentoo... then for any other plugin
> from nagios-plugins RPMs, such as "nagios-plugins-file_age", same
> thing. So you get a big mess with duplicate definitions for the main
> Gentoo "nagios-plugins" package, or you need yet another layer of high
> level code hack :-/
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
--
Walter Heck
--
follow @walterheck on twitter to see what I'm up to!
--
Check out my new startup: Server Monitoring as a Service @ http://tribily.com
Follow @tribily on Twitter and/or 'Like' our Facebook page at
http://www.facebook.com/tribily
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Cross-module (package) dependencies
On 01/24/2012 12:38 PM, Tim Sharpe wrote: > The only "clean" way to handle this that I can think of off the top of > my head > is for Puppet to start silently discarding duplicate definitions and > just using > the first one it comes across (with perhaps a message being logged at info > level so that it's not completely invisible). > > This is far from ideal but so are the other proposed solutions so far :) No, it really adheres to KISS very nicely. This reminds me of the discussion of how resource defaults should stack when they combine from different scopes. (How did that turn out, anyway?) The vision here would probably be that the compiler collects all definitions or the same resource, and only dies if any attributes differ. Sadly, *due* to resource defaults, this could be more likely than one would think. Also, depending on how the compiler is implemented, it may be Very Hard to do. Cheers, Felix -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Cross-module (package) dependencies
The only "clean" way to handle this that I can think of off the top of my head is for Puppet to start silently discarding duplicate definitions and just using the first one it comes across (with perhaps a message being logged at info level so that it's not completely invisible). This is far from ideal but so are the other proposed solutions so far :) On Tuesday, 24 January 2012 at 10:20 PM, Matthias Saou wrote: > On Tue, 24 Jan 2012 10:32:31 +0100 > Felix Frank (mailto:felix.fr...@alumni.tu-berlin.de)> wrote: > > > Perhaps there needs to be some kind of "Forge common" module that by > > policy can only ever declare virtual resources (packages are a > > prominent example). > > > > > This only takes care of the Forge case. The problem of having the same > package being required in multiple places isn't limited to Forge, it's > one I run into locally all the time (and I'm sure I'm not the only one). > > On top of that, once you try to cover multiple distributions where > package names diverge, it starts getting hard. And once the packaging > itself is different, as in (non-existing) sub-packages... there is no > end (1). > > I don't have the slightest idea as to how all this could be solved in > a clean way, but "requiring a common bit of high level code" isn't the > global solution I'm myself hoping for. > > Matthias > > (1) Example where a nagios plugin requires the package for the nagios > perl bindings. It's "nagios-perl" for most RPMs, but not split out and > in the main "nagios-plugins" on Gentoo... then for any other plugin > from nagios-plugins RPMs, such as "nagios-plugins-file_age", same > thing. So you get a big mess with duplicate definitions for the main > Gentoo "nagios-plugins" package, or you need yet another layer of high > level code hack :-/ > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com > (mailto:puppet-users@googlegroups.com). > To unsubscribe from this group, send email to > puppet-users+unsubscr...@googlegroups.com > (mailto:puppet-users+unsubscr...@googlegroups.com). > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Cross-module (package) dependencies
On Tue, 24 Jan 2012 10:32:31 +0100 Felix Frank wrote: > Perhaps there needs to be some kind of "Forge common" module that by > policy can only ever declare virtual resources (packages are a > prominent example). This only takes care of the Forge case. The problem of having the same package being required in multiple places isn't limited to Forge, it's one I run into locally all the time (and I'm sure I'm not the only one). On top of that, once you try to cover multiple distributions where package names diverge, it starts getting hard. And once the packaging itself is different, as in (non-existing) sub-packages... there is no end (1). I don't have the slightest idea as to how all this could be solved in a clean way, but "requiring a common bit of high level code" isn't the global solution I'm myself hoping for. Matthias (1) Example where a nagios plugin requires the package for the nagios perl bindings. It's "nagios-perl" for most RPMs, but not split out and in the main "nagios-plugins" on Gentoo... then for any other plugin from nagios-plugins RPMs, such as "nagios-plugins-file_age", same thing. So you get a big mess with duplicate definitions for the main Gentoo "nagios-plugins" package, or you need yet another layer of high level code hack :-/ -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Cross-module (package) dependencies
Hi,
there was a discussion in the "can we deprecate defined() in Telly"
thread about how we can even begin to design Forge modules without it.
A recurring problem is that multiple modules rely on certain packages,
and there is no good model (yet) to unite their resource declarations.
Therefore it's a common (although imho disgusting) workaround to do
things like
if !defined(Package[foo]) { package { "foo": ensure => installed } }
On 01/20/2012 11:34 PM, Cody wrote:
> > Defining all somewhat common packages in a central location becomes
> > unrealistic when you no longer "control" the code that is in every
> > module you use. If you obtain five modules from the forge and they
> > all require a specific package and so all define that package your not
> > going to convince, nor is it a good design to require everyone to move
> > the package definitions from that collection of modules. They need to
> > function as a collection out of the box.
Agreed. How can this be accomplished?
Perhaps there needs to be some kind of "Forge common" module that by
policy can only ever declare virtual resources (packages are a prominent
example).
A user who wishes to retain the capability of using modules from the
Forge would be required to install this common module, and replace their
own resource declarations with realizations of the common resources.
For this to work, it's definitely a plus that you can override
attributes in collections:
Package<| title == "apache2": |> { ensure => "2.2.12" }
...although that does bear some caveats. Does this still work in recent
versions?
If we can take this for granted, all Forge modules can adhere to that
same standard.
This is a rough sketch of how things might possibly work, and surely has
lots of wrinkles of its own. Still, I'm quite sure we need a proper way
to rid ourselves of the horror that is the parse order dependent check
for defined resources ;-)
Cheers,
Felix
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Cross-module (package) dependencies
Hi,
there was a discussion in the "can we deprecate defined() in Telly"
thread about how we can even begin to design Forge modules without it.
A recurring problem is that multiple modules rely on certain packages,
and there is no good model (yet) to unite their resource declarations.
Therefore it's a common (although imho disgusting) workaround to do
things like
if !defined(Package[foo]) { package { "foo": ensure => installed } }
On 01/20/2012 11:34 PM, Cody wrote:
> Defining all somewhat common packages in a central location becomes
> unrealistic when you no longer "control" the code that is in every
> module you use. If you obtain five modules from the forge and they
> all require a specific package and so all define that package your not
> going to convince, nor is it a good design to require everyone to move
> the package definitions from that collection of modules. They need to
> function as a collection out of the box.
Agreed. How can this be accomplished?
Perhaps there needs to be some kind of "Forge common" module that by
policy can only ever declare virtual resources (packages are a prominent
example).
A user who wishes to retain the capability of using modules from the
Forge would be required to install this common module, and replace their
own resource declarations with realizations of the common resources.
For this to work, it's definitely a plus that you can override
attributes in collections:
Package<| title == "apache2": |> { ensure => "2.2.12" }
...although that does bear some caveats. Does this still work in recent
versions?
If we can take this for granted, all Forge modules can adhere to that
same standard.
This is a rough sketch of how things might possibly work, and surely has
lots of wrinkles of its own. Still, I'm quite sure we need a proper way
to rid ourselves of the horror that is the parse order dependent check
for defined resources ;-)
Cheers,
Felix
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
