Re: [Puppet Users] defined() implicitly requiring resource reference
Hi John
On 14/06/2013, at 12:06 AM, jcbollinger wrote:
>>
>> On Wednesday, June 12, 2013 9:15:22 AM UTC-5, Tom Lanyon wrote:
>> On 05/06/2013, at 11:51 PM, jcbollinger [...] wrote:
>> > I don't much like that general approach in the first place on account of
>> > the $requested_package parameter. That you encounter difficulty when you
>> > try something a bit dodgy should not be surprising.
>>
>> Can you explain this further so I can understand the issue?
>
> Initially, it was mostly a gut feeling. After having had time to step back
> from the issue and return to it fresh, I think it's a combination of things,
> mostly revolving around what you're actually modeling, and how you're
> modeling it.
>
> Basically, the 'myapp' definition represents one package chosen from a list
> of mutually exclusive packages. If that's all it is, then its name is
> misleading -- it should be more generic -- and it should probably take the
> exclusive list as a second parameter. On the other hand, if it is indeed
> supposed to be something specific, then it doesn't take much advantage of
> that. In particular -- and here's where my previous comment came from -- if
> it supposed to represent something specific to your application, then why
> doesn't it know anything about the application's package names?
>
> Also, if the point is supposed to be that only one version of the application
> can be installed at a time, and the definition is specific to that
> application, then it really ought to be a class instead.
Alas, the whole intention of this is that multiple versions of the package must
be installed at the same time. This is where the problem lies, in that we have
no way to clean up old unused versions once they're no longer needed.
We have instances of an application MyApp, which utilise a shared Package in
many to one relationship:
Package[one] ___ MyApp[app1]
Package[two] ___ MyApp[app2]
\___ MyApp[app3]
Package[three] _ MyApp[app4]
\_ MyApp[app5]
\_ MyApp[app6]
In the above example, if we upgraded MyApp[app2] and MyApp[app3] to
Package[three], we'd still have Package[two] installed on all of the hosts but
no MyApp instances would be using it. This is the unused Package we're trying
to clean up with Puppet (there's no longer a MyApp resource definition which
references Package[two]).
There is no mutually exclusive packages or any other such conflicts.
>> > In fact, despite my dissatisfaction with your approach, you can indeed do
>> > this without defined(), and without even disrupting your current structure
>> > very much. Here's one way I think would work:
>> >
>> > # This class ensures all known app packages are
>> > # by default purged
>> > class app::packages {
>> >$apps = split($::app_packages, ',')
>> >package { $apps:
>> >ensure => 'purged'
>> >}
>> > }
>> >
>> > # Overrides the requested package to be declared
>> > # present instead of purged.
>> > define app::myapp($requested_package) {
>> >include 'app::packages'
>> >Package<| title == $requested_package |> {
>> >ensure => 'present'
>> >}
>> > }
>> >
>> > # no separate package_cleanup required
>>
>>
>> OK, I wondered whether we could do something like this however - forgive my
>> naivety - I still can't see how this could be a complete solution without
>> something like defined().
>>
>> As an example... your above snippet works fine to ensure already installed
>> packages remain installed, but what if we wanted to install a brand new
>> version of app::myapp? Because a 'package' resource with title
>> $requested_package does not yet exist, the Package<||> collector matches no
>> resources and the new package is not installed. The only solution that I
>> can come up with is to check whether such a resource is already defined and,
>> if not, define one.
>
> You appear to have a serious misunderstanding. Resource collectors have no
> direct relationship with or dependency on which resources are already
> installed on the target system. They work exclusively with resource
> declarations in your manifests, and they do so at catalog compilation time.
> Moreover, they are independent of parse order (though the example anyway
> ensures a parse order that would work if collectors were parse-order
> depend
Re: [Puppet Users] defined() implicitly requiring resource reference
Hi John,
Sorry for the delayed reply.
On 05/06/2013, at 11:51 PM, jcbollinger wrote:
>
>> Sorry, I should have been clearer that this occurs when Package[package-434]
>> IS declared elsewhere. "!defined(Package[package-434])" therefore is false,
>> so just by referencing the existing declaration within the defined() call it
>> seems to incite an implicit dependency.
>>
> If that's really what's happening then you should be able to create a simple
> test case that demonstrates it. That would be a worthy subject for a bug
> report.
I'll see what I can do.
Is this implicit dependency expected behaviour or am I doing something
Bad(tm)?
>>>
>>> Both.
>>>
>>> Supposing that the target package is not declared elsewhere (so that the
>>> !defined() condition is true) the definition will declare the package
>>> itself to ensure it absent, and in that case you would expect a
>>> relationship between the defined-type instance and the resource declared by
>>> it. If elsewhere you have specific references to that package, applicable
>>> resource parameter defaults, or collectors that will match that package,
>>> then you can get relationships with it that are not evident from the
>>> defined type body.
>>>
>>> On the other hand, defined() is evil. Do not use it. Ever.
>>
>> I had this discussion with someone on #puppet IRC earlier and they ended up
>> with "Oh, in your case, defined() is probably actually what you want."
>
> No. defined() is never what you want. It may at times seem expedient, but
> it's bad news every time.
OK, understood.
>> define myapp ($requested_package){
>>
>> package { $requested_package:
>>ensure => present
>> }
>>
>> define package_cleanup {
>>$installed_package = $title
>>
>>if $installed_package != $requested_package {
>> package { $installed_package:
>>ensure => purged
>> }
>>}
>> }
>>
>> # assuming a facter fact named 'installed_packages'
>> package_cleanup { split($::installed_packages, ','): }
>> }
>
> I don't much like that general approach in the first place on account of the
> $requested_package parameter. That you encounter difficulty when you try
> something a bit dodgy should not be surprising.
Can you explain this further so I can understand the issue?
> In fact, despite my dissatisfaction with your approach, you can indeed do
> this without defined(), and without even disrupting your current structure
> very much. Here's one way I think would work:
>
> # This class ensures all known app packages are
> # by default purged
> class app::packages {
>$apps = split($::app_packages, ',')
>package { $apps:
>ensure => 'purged'
>}
> }
>
> # Overrides the requested package to be declared
> # present instead of purged.
> define app::myapp($requested_package) {
>include 'app::packages'
>Package<| title == $requested_package |> {
>ensure => 'present'
>}
> }
>
> # no separate package_cleanup required
OK, I wondered whether we could do something like this however - forgive my
naivety - I still can't see how this could be a complete solution without
something like defined().
As an example... your above snippet works fine to ensure already installed
packages remain installed, but what if we wanted to install a brand new version
of app::myapp? Because a 'package' resource with title $requested_package does
not yet exist, the Package<||> collector matches no resources and the new
package is not installed. The only solution that I can come up with is to
check whether such a resource is already defined and, if not, define one.
Your guidance is appreciated.
Tom
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] defined() implicitly requiring resource reference
Hi John,
Thanks for the reply.
On 05/06/2013, at 12:33 AM, jcbollinger wrote:
> On Tuesday, June 4, 2013 1:22:08 AM UTC-5, Tom Lanyon wrote:
> I'm testing a 'cleanup' stage which runs after Stage[main] and removes a
> bunch of package resources.
>
> To do this, I tried a simple check of defined(Package[]) combined with a
> custom facter fact (called 'app_packages'):
>
> > class app::package::cleaner {
> >
> > define check_and_remove {
> > if !defined(Package[$title]) {
> > package { $title:
> > ensure => absent
> > }
> > }
> > }
> >
> > $apps = split($::app_packages, ',')
> > check_and_remove { $apps: }
> >
> > }
> >
> > node 'foo' {
> > class { 'app::package::cleaner': stage => 'cleanup' }
>
> > }
>
> Unfortunately, this results in a dependency cycle. It appears that putting
> the Package[$title] resource reference in defined() actually invokes an
> implicit dependency between my cleanup helper resource in the cleanup stage
> and the original Package resource in the main stage.
>
> > Augeas[redacted] => Service[iptables] => Class[Iptables] => Stage[main] =>
> > Stage[cleanup] => Class[App::Package::Cleaner] =>
> > App::Package::Cleaner::Check_and_remove[package-434] =>
> > Package[package-434] => Exec[app-graceful-restart] => Class[App] =>
> > Stage[main]
>
>> Does it do that when Package[package-434] is already declared elsewhere, or
>> only when it is not?
Sorry, I should have been clearer that this occurs when Package[package-434] IS
declared elsewhere. "!defined(Package[package-434])" therefore is false, so
just by referencing the existing declaration within the defined() call it seems
to incite an implicit dependency.
>> Is this implicit dependency expected behaviour or am I doing something
>> Bad(tm)?
>
> Both.
>
> Supposing that the target package is not declared elsewhere (so that the
> !defined() condition is true) the definition will declare the package itself
> to ensure it absent, and in that case you would expect a relationship between
> the defined-type instance and the resource declared by it. If elsewhere you
> have specific references to that package, applicable resource parameter
> defaults, or collectors that will match that package, then you can get
> relationships with it that are not evident from the defined type body.
>
> On the other hand, defined() is evil. Do not use it. Ever.
I had this discussion with someone on #puppet IRC earlier and they ended up
with "Oh, in your case, defined() is probably actually what you want."
> I usually attribute its malignancy to the parse-order dependency it
> inherently creates -- which is indeed a serious problem -- but in this case I
> think trying to use it to approach your problem it has also obfuscated your
> manifests enough to confuse you about the scope and nature of some of your
> other declarations.
>
> Instead of using defined(), you can apply logic farther upstream to make the
> correct declaration in the first (one) place or to apply resource parameter
> overrides to the correct resources. Alternatively, you can simply determine
> by other means what packages need to be ensured absent, such as by filtering
> a list of possible packages against a list of packages that are supposed to
> be installed. Some of those options may still susceptible to the problem you
> observed, however, if relevant relationships spring from declarations
> elsewhere, as I described they may do.
I've tried this other ways, but here's an example of why farther upstream logic
doesn't work:
define myapp ($requested_package){
package { $requested_package:
ensure => present
}
define package_cleanup {
$installed_package = $title
if $installed_package != $requested_package {
package { $installed_package:
ensure => purged
}
}
}
# assuming a facter fact named 'installed_packages'
package_cleanup { split($::installed_packages, ','): }
}
# now in the case of:
# $::installed_packages = 'one,two,three'
# with:
myapp { 'oneA': requested_package => 'one' }
myapp { 'twoA': requested_package => 'two' }
myapp { 'oneB': requested_package => 'one' }
# we'd end up with package conflicts because
# Myapp[oneA] will define Package[one] (present)
# then define Package[two], Package[three] (absent),
# and Myapp[twoA] will try and define Package[two]
# (present) and fail with a non-unique
[Puppet Users] defined() implicitly requiring resource reference
I'm testing a 'cleanup' stage which runs after Stage[main] and removes a bunch
of package resources.
To do this, I tried a simple check of defined(Package[]) combined with a
custom facter fact (called 'app_packages'):
> class app::package::cleaner {
>
> define check_and_remove {
> if !defined(Package[$title]) {
> package { $title:
> ensure => absent
> }
> }
> }
>
> $apps = split($::app_packages, ',')
> check_and_remove { $apps: }
>
> }
>
> node 'foo' {
> class { 'app::package::cleaner': stage => 'cleanup' }
> }
Unfortunately, this results in a dependency cycle. It appears that putting the
Package[$title] resource reference in defined() actually invokes an implicit
dependency between my cleanup helper resource in the cleanup stage and the
original Package resource in the main stage.
> Augeas[redacted] => Service[iptables] => Class[Iptables] => Stage[main] =>
> Stage[cleanup] => Class[App::Package::Cleaner] =>
> App::Package::Cleaner::Check_and_remove[package-434] => Package[package-434]
> => Exec[app-graceful-restart] => Class[App] => Stage[main]
Is this implicit dependency expected behaviour or am I doing something Bad(tm)?
Tom
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
