Re: [Puppet Users] Re: Cross-module dependencies

2012-01-31 Thread Felix Frank 
Hi,

On 01/30/2012 10:28 PM, Nick wrote:
 It did sound similar, yes - but unless I misunderstand it, not identical.  For
 example, I don't understand how Constraints would avoid the problems with
 unifying resources that Nan mentioned.

as far as I understand, there is no need to merge anything. The catalog
will or will not contain a certain resource. If existing, the resource
will have a certain set of parameteres and metaparameters. Each
constraint can be individually compared to this state. If a constrained
resource does not exist, or any of its (meta)parameters deviate from
what the constraint defines, the catalog is no longer valid.

The beauty of this design is that the language is very expressive and
all validation can be done by the master.

Err, right, John? :-)

 John's example appeared to be wrapping an existing Resource with something 
 which
 puts constraints on it, i.e. a layer on top of Resources. It implies a 
 regular
 Resource to wrap somewhere.
 
 Whereas what I had in mind was something which in principle at least, was more
 basic than a Resource.  With an Assertion there is nothing being managed, or
 mutated, by definition. It defines conditions on a resource (lower-case 'r')
 which can be checked, and merged,  but doesn't imply that any Resource
 (upper-case 'R') need to be declared.  It's quite possible that one wouldn't
 bother, if you don't need to manage or mutate anything.

Ah, so you'd have the agent verify if all assertions (which need to
appear as first-class citizens in the catalog) hold true, and otherwise
fail the catalog?

That strikes me as very elegant indeed.

How will situations be handled where assertions won't hold true until
parts of the catalog have been applied?

 So Resources (upper case 'R') could be thought of as extensions to Assertions
 which also supply rules to mutate a system's state, should the conditions of 
 the
 Assertion not be met, so that the conditions *are* met.

Let's not alienate the community by declassing the proven and beloved
Resources ;-) but I've got to say, this idea does hold merit.

So does the constraint idea. Something tells me that both might be of
benefit, but I'm afraid of years of user confusion to come when everyone
is tasked with understanding the difference between the two and to
decide when to use which.

If we need to take a pick, there's two things I'd have to say for
constraints:
1. They're more closely tailored to the problem at hand.
2. They're stronger in chorus with what puppet is today.

Assertions would probably widen the borders of what's possible with
puppet (and how easy it is), and they would allow/require us to part
with some paradigms. I'm torn whether we want this sooner than seeing
the multiple declaration problem sorted out in a less intrusive way.

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: Cross-module dependencies

2012-01-31 Thread Nick 
On 31/01/12 09:01, Felix Frank wrote:
 Ah, so you'd have the agent verify if all assertions (which need to
 appear as first-class citizens in the catalog) hold true, and otherwise
 fail the catalog?
 
 That strikes me as very elegant indeed.
 
 How will situations be handled where assertions won't hold true until
 parts of the catalog have been applied?

I'm not familiar enough with Puppet's internals to answer that very confidently.
My guess is that one might be able to express requires/before relationships
between Assertions and Resources in order to enforce things like this.

The main implication of that would be to restrict the freedom to assume that
assertions can be applied in any order, because the agent's application of the
catalog would need to be split into a sequence of assertion and mutation
steps.  An Assertion must then not be moved outside the part of the sequence
assigned to it.

N

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Re: Cross-module dependencies

2012-01-31 Thread jcbollinger 


On Jan 31, 3:01 am, Felix Frank felix.fr...@alumni.tu-berlin.de
wrote:
 Hi,

 On 01/30/2012 10:28 PM, Nick wrote:

  It did sound similar, yes - but unless I misunderstand it, not identical.  
  For
  example, I don't understand how Constraints would avoid the problems with
  unifying resources that Nan mentioned.

 as far as I understand, there is no need to merge anything. The catalog
 will or will not contain a certain resource.


That is correct.  Constraints do not have their own separate Resource
instances.  Instead, they use resource type and title, or perhaps a
resource reference, to identify a resource to which they apply.
Traditional Puppet rules are in effect -- there is never more than one
Resource per catalog for each (type, title) pair.


 If existing, the resource
 will have a certain set of parameteres and metaparameters. Each
 constraint can be individually compared to this state. If a constrained
 resource does not exist, or any of its (meta)parameters deviate from
 what the constraint defines, the catalog is no longer valid.


And that's getting around to the interesting part.  Constraints could
leave it there, and that would be a good and useful solution.
Something like that could even be implemented in 2.7, as it introduces
no incompatibility whatever.

I am proposing that constraints might go even further, however, in one
or both of these ways:

1) Where the resource definition to which one or more constraints
apply does not explicitly specify a value for a constrained parameter
or property, applicable constraints are used to *choose* a value
(rather than allowing that property to take a default value).  This
would help with the problem of writing compatible resource definitions
in the first place.  Example:

Somewhere:

package { 'foo':
  provider = 'yum'
}


Somewhere else:

constraint { 'my-package-foo-ensure':
   resource = Package['foo'],
   property = 'ensure',
   forbidden_values = ['absent', 'purged'],
   preferred_values = 'latest'
}


Yet another other place:

constraint { 'your-package-foo-provider':
   resource = Package['foo'],
   property = 'provider',
   allowed_values = 'yum',
}


The definition of Package['foo'] explicitly specifies the 'provider'
property, so Constraint['your-package-foo-provider'] only validates
it, failing catalog compilation if the value does not satisfy the
constraint.  The definition does not explicitly specify a value for
the 'ensure' property, however, so Puppet *chooses* one that is
mutually agreeable to the relevant constraints.  In this case that
would work out to 'latest'.  If there is no mutually agreeable value
then catalog compilation fails.


2) Extending (1), if constraints are declared for a resource that is
not otherwise itself declared for a node, then Puppet synthesizes a
declaration, using the applicable contraints to set values for all
constrained properties.  This could be considered a create_resources()
on steroids.


These would make Constraints much more useful and powerful because
they lessen the burden on module users to write suitable resource
declarations.  Instead of merely testing after the fact whether an
appropriate Resource was declared, item (1) would allow modules to
*collaborate* to ensure that suitable Resources are declared, without
requiring those modules to have any knowledge of one another.  Item
(2) would further reduce the burden on module users by alleviating any
need for them to explicitly declare modules' needed resources
anywhere, though they could provide such declarations if they wished,
subject to the same rules that apply already.


 The beauty of this design is that the language is very expressive and
 all validation can be done by the master.

 Err, right, John? :-)


Indeed, the language is not only expressive, but also completely
compliant with current DSL syntax.  I'm not sure how magical the
implementation of the Constraint [meta]resource type would need to be,
but at least in principle I think it fits very cleanly into Puppet's
model.  Furthermore, because all the work is done on the master,
*nothing* about the agent or any provider needs to change.  Probably
no resource type implementation needs to change, either.


  John's example appeared to be wrapping an existing Resource with something 
  which
  puts constraints on it, i.e. a layer on top of Resources. It implies a 
  regular
  Resource to wrap somewhere.

  Whereas what I had in mind was something which in principle at least, was 
  more
  basic than a Resource.  With an Assertion there is nothing being managed, 
  or
  mutated, by definition. It defines conditions on a resource (lower-case 'r')
  which can be checked, and merged,  but doesn't imply that any Resource
  (upper-case 'R') need to be declared.  It's quite possible that one wouldn't
  bother, if you don't need to manage or mutate anything.

 Ah, so you'd have the agent verify if all assertions (which need to
 appear as first-class citizens in the catalog)

Re: [Puppet Users] Re: Cross-module dependencies

2012-01-30 Thread Felix Frank 
Hi,

On 01/28/2012 04:35 PM, Trevor Vaughan wrote:
 Drawbacks:

  * Requires the user to have an explicit working knowledge of all
modules and namespaces
  * Adds a lot of random logic to the code (unless it becomes a
metaparam of some sort)

You skipped the most important drawback: Commitment to parameterized
classes. The fact that there can be only one place that includes those
classes, and that this singular place must have the whole picture of
what requirements are met, is conceivably a show stopper from my point
of view.

This will work for people that have a functional ENC, I guess, but
should that be a requirement for using Forge modules?

Furthermore, how can modules hope to ever interoperate like this? If all
module classes get parameterized, it will be outright impossible for one
module to ever include another module's classes.
Say module A includes class B::C. As soon as a user installs module A in
addition to B, they have to clean their manifests of inclusions of B::C.

On 01/29/2012 07:39 AM, Brian Gupta wrote:
 It frightens me a bit that I think the correct solution, will be to
 replicate what the distros are doing in Puppetforge. Basically turning
 puppetforge into a massive cross distro metadata repo, with very strict
 contribution standards and rules. This would involve strong rules for
 curated modules that would require manpower to vet (and to
 contribute the modules). 

I honestly don't see the problem. Imagine CPAN was limited to downloads
of tarballs from the website (or even souce control checkouts). I
disbelieve it would be as significant today as it has become.
The same goes for Ruby Gems and all such systems.

As this seems to be a recurring theme: Am I wrong to compare these to
the Forge?

Sincerely,
Felix

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: Cross-module dependencies

2012-01-30 Thread Walter Heck 
Had a bunch more thoughts on this topic, and I feel I agree with
Brian: a forge with one module for every purpose, supported maybe by
library modules (like linux) would be the best. It would mean
consolidating the 17 ssh modules that exist now into one all
encompassing ssh module. In the case of SSH that is totally doable,
but consider the 300 gazillion different use cases of apache and
things get interesting. I think this would mean we'd have appointed
maintainers for each module, and if possible a puppetlabs provided
jenkins install (with community provided jenkins slaves?) that would
do the CI testing every night. That would then make it possible to
have an auto-updated overview for each module of the latest supported
OS's for puppet, and wether that module passes tests for that OS.

This seems like a massive undertaking from where we are now, but it
would in the end make all of our lives a ton easier (one trusted
source for good high quality modules) and reduce the 'problem' of
inter-module dependencies to a minimum. Of course it still exists for
in-house applications that are being puppetised, but it would already
mean the world if they would be able to depend on what the public
trusted modules define.

I personally like the way the drupal module projects work: anyone can
start a project, but they are all hosted on the drupal.org site within
drupal.org version control, and they have teams of code reviewers
maintaining integrity of the module base that lives on drupal.org.

cheers,

Walter

On Mon, Jan 30, 2012 at 10:51, Felix Frank
felix.fr...@alumni.tu-berlin.de wrote:
 Hi,

 On 01/28/2012 04:35 PM, Trevor Vaughan wrote:
 Drawbacks:

  * Requires the user to have an explicit working knowledge of all
 modules and namespaces
  * Adds a lot of random logic to the code (unless it becomes a
 metaparam of some sort)

 You skipped the most important drawback: Commitment to parameterized
 classes. The fact that there can be only one place that includes those
 classes, and that this singular place must have the whole picture of
 what requirements are met, is conceivably a show stopper from my point
 of view.

 This will work for people that have a functional ENC, I guess, but
 should that be a requirement for using Forge modules?

 Furthermore, how can modules hope to ever interoperate like this? If all
 module classes get parameterized, it will be outright impossible for one
 module to ever include another module's classes.
 Say module A includes class B::C. As soon as a user installs module A in
 addition to B, they have to clean their manifests of inclusions of B::C.

 On 01/29/2012 07:39 AM, Brian Gupta wrote:
 It frightens me a bit that I think the correct solution, will be to
 replicate what the distros are doing in Puppetforge. Basically turning
 puppetforge into a massive cross distro metadata repo, with very strict
 contribution standards and rules. This would involve strong rules for
 curated modules that would require manpower to vet (and to
 contribute the modules).

 I honestly don't see the problem. Imagine CPAN was limited to downloads
 of tarballs from the website (or even souce control checkouts). I
 disbelieve it would be as significant today as it has become.
 The same goes for Ruby Gems and all such systems.

 As this seems to be a recurring theme: Am I wrong to compare these to
 the Forge?

 Sincerely,
 Felix

 --
 You received this message because you are subscribed to the Google Groups 
 Puppet Users group.
 To post to this group, send email to puppet-users@googlegroups.com.
 To unsubscribe from this group, send email to 
 puppet-users+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/puppet-users?hl=en.




-- 
Walter Heck

--
follow @walterheck on twitter to see what I'm up to!
--
Check out my new startup: Server Monitoring as a Service @ http://tribily.com
Follow @tribily on Twitter and/or 'Like' our Facebook page at
http://www.facebook.com/tribily

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: Cross-module dependencies

2012-01-30 Thread Trevor Vaughan 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

It is annoying to have everything in a single place that defines the state of 
your nodes but, as you point out, this seems to be the model if you're
using an ENC and that seems to be the recommended practice across the mailing 
list for any sort of scale.

But, you don't need a functional ENC to make this work, you simply need to have 
everything defined at the top level whether it be via node, higher
level class, or ENC.

The main issue here seems to be modules that are trying to be too helpful 
from my reading of the mailing list.

It seems that many would like this to be an anti-pattern:

class foo {

  if not defined(Class['bar']) {
include 'bar'
bar::baz { ... }
  }
}

include 'foo'

Instead, you should be less helpful, and do the following:

include 'foo'
include 'bar'

bar::baz { ... }

So, instead of doing something like, say, setting up IPTables in your module 
(thus creating a cross-module dependency), you should do all of this in
one monolithic place at the node level or a higher level aggregation class 
level.

While this keeps your modules clean, it seems like a lot more effort to 
maintain since the module for nginx should really know what ports it's using
and know how to set up its own firewall rules.

So, the tradeoff is an ENC vs. a large collection of cluttered classes at the 
top level to make sure you don't have cross-module dependencies.

I'm not sure if either is better (or if either is any good at all) but they're 
both functional.

The ability to tag modules as requiring other modules of a particular version 
(ala CPAN, Gems, everything else) would solve this issue as Puppet
would be able to check to make sure that you have the correct version of the 
modules installed prior to compiling the catalog.

Trevor

On 01/30/2012 03:51 AM, Felix Frank wrote:
 Hi,
 
 On 01/28/2012 04:35 PM, Trevor Vaughan wrote:
 Drawbacks:

  * Requires the user to have an explicit working knowledge of all
 modules and namespaces
  * Adds a lot of random logic to the code (unless it becomes a
 metaparam of some sort)
 
 You skipped the most important drawback: Commitment to parameterized
 classes. The fact that there can be only one place that includes those
 classes, and that this singular place must have the whole picture of
 what requirements are met, is conceivably a show stopper from my point
 of view.
 
 This will work for people that have a functional ENC, I guess, but
 should that be a requirement for using Forge modules?
 
 Furthermore, how can modules hope to ever interoperate like this? If all
 module classes get parameterized, it will be outright impossible for one
 module to ever include another module's classes.
 Say module A includes class B::C. As soon as a user installs module A in
 addition to B, they have to clean their manifests of inclusions of B::C.
 
 On 01/29/2012 07:39 AM, Brian Gupta wrote:
 It frightens me a bit that I think the correct solution, will be to
 replicate what the distros are doing in Puppetforge. Basically turning
 puppetforge into a massive cross distro metadata repo, with very strict
 contribution standards and rules. This would involve strong rules for
 curated modules that would require manpower to vet (and to
 contribute the modules). 
 
 I honestly don't see the problem. Imagine CPAN was limited to downloads
 of tarballs from the website (or even souce control checkouts). I
 disbelieve it would be as significant today as it has become.
 The same goes for Ruby Gems and all such systems.
 
 As this seems to be a recurring theme: Am I wrong to compare these to
 the Forge?
 
 Sincerely,
 Felix
 

- -- 
Trevor Vaughan
 Vice President, Onyx Point, Inc.
 email: tvaug...@onyxpoint.com
 phone: 410-541-ONYX (6699)
 pgp: 0x6C701E94

- -- This account not approved for unencrypted sensitive information --
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)

iQEcBAEBAgAGBQJPJoUxAAoJECNCGV1OLcyp7igH/0rroAjC8Ewc9Aw2bdE7gO0N
0KfvzYCTZtJLFTBeNIAErliWd9iR5W84H0j8KJGjPg18qcRcDHjC/hnf5+GV8lIS
6kG3EgwYwyDg8Xc0qAbWubJv7bJ29X4Fc8CCHkq13CkXFM/OqnKpUbXA6X6+o5a/
Hv5Z6WXQjPC7uCupwyqktkjj5sjwvdgniSvKsj6EK3bhGRyMsvJAzmDjucwcRNsM
vz6IG05aFJrYTUp0rZzTJf/HjIPgmD90puoXXSa/RVQnsb3WSw0AwYe1jBAHWap4
pDw+F2qrMdwc9XgQv4ZFFNp/A1OCFh21uW3B1D7XjM+U3QRpmXTKhX71lcqbX08=
=XRv1
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

attachment: tvaughan.vcf

[Puppet Users] Re: Cross-module dependencies

2012-01-30 Thread jcbollinger 


On Jan 30, 2:51 am, Felix Frank felix.fr...@alumni.tu-berlin.de
wrote:
 Hi,

 On 01/28/2012 04:35 PM, Trevor Vaughan wrote: Drawbacks:

   * Requires the user to have an explicit working knowledge of all

 modules and namespaces  * Adds a lot of random logic to the code (unless it 
 becomes a

 metaparam of some sort)

 You skipped the most important drawback: Commitment to parameterized
 classes. The fact that there can be only one place that includes those
 classes, and that this singular place must have the whole picture of
 what requirements are met, is conceivably a show stopper from my point
 of view.


+1


 This will work for people that have a functional ENC, I guess, but
 should that be a requirement for using Forge modules?

 Furthermore, how can modules hope to ever interoperate like this? If all
 module classes get parameterized, it will be outright impossible for one
 module to ever include another module's classes.
 Say module A includes class B::C. As soon as a user installs module A in
 addition to B, they have to clean their manifests of inclusions of B::C.


Right.  The approach depends on using class parameters to pass
requirements, but it depends on classes being *un*parameterized to
apply those requirements -- catch-22.  There is a solution to that
particular problem, however: define requirements via external
[meta]data instead of via parameters.

That's not to say I like the proposal, however. I don't.  In fact I
think it's completely inside-out.  Classes should *know* what other
classes they depend upon; they should not need to be told.

Moreover, I don't see what is gained by passing requirements as
parameters rather than, say, having the user separately declare
optional classes that carry the same information.  For example,
instead of a structure requiring this:

class { 'foo': reqs = ['bar'] }

it should be possible to structure a module's classes so that you can
do this instead:

include 'foo::bar'
include 'foo'

(perhaps in the opposite order).


 On 01/29/2012 07:39 AM, Brian Gupta wrote:

  It frightens me a bit that I think the correct solution, will be to
  replicate what the distros are doing in Puppetforge. Basically turning
  puppetforge into a massive cross distro metadata repo, with very strict
  contribution standards and rules. This would involve strong rules for
  curated modules that would require manpower to vet (and to
  contribute the modules).

 I honestly don't see the problem. Imagine CPAN was limited to downloads
 of tarballs from the website (or even souce control checkouts). I
 disbelieve it would be as significant today as it has become.
 The same goes for Ruby Gems and all such systems.

 As this seems to be a recurring theme: Am I wrong to compare these to
 the Forge?


Yes and no.

On one hand, clearly there are structural similarities between, say,
Puppet modules and Perl modules.  The Forge serves a similar role for
the former to the role CPAN serves for the latter.  I can totally see
the usefulness of tools for automatically installing and removing
Forged Puppet modules.

On the other hand, there are several problems with adopting such an
approach to solve the Puppet module dependency issue, among them:

1) The Forge doesn't address the problem AT ALL as it now stands.  The
relevant resource that package management systems offer is not the
packages themselves (which the Forge has), but rather the metadata
database that each system relies on.  Defining, building, and
maintaining such a database would be a massive and ongoing endeavor,
quite separate from developing and managing the Puppet and module
codebases.

2) A package management system would only work around the problem, not
solve it, and then only among modules from the same repository.

3) In fact, it would not really be the package management system
itself that made anything work.  Instead, authors of Forge modules
would still need to find some way to make their modules interoperate,
and Forge curators would need somehow to ensure that they did.  All
the management software would really contribute is a dependency-aware
ability to automatically install/remove packages.  The system overall
would provide a promise that the particular modules it contains all do
work together, but that's altogether different from a means by which
anyone -- the Forge or otherwise -- can fulfill such a promise.


John

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: Cross-module dependencies

2012-01-30 Thread Nigel Kersten 
On Sat, Jan 28, 2012 at 10:39 PM, Brian Gupta brian.gu...@brandorr.comwrote:

 Nigel,

 It frightens me a bit that I think the correct solution, will be to
 replicate what the distros are doing in Puppetforge. Basically turning
 puppetforge into a massive cross distro metadata repo, with very strict
 contribution standards and rules. This would involve strong rules for
 curated modules that would require manpower to vet (and to contribute the
 modules).


It's worth pointing out here that a massive cross distro repository of
modules doesn't necessarily require strict contribution standards and rules.

There are other options to ensure that high quality modules are available
other than creating a high barrier to entry for the modules themselves.
Exposing community opinion on modules and success rates of people deploying
them can help us all achieve the same goal.

What kind of a repository do we want?

-- 
Nigel Kersten
Product Manager, Puppet Labs

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: Cross-module dependencies

2012-01-30 Thread Felix Frank 
Hi,

thanks for your elaborate design sketch.

Sorry for limiting my quote severely.

On 01/30/2012 06:28 PM, Nick wrote:
   +package { 'libfoo': ensure = present }

Is this different from John's constraint proposal?

To me this didn't become clear: Does the manifest still need to declare
an actual package { libfoo } somewhere, or is this implied by at least
one assertion regarding any of its parameters?

Of the latter is the case, then this is not different from just allowing
puppet to consume multiple declarations of the same resource, along with
all the oft-discussed difficulties.

If instead, there still is that one central resource declaration
somewhere, I'm pretty sure this is the same as constraints.

Which is probably a really neat idea.

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: Cross-module dependencies

2012-01-30 Thread Nick 
On 30/01/12 17:48, Felix Frank wrote:
 Hi,
 
 thanks for your elaborate design sketch.
 
 Sorry for limiting my quote severely.
 
 On 01/30/2012 06:28 PM, Nick wrote:
   +package { 'libfoo': ensure = present }
 
 Is this different from John's constraint proposal?

It did sound similar, yes - but unless I misunderstand it, not identical.  For
example, I don't understand how Constraints would avoid the problems with
unifying resources that Nan mentioned.

John's example appeared to be wrapping an existing Resource with something which
puts constraints on it, i.e. a layer on top of Resources. It implies a regular
Resource to wrap somewhere.

Whereas what I had in mind was something which in principle at least, was more
basic than a Resource.  With an Assertion there is nothing being managed, or
mutated, by definition. It defines conditions on a resource (lower-case 'r')
which can be checked, and merged,  but doesn't imply that any Resource
(upper-case 'R') need to be declared.  It's quite possible that one wouldn't
bother, if you don't need to manage or mutate anything.

So Resources (upper case 'R') could be thought of as extensions to Assertions
which also supply rules to mutate a system's state, should the conditions of the
Assertion not be met, so that the conditions *are* met.

 To me this didn't become clear: Does the manifest still need to declare
 an actual package { libfoo } somewhere, or is this implied by at least
 one assertion regarding any of its parameters?

To be explicit: if an Assertion +package { libfoo: } is declared, it just
means libfoo must be installed for this manifest to work. I don't think it
needs to mandate a declaration of a full-blown package { libfoo: } somewhere.

In fact, I can probably imagine circumstances when something might be invoked
which indirectly takes care of the libfoo package (or file, or whatever) - and
then being forced to manage the libfoo package in Puppet just because you want
to assert its presence could be a liability.


N

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: Cross-module dependencies

2012-01-29 Thread Jeff McCune 
On Fri, Jan 27, 2012 at 5:20 AM, Felix Frank
felix.fr...@alumni.tu-berlin.de wrote:
 Jeff has made a strong point against using virtual resources in modules
 at all, causing me to shift my own views as well.
 If I understand him correctly, one of the chief problems is the high
 probability of accidental collection/realisation of such resources by
 the end user's manifest.

Yep, that's it.

-Jeff

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: Cross-module dependencies

2012-01-29 Thread Brian Gupta 
Nigel,

I just wanted to add, if we do go this route, we should work to support
private forges (module repos) as well.

Cheers,
Brian

On Sun, Jan 29, 2012 at 1:39 AM, Brian Gupta brian.gu...@brandorr.comwrote:

 Nigel,

 It frightens me a bit that I think the correct solution, will be to
 replicate what the distros are doing in Puppetforge. Basically turning
 puppetforge into a massive cross distro metadata repo, with very strict
 contribution standards and rules. This would involve strong rules for
 curated modules that would require manpower to vet (and to contribute the
 modules).

 Maybe, we might want to extend modules so that there are two namespaces,
 one for curated forge modules and another for local modules. (Making the
 local module namespace the default for backwards compatibility.) One
 example of a potential rule would be to require that official modules
 support a mandatory set of popular OSes. Don't allow more than one official
 module per package. e.g.: In the curated section of forge there will be one
 MySQL module. (not to get too ahead of the cart, but I envision a time when
 you can instantiate a MySQL service by just telling puppet to use the Forge
 MySQL module, puppet handles downloading and installing the module and
 figuring out what OS you are running and with the help of the native
 package management, installs and configures the basic MySQL service.) The
 official modules will be curated such that if there is a a common resource
 they need to manage that resource will be split into a separate singular
 dependency module, that incorporates the requirements of
 all dependent forge modules. (Not a bag of common resources in a single
 module, but rather a separate module for each shared resource.)

 Maybe, I am overthinking this, but I think this is the right solution,
 that may require more resources to implement than the community has
 available.

 That leaves us what to do about defined. (Or was that a different
 thread?) In the case of defined, my group has only ever used it once in
 almost 4 years, but it seems from the discussions that there are others
 still using it. Maybe the answer is provide a real alternative first, and
 then go about deprecating it? We wouldn't miss it, but I could see the
 challenges of rewriting a codebase that depends on it, and I wouldn't want
 that rewrite enforced on me, without a solid alternative.

 Cheers,
 Brian

 P.S. - No one is going to really love this solution, including myself, but
 that doesn't necessarily mean it shouldn't be done.


 On Sat, Jan 28, 2012 at 9:18 PM, Nigel Kersten ni...@puppetlabs.comwrote:

 I just wanted to post to this thread to primarily encourage you all to
 keep brainstorming, and to make it clear that I'm paying close attention. :)



 --
 Nigel Kersten
 Product Manager, Puppet Labs



  --
 You received this message because you are subscribed to the Google Groups
 Puppet Users group.
 To post to this group, send email to puppet-users@googlegroups.com.
 To unsubscribe from this group, send email to
 puppet-users+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/puppet-users?hl=en.




 --
 http://aws.amazon.com/solutions/solution-providers/brandorr/




-- 
http://aws.amazon.com/solutions/solution-providers/brandorr/

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: Cross-module dependencies

2012-01-28 Thread Trevor Vaughan 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I've been mulling this over and wanted to get opinions on an, ugly, but 
completely functional approach.

We've been talking about people supplying metadata to describe inter-class 
dependencies (inter-module dependencies really, but hear me out). With the
advent of the parameterized class, you could simply write all ifdef 
dependencies directly into a parameter like follows:

# Get this for the 'contains' function
include 'stdlib'

class foo (
  # No requirements by default
  $reqs = ['']
) {
  if contains($reqs,'bar') {
include 'bar'
  }

  ...some stuff...

  if contains($reqs, 'bar') {
bar::baz { 'something': ... }
  }
}

It's not elegant by any means, but it is functional and since (in theory) 
puppet only includes a class once, then all of the various includes would be
completely skipped.

If would be nice if, in this example, $reqs was actually a class metaparameter 
and Puppet would automatically try to include the class when passed
into that variable.

Benefits:

 * Works with the current language structure
 * Does what you want it to in terms of not needing defined

Drawbacks:

 * Requires the user to have an explicit working knowledge of all modules and 
namespaces
 * Adds a lot of random logic to the code (unless it becomes a metaparam of 
some sort)

Thanks,

Trevor

On 01/27/2012 08:52 AM, Walter Heck wrote:
 Hello,
 
 On Fri, Jan 27, 2012 at 15:20, Felix Frank
 felix.fr...@alumni.tu-berlin.de wrote:
 how I see need for
 explicit module dependencies and a system that can automatically
 download required modules from the forge. I can see this supplementing
 your idea of constraints nicely, but without it, downloading modules
 could quickly become a nightmare for users.
 There's something else we need to think about here. Some modules have
 a soft/conditional requirement for other modules. What I mean is that
 if you don't use certain parts of a module, you don't need the module
 that that part of the code refers to. the only decent way I can come
 up with to solve that is to use what for instance in C is done with
 #IFDEF. That way the module could just ignore modules that it doesn't
 _really_ require.
 
 I for instance have modules that allow you to use different backends
 for monitoring or backups. If requirements were done automatically
 based on the whole module, it would need a myriad of other modules,
 only one of which is ever used.
 
 cheers,

- -- 
Trevor Vaughan
 Vice President, Onyx Point, Inc.
 email: tvaug...@onyxpoint.com
 phone: 410-541-ONYX (6699)
 pgp: 0x6C701E94

- -- This account not approved for unencrypted sensitive information --
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)

iQEcBAEBAgAGBQJPJBXGAAoJECNCGV1OLcypcXkH/3Y2nqqGgJzAKg9YVj/DjiB7
8zbtA7/nVvC8LwtIwwGi7jY+VcbietGwNC8JOoxnTdFN4dCb1xsAcTqzt8p/NXHE
HhwGIG9YGaMoZzvwtfUGc6wrOeqxLvInq2g6e0Qk5QkhBZVg7T5DV4/mvXfheZOR
n1mENjPNMoRONifb24PqxK91CbRtBmJGxEX8b6pDB529oU6aZxNQi6xSn1KSkCJM
SZjVaDoxPqHC4V9L3/J34Rq8H96tfMvTHvSjI3+/nrX80k9MRTkIw5LMIESfTktM
oHKmIXeYcf1yymepuwFmjEgvQ/hp0P5YWsXX3xhE+OCEoaby0AQ6FRHtSJq2y8E=
=qNAo
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

attachment: tvaughan.vcf

Re: [Puppet Users] Re: Cross-module dependencies

2012-01-27 Thread Felix Frank 
Hi John,

thanks for coming up with such elaborate ideas, your input to this group
adds a lot of meat to many discussions.

I can agree with a lot of what you wrote, barring the following remarks:

On 01/26/2012 06:00 PM, jcbollinger wrote:
 Modules provide definitions of resources that they own.  For the most
 part, those definitions should be virtual to avoid unnecessary inter-
 module coupling, but some resources are reasonable to define
 concretely.

Jeff has made a strong point against using virtual resources in modules
at all, causing me to shift my own views as well.
If I understand him correctly, one of the chief problems is the high
probability of accidental collection/realisation of such resources by
the end user's manifest.

On 01/26/2012 06:48 PM, jcbollinger wrote:
 I can imagine many -- perhaps most -- resource definitions being
 replaced or supplemented by constraint declarations.

The model is intriguing, but gives me another usability headache.
Wouldn't this put an end to self-contained modules?

I wrote in a latter mail (is this the same thread? Sorry, I use this
only through Thunderbird and get confused sometimes) how I see need for
explicit module dependencies and a system that can automatically
download required modules from the forge. I can see this supplementing
your idea of constraints nicely, but without it, downloading modules
could quickly become a nightmare for users.

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: Cross-module dependencies

2012-01-27 Thread Walter Heck 
Hello,

On Fri, Jan 27, 2012 at 15:20, Felix Frank
felix.fr...@alumni.tu-berlin.de wrote:
 how I see need for
 explicit module dependencies and a system that can automatically
 download required modules from the forge. I can see this supplementing
 your idea of constraints nicely, but without it, downloading modules
 could quickly become a nightmare for users.
There's something else we need to think about here. Some modules have
a soft/conditional requirement for other modules. What I mean is that
if you don't use certain parts of a module, you don't need the module
that that part of the code refers to. the only decent way I can come
up with to solve that is to use what for instance in C is done with
#IFDEF. That way the module could just ignore modules that it doesn't
_really_ require.

I for instance have modules that allow you to use different backends
for monitoring or backups. If requirements were done automatically
based on the whole module, it would need a myriad of other modules,
only one of which is ever used.

cheers,
-- 
Walter Heck

--
follow @walterheck on twitter to see what I'm up to!
--
Check out my new startup: Server Monitoring as a Service @ http://tribily.com
Follow @tribily on Twitter and/or 'Like' our Facebook page at
http://www.facebook.com/tribily

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: Cross-module dependencies

2012-01-27 Thread Nick 
On 26/01/12 17:48, jcbollinger wrote:
 In particular, it is useful to recognize that dependencies are not just on a
 particular resource generally -- rather, they are on a particular resource
 having certain specific properties.

Yes.

Also: currently in Puppet one cannot say anything about a resource without a
declaration that you will manage it. (Unless perhaps that state happens to be
encapsulated by a Fact, which typically isn't and in many cases couldn't
feasibly be - like the case of file attributes.)

Therefore many dependencies are created only because of a need to check some
state of a resource - which one may not want or need to manage.


 Consider, then, a new metaresource type, Constraint.  The purpose of the
 Constraint resource type would be to allow multiple unrelated classes to
 collaborate on defining the properties of a single resource, and it would do
 so by allowing classes to limit the values that chosen resource properties
 may have.
 
 At compilation time, Puppet would collect and combine all the constraints on
 any particular resource, and use the results to set unspecified property
 values and validate specified ones.  Usage might look something like this:
 
 constraint { 'webserver-httpd_package-present':
 resource = Package['httpd'],
 property = 'ensure',
 forbidden_value = [ 'absent', 'purged' ],
 # also available: allowed_value
 # maybe: default_value
 }
 
 Not only would this nicely meet the needs of different modules to express
 their requirements on shared resources, it would also make it much easier to
 recognize resource conflicts.  If Puppet automatically generated empty
 resource definitions to constrain when it discovered constraints on
 otherwise-undefined resources, then that would also overcome the problem of
 deciding where to define particular resources.
 
 I can imagine many -- perhaps most -- resource definitions being replaced or
 supplemented by constraint declarations.


Here's a slightly different angle.

(Note, I'll use capitalisation distinguish between the resources that exist
outside Puppet, and the Resource instances inside, which model them.)

I think there *is* a case to be made that Puppet needs a new kind of Resource
declaration.  One which promises never to change the state of anything.

Immutability is, I think, the key to allowing multiple declarations of this sort
co-exist. Resources currently have to uniquely own a resource so that they can
safely change it.  As I said, one doesn't always need or want that ownership: we
know the kind of baggage it carries.

Ideally we'd be able to separate out the aspects of a Resource which merely
assert what *should* be the case (ensure = present etc.) from those bits which
would then change the state of the resource if it deviates.

For the sake of discussion I'll call that former kind of declaration an 
Assertion.

To briefly address the points from earlier email: Nan asked how would one
address merging, with respect to the following aspects of a Resource?

 a) unifying before/requires attributes
 b) unifying if/then/else constructs
 c) auditing changes back to their source
 d) unifying hash/array attributes

Well, when using Resources, yes, these make them very hard to unify in general.

However - although this needs more thought - if we could invent some way to
declare mere Assertions, they might do instead of Resources for many cases, and
it might be possible to unify them more simply, because:

 a) Problems related to ordering mostly disappear,

...since nothing is being changed. (Of course, external things which might
change things could be a problem.)

 b) If/then/else clauses can still make sense within definitions of custom
Assertions,

...if they can compose only other Assertions, and the conditions don't change.

 c) Auditing is not a problem: since nothing is changed, there is nothing to
audit.

 d) Hash/array attribute values would need to be resolved on a case-by-case
basis,

...depending on the semantics.  When hash and arrays are semantically
representing are sets, this should be a straightforward and or or operation.
 When the order matters, like a search path, or there is no obvious way to unify
two attributes, then this is an unresolvable contradiction and should generate
an error.  There may be cases in between.

Possibly this doesn't fit all the use-cases which run into cross-module
dependency problems, but might significantly reduce the need to create the
dependencies in the first place.

Anyway, I need to get back to work, I'll try to say more in a later email.


Cheers,

N



-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: Cross-module dependencies

2012-01-27 Thread Felix Frank 
Hi,

On 01/27/2012 02:52 PM, Walter Heck wrote:
 There's something else we need to think about here. Some modules have
 a soft/conditional requirement for other modules. What I mean is that
 if you don't use certain parts of a module, you don't need the module
 that that part of the code refers to. the only decent way I can come
 up with to solve that is to use what for instance in C is done with
 #IFDEF. That way the module could just ignore modules that it doesn't
 _really_ require.

thanks for pointing this out, but it has been covered (I think) in
another thread already:

On 01/19/2012 09:17 PM, Nick Fagerlund wrote:
 So, you can conditionally declare the rule if the defined type is
 available to the autoloader, and otherwise you don't attempt to manage
 the firewall and expect that the user has read the documentation and
 will make a hole for the service themselves.

 if defined(firewall::iptables::rule) {
   firewall::iptables::rule {'mysql_server':
 ...etc. etc.
   }
 }

 See? It's just a way to peek around at what the user has installed.

Thanks again to Nick for this quote, it keeps proving useful ;-)

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Re: Cross-module dependencies

2012-01-27 Thread jcbollinger 


On Jan 27, 7:20 am, Felix Frank felix.fr...@alumni.tu-berlin.de
wrote:
 Hi John,

 thanks for coming up with such elaborate ideas, your input to this group
 adds a lot of meat to many discussions.

 I can agree with a lot of what you wrote, barring the following remarks:

 On 01/26/2012 06:00 PM, jcbollinger wrote:

  Modules provide definitions of resources that they own.  For the most
  part, those definitions should be virtual to avoid unnecessary inter-
  module coupling, but some resources are reasonable to define
  concretely.

 Jeff has made a strong point against using virtual resources in modules
 at all, causing me to shift my own views as well.
 If I understand him correctly, one of the chief problems is the high
 probability of accidental collection/realisation of such resources by
 the end user's manifest.


The core problem to solve is that multiple modules may depend on the
same resources, but each resource can be defined only once.

If modules' resource requirements were invariant then I could be
satisfied with documenting them and requiring the user to provide them
concretely; indeed, that's not very far from what I just proposed.  On
the contrary, however, whether a particular resource is needed by some
module may depend on which of that module's classes are used, and on
which parameters and external data are supplied.  This approach
therefore has its own inherent problem for users: to avoid declaring
unneeded resources.

Given a choice between the problems of avoiding declaring unneeded
resources and of avoiding collecting unwanted resources, I would
choose the latter.  It is easier to deal with, and failure to
completely solve it is no more costly than failure to solve the
alternative problem.


 On 01/26/2012 06:48 PM, jcbollinger wrote:

  I can imagine many -- perhaps most -- resource definitions being
  replaced or supplemented by constraint declarations.

 The model is intriguing, but gives me another usability headache.


From a usability perspective, I think this is a far better proposal
than anything else on the table:
(+) It enables modules to declare their requirements more precisely,
reducing the risk of modules compiling and applying correctly, but
nevertheless failing to achieve the desired configuration.
Constraints are the only proposal currently on the table that provide
that.
(+) A key aspect is that it allows Puppet to pinpoint and diagnose
conflicts at compilation time.
(+) Furthermore, it allows each module to express *only* its
requirements, ignoring properties that it doesn't care about (though
other modules might care about them).  In fact, it rewards modules for
doing so by making them more interoperable.
(+) It does not rely on virtual resources (though it is compatible
with them), so it does not impose any risk of accidental resource
collection.
(+) It could be implemented in a manner that forestalls inter-module
parse order dependencies.

The main cost I see is:
(-) It creates a new distinction between resource properties that are
explicitly specified in resource definitions and those that are not.
For this reason (at least), such a feature could not be introduced
until Telly at the earliest.

That cost is incurred, however, only in manifest sets that use
Constraints, and then only with respect to resources that are targeted
by Constraints.  It is anyway not a usability issue, but rather an
upgrade / conversion / compatibility issue.


 Wouldn't this put an end to self-contained modules?


No.  What makes you think it would?  I think it allows modules to be
*more* self-contained than they currently are.


 I wrote in a latter mail (is this the same thread? Sorry, I use this
 only through Thunderbird and get confused sometimes) how I see need for
 explicit module dependencies and a system that can automatically
 download required modules from the forge. I can see this supplementing
 your idea of constraints nicely, but without it, downloading modules
 could quickly become a nightmare for users.


I have considered that possibility, essentially a package management
system for Puppet modules.  Although such a thing could undoubtedly be
implemented, I don't think it would be very suitable for addressing
the inter-module dependency problem.  Package management systems such
as yum and apt rely on the fact that software dependency graphs are
more or less trees, but I don't think module dependency graphs tend to
be anywhere near as treelike.  I suppose module repository managers
could enforce tree-shaped dependency graphs, but that imposes the
tool's requirements on the problem space instead of relying on a tool
that suits well in the first place.

I do not mean to say that a tool that automatically downloaded and
installed modules from the Forge would be useless -- far from it.  I
just don't think that it would adequately address the inter-module
dependency issue, and therefore I would recommend that such a tool not
even try to do so.  It solves an altogether

Re: [Puppet Users] Re: Cross-module dependencies

2012-01-27 Thread Felix Frank 
Hi,

On 01/27/2012 04:22 PM, jcbollinger wrote:
 From a usability perspective, I think this is a far better proposal
 than anything else on the table:

I've thought of another plus. Even though the design proposal adds to
the DSL (and complexity is generally to be avoided), it does so in a
manner that will not make it necessary for novices (or any end user) to
deal with the particular featureset, but instead limits its target
audience to developers of public modules.

 Wouldn't this put an end to self-contained modules?

 No.  What makes you think it would?  I think it allows modules to be
 *more* self-contained than they currently are.

I was thinking along the lines of currently modules will just install
the packages they need, for instance. But you're right of course -
currently modules can break each other because of this, so a way for the
compiler to clearly pinpoint reasons and locations of mismatches would
indeed be superior.

 I do not mean to say that a tool that automatically downloaded and
 installed modules from the Forge would be useless -- far from it.  I
 just don't think that it would adequately address the inter-module
 dependency issue, and therefore I would recommend that such a tool not
 even try to do so.  It solves an altogether different problem.

Again, I'm inclined to concur. The issue at hand is to stop modules from
breaking each other horribly. Once that's off the table, a module
management system is a whole new issue in and of itself.

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Re: Cross-module dependencies

2012-01-27 Thread jcbollinger 


On Jan 27, 8:01 am, Felix Frank felix.fr...@alumni.tu-berlin.de
wrote:
 Hi,

 On 01/27/2012 02:52 PM, Walter Heck wrote:

  There's something else we need to think about here. Some modules have
  a soft/conditional requirement for other modules. What I mean is that
  if you don't use certain parts of a module, you don't need the module
  that that part of the code refers to. the only decent way I can come
  up with to solve that is to use what for instance in C is done with
  #IFDEF. That way the module could just ignore modules that it doesn't
  _really_ require.

 thanks for pointing this out, but it has been covered (I think) in
 another thread already:

 On 01/19/2012 09:17 PM, Nick Fagerlund wrote:

[...]

And Nick's approach seems good and useful, but I don't think it
addresses Walter's concern.  Walter observes that modules' runtime
inter-module dependencies can depend on how they are used, and he
argues, I think, that that complicates the problem of managing such
dependencies via a package management system.

For example, if module X depends on module Y only for feature
X::ySupport, and I intend never to use X::ySupport, then I might like
to avoid installing module Y.  If I furthermore want to use module Y2
that is incompatible with module Y, then I might *need* to avoid
installing Y.  There are ways to handle problems such as those, but it
all creates a lot of overhead to make everything work together
properly in such a scheme.


John
A hollow voice says Plugh.

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Re: Cross-module dependencies

2012-01-27 Thread jcbollinger 


On Jan 27, 7:58 am, Nick oinksoc...@letterboxes.org wrote:
 On 26/01/12 17:48, jcbollinger wrote:

  In particular, it is useful to recognize that dependencies are not just on a
  particular resource generally -- rather, they are on a particular resource
  having certain specific properties.

 Yes.

 Also: currently in Puppet one cannot say anything about a resource without a
 declaration that you will manage it. (Unless perhaps that state happens to 
 be
 encapsulated by a Fact, which typically isn't and in many cases couldn't
 feasibly be - like the case of file attributes.)


Of course, this is a question of scale and foreknowledge.  One can
easily create a custom fact expressing any or all of the attributes of
a specific file.  I think one could even do it for all of the files in
a directory, or even for an entire directory tree.  Before too long,
however, that's going to become quite slow, and it would be difficult
to use on the Puppet side.  I can't think of a persistent node state
detail that couldn't be expressed via a custom fact, but if you need
many such details, or if you can't be sure in advance which you will
need, then you do have a problem.


 Therefore many dependencies are created only because of a need to check some
 state of a resource - which one may not want or need to manage.


It sounds like you want to perform conditional logic on the agent.  I
am not at all comfortable with that idea, though perhaps I could
become more so.  It entails a much more fundamental change to Puppet
than anything else we have been discussing.  Or did you have something
different in mind?


 Here's a slightly different angle.

[...]

 Ideally we'd be able to separate out the aspects of a Resource which merely
 assert what *should* be the case (ensure = present etc.) from those bits 
 which
 would then change the state of the resource if it deviates.

 For the sake of discussion I'll call that former kind of declaration an 
 Assertion.

[...]

 Possibly this doesn't fit all the use-cases which run into cross-module
 dependency problems, but might significantly reduce the need to create the
 dependencies in the first place.


I could see implementing a version of your Assertion idea as a check-
only Constraint, but I don't think there is yet an infrastructure to
support that part.  However, I don't see at all how the Assertion idea
fits into the cross-module dependency picture.  Perhaps this is
because I don't see the purpose of declaring resources that you do not
intend to manage.

When I consider cross-module dependencies, I think the usual case is
that a module *does* want ensure that particular resource properties
are managed, but it may not care about other properties, and it cares
about who owns Resources only insofar as someone has to own them and
it affects who can manage them.  I can imagine wanting to _assert_
Resource properties only as a workaround for not being able actually
to manage them.  Are there other reasons to want such a thing?


 Anyway, I need to get back to work, I'll try to say more in a later email.


That would help me to determine what I think about the idea.  As it
is, I suspect I don't quite understand what you are hoping to
accomplish with it.


John

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Re: Cross-module dependencies

2012-01-26 Thread jcbollinger 

On Jan 26, 11:00 am, jcbollinger john.bollin...@stjude.org wrote:
 If we must rely only on features already present in Puppet, then I
 think that's heading in the right direction.

On the other hand, if we can wish for new features to address this
area, then there may be altogether different approaches available.  In
particular, it is useful to recognize that dependencies are not just
on a particular resource generally -- rather, they are on a particular
resource having certain specific properties.  For example, a web
server module doesn't just rely on, say, Package['httpd'].  Rather, it
relies on that package having an 'ensure' parameter different from
'absent' and 'purged'.  Puppet DSL does not currently have a means to
express that.

Consider, then, a new metaresource type, Constraint.  The purpose of
the Constraint resource type would be to allow multiple unrelated
classes to collaborate on defining the properties of a single
resource, and it would do so by allowing classes to limit the values
that chosen resource properties may have.

At compilation time, Puppet would collect and combine all the
constraints on any particular resource, and use the results to set
unspecified property values and validate specified ones.  Usage might
look something like this:

constraint { 'webserver-httpd_package-present':
resource = Package['httpd'],
property = 'ensure',
forbidden_value = [ 'absent', 'purged' ],
# also available: allowed_value
# maybe: default_value
}

Not only would this nicely meet the needs of different modules to
express their requirements on shared resources, it would also make it
much easier to recognize resource conflicts.  If Puppet automatically
generated empty resource definitions to constrain when it discovered
constraints on otherwise-undefined resources, then that would also
overcome the problem of deciding where to define particular resources.

I can imagine many -- perhaps most -- resource definitions being
replaced or supplemented by constraint declarations.


John

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Re: Cross-module dependencies

2011-05-25 Thread treydock 
I don't have that directory.  However I came across this module
https://github.com/camptocamp/puppet-sudo which among many things
enlightened me to the new feature in sudo-1.7.2+ of using includedir
and /etc/sudoers.d that you mentioned.  Everything I am using is
defined there except now my backuppc_client module has this...

sudo::directive { backuppc:
ensure  = present,
source  = puppet:///modules/backuppc_client/
backuppc_sudocmd,
}

The backuppc_sudocmd file contains the sudo lines.

Thanks!
- Trey

On May 24, 11:00 pm, Patrick kc7...@gmail.com wrote:
 On May 24, 2011, at 7:58 PM, treydock wrote:









  I have a similar problem I can't seem to solve...here's what I'm
  trying to do.

  I have two modules, backuppc_client and sudo.  Right now I have a node
  variable that I assign to each node that is used in the sudo module's
  template to add the necessary entry.  For example

  ---manifests/node.pp
  node 'client1' {
         $backuppc_sudocmd = backupuser ALL=NOPASSWD: /usr/bin/rsync

         include baseclass
         include backuppc_client::base
  }

  ---modules/sudo/templates/sudoers.erb has...

  % if has_variable?(backuppc_sudocmd) then %
  %= backuppc_sudocmd %
  % end %

  Is there a way for the sudo module to find out or be aware if a client
  has the include backuppc_client in their definition?  Basically I'd
  like the backuppc_client module's to alter the contents of sudoers
  file which I can't imagine how to do as sudoers is defined by the sudo
  module which is part of baseclass and applied to all systems.

 First, do you have a /etc/sudoers.d/ directory?  That would make things 
 much easier.  

 If not, you might want to look into puppet_concat which allows you to build 
 fragments that can be templated to build your file.

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: Cross-module dependencies

2011-05-25 Thread Patrick 
Glad to hear it worked.

On May 24, 2011, at 11:40 PM, treydock wrote:

 I don't have that directory.  However I came across this module
 https://github.com/camptocamp/puppet-sudo which among many things
 enlightened me to the new feature in sudo-1.7.2+ of using includedir
 and /etc/sudoers.d that you mentioned.  Everything I am using is
 defined there except now my backuppc_client module has this...
 
sudo::directive { backuppc:
ensure  = present,
source  = puppet:///modules/backuppc_client/
 backuppc_sudocmd,
}
 
 The backuppc_sudocmd file contains the sudo lines.
 
 Thanks!
 - Trey
 
 On May 24, 11:00 pm, Patrick kc7...@gmail.com wrote:
 On May 24, 2011, at 7:58 PM, treydock wrote:
 
 
 
 
 
 
 
 
 
 I have a similar problem I can't seem to solve...here's what I'm
 trying to do.
 
 I have two modules, backuppc_client and sudo.  Right now I have a node
 variable that I assign to each node that is used in the sudo module's
 template to add the necessary entry.  For example
 
 ---manifests/node.pp
 node 'client1' {
$backuppc_sudocmd = backupuser ALL=NOPASSWD: /usr/bin/rsync
 
include baseclass
include backuppc_client::base
 }
 
 ---modules/sudo/templates/sudoers.erb has...
 
 % if has_variable?(backuppc_sudocmd) then %
 %= backuppc_sudocmd %
 % end %
 
 Is there a way for the sudo module to find out or be aware if a client
 has the include backuppc_client in their definition?  Basically I'd
 like the backuppc_client module's to alter the contents of sudoers
 file which I can't imagine how to do as sudoers is defined by the sudo
 module which is part of baseclass and applied to all systems.
 
 First, do you have a /etc/sudoers.d/ directory?  That would make things 
 much easier.  
 
 If not, you might want to look into puppet_concat which allows you to 
 build fragments that can be templated to build your file.
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 Puppet Users group.
 To post to this group, send email to puppet-users@googlegroups.com.
 To unsubscribe from this group, send email to 
 puppet-users+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/puppet-users?hl=en.
 

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Re: Cross-module dependencies

2011-05-24 Thread treydock 
I have a similar problem I can't seem to solve...here's what I'm
trying to do.

I have two modules, backuppc_client and sudo.  Right now I have a node
variable that I assign to each node that is used in the sudo module's
template to add the necessary entry.  For example

---manifests/node.pp
node 'client1' {
$backuppc_sudocmd = backupuser ALL=NOPASSWD: /usr/bin/rsync

include baseclass
include backuppc_client::base
}

---modules/sudo/templates/sudoers.erb has...

% if has_variable?(backuppc_sudocmd) then %
%= backuppc_sudocmd %
% end %


Is there a way for the sudo module to find out or be aware if a client
has the include backuppc_client in their definition?  Basically I'd
like the backuppc_client module's to alter the contents of sudoers
file which I can't imagine how to do as sudoers is defined by the sudo
module which is part of baseclass and applied to all systems.

Thanks
- Trey

On May 23, 10:11 am, Nigel Kersten ni...@puppetlabs.com wrote:
 On Sun, May 22, 2011 at 11:06 PM, Matthew Macdonald-Wallace 

 li...@truthisfreedom.org.uk wrote:
  Hi all,

  I've got modules for NGINX and PHP-FPM which are working fine, the issue
  I'm running into is that quite often NGinx will start before php-fpm is
  even installed.

  I'm wondering if there's a way in the nginx module to Require the
  PHP-FPM service is started as a dependency of the nginx service,
  something like the following:

 There's also the 'require' function which does an include and sets up Class
 - Class relationships.











  class nginx {

         service { 'nginx':
                 require = Service['php-fpm::php-fpm'],
                 running = true;
         }
  }

  Is this possible in puppet 0.25 on CentOS?

  Thanks in advance,

  Matt

  --
  You received this message because you are subscribed to the Google Groups
  Puppet Users group.
  To post to this group, send email to puppet-users@googlegroups.com.
  To unsubscribe from this group, send email to
  puppet-users+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/puppet-users?hl=en.

 --
 Nigel Kersten
 Product, Puppet Labs
 @nigelkersten

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: Cross-module dependencies

2011-05-24 Thread Patrick 

On May 24, 2011, at 7:58 PM, treydock wrote:

 I have a similar problem I can't seem to solve...here's what I'm
 trying to do.
 
 I have two modules, backuppc_client and sudo.  Right now I have a node
 variable that I assign to each node that is used in the sudo module's
 template to add the necessary entry.  For example
 
 ---manifests/node.pp
 node 'client1' {
$backuppc_sudocmd = backupuser ALL=NOPASSWD: /usr/bin/rsync
 
include baseclass
include backuppc_client::base
 }
 
 ---modules/sudo/templates/sudoers.erb has...
 
 % if has_variable?(backuppc_sudocmd) then %
 %= backuppc_sudocmd %
 % end %
 
 
 Is there a way for the sudo module to find out or be aware if a client
 has the include backuppc_client in their definition?  Basically I'd
 like the backuppc_client module's to alter the contents of sudoers
 file which I can't imagine how to do as sudoers is defined by the sudo
 module which is part of baseclass and applied to all systems.

First, do you have a /etc/sudoers.d/ directory?  That would make things much 
easier.  

If not, you might want to look into puppet_concat which allows you to build 
fragments that can be templated to build your file.

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.