[Puppet Users] Re: extlookup()
Thanks but it remains the same.
Even if I put an ${fqdn}.csv it will read that file but only take the
first entry in the CSV and apply that to every extlookup call in the
module.
It's a little curious. This is running on PE1.2 but I suspect this is
running the same code base as 2.6.9
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: extlookup()
On Oct 24, 2:25 am, "ollies...@googlemail.com"
wrote:
> On Oct 21, 12:55 pm, Andrew Thompson wrote:> On Oct
> 20, 10:18 am, "ollies...@googlemail.com" wrote:
> > > Taking a brief look at extlookup()
>
> > > We have a module to setup resolv.conf based on location so we have a %
> > > {domain}.csv file
>
> > Can you post your $extlookup_datadir and $extlookup_precedence values?
>
> Sure it's set like this so host based, datacenter, country, region
> based on custom fact(s) which would out the location based on a subnet
> lookup to a backend system.
>
> #
> # Global configuration
> #
> $extlookup_datadir = "/etc/puppetlabs/puppet/envs/legacy/extdata"
> $extlookup_precedence = ["%{fqdn}", "%{datacentreid}", "%{ctryid}", "%
> {rgnid}", "common"]
My environment sets the $extlookup_precendence variable a little
different:
$extlookup_precedence = ["${fqdn}", "${datacentreid}", "${ctryid}", "$
{rgnid}", "common"]
I'm not sure of the difference but, the ${} syntax works just fine for
us.
Also you may want to check that the facts are set properly and keep in
mind that they are case sensitive. I make sure that my fqdn is
lowercase before doing anything.
>
>
>
>
>
>
>
>
>
> > > more /etc/puppetlabs/puppet/envs/legacy/extdata/ldt.csv
> > > domainname,my.domain.com
> > > resolvers,10.10.10.10 10.10.10.11
> > > searchpath,sub.my.domain.com sub2.my.domain.com
>
> > > class resolver::setup {
> > > $searchpath = extlookup("searchpath")
> > > $resolvers = extlookup("resolvers")
> > > $resolvdomainname = extlookup("domainname")
>
> > > }
>
> > This looks correct.
>
> > > Each value is getting set with the “domainname” value.
>
> > What happens if you change the order of the values in the csv file?
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: extlookup not working correctly in 2.7.6
looks like that's it. i'll use the workaround nigel recommended in that issue (same as my diff above) until that issue is resolved. thanks for the quick response, R.I.! On Oct 24, 6:10 pm, "R.I.Pienaar" wrote: > - Original Message - > > I'm trying to upgrade from 2.6.7 to to 2.7.6 and I'm running into an > > issue in my extlookup() calls. A lot of the variables which should > > be > > defined aren't defined within extlookup.rb. I've tested 2.6.7 agent > > and 2.7.5 agent against the 2.6.7 master and 2.7.6 master. I > > adjusted > > extlookup.rb slightly to provide some debug output. > > >http://pastie.org/private/jsmyxrfhggcvflxcioibqshows: > > * my 2.7.5 puppet agent failure when run against the 2.7.6 puppet > > master > > * git diff of extlookup.rb in the 2.7.6 branch > > * debug extlookup log output from the 2.7.6 master > > * debug extlookup log output from the 2.6.7 master > > * extlookup precedence set in site.pp > > > The same manifests/extlookup configuration work on the "newpuppet" > > environment when using a 2.6.7 agent against the 2.6.7 master. > > might be this, not sure. > > http://projects.puppetlabs.com/issues/9854 -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: extlookup not working correctly in 2.7.6
I diff'd the extlookup.rb between 2.6.7 and 2.7.6 and noticed this was
different, so I changed it back and things are working again:
--> git diff ./lib/puppet/parser/functions/extlookup.rb
diff --git a/lib/puppet/parser/functions/extlookup.rb b/lib/puppet/
parser/functions/extlookup.rb
index 5fbf26c..818f7c7 100644
--- a/lib/puppet/parser/functions/extlookup.rb
+++ b/lib/puppet/parser/functions/extlookup.rb
@@ -93,7 +93,7 @@ This is for back compatibility to interpolate
variables with %. % interpolation
extlookup_datadir = undef_as('',lookupvar('::extlookup_datadir'))
- extlookup_precedence =
undef_as([],lookupvar('::extlookup_precedence')).collect { |var|
var.gsub(/%\{(.+?)\}/) { lookupvar("::#{$1}") } }
+ extlookup_precedence =
undef_as([],lookupvar('::extlookup_precedence')).collect { |var|
var.gsub(/%\{(.+?)\}/) { lookupvar($1) } }
datafiles = Array.new
On Oct 24, 6:04 pm, Jason Koppe wrote:
> I'm trying to upgrade from 2.6.7 to to 2.7.6 and I'm running into an
> issue in my extlookup() calls. A lot of the variables which should be
> defined aren't defined within extlookup.rb. I've tested 2.6.7 agent
> and 2.7.5 agent against the 2.6.7 master and 2.7.6 master. I adjusted
> extlookup.rb slightly to provide some debug output.
>
> http://pastie.org/private/jsmyxrfhggcvflxcioibqshows:
> * my 2.7.5 puppet agent failure when run against the 2.7.6 puppet
> master
> * git diff of extlookup.rb in the 2.7.6 branch
> * debug extlookup log output from the 2.7.6 master
> * debug extlookup log output from the 2.6.7 master
> * extlookup precedence set in site.pp
>
> The same manifests/extlookup configuration work on the "newpuppet"
> environment when using a 2.6.7 agent against the 2.6.7 master.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: extlookup()
On Oct 21, 12:55 pm, Andrew Thompson wrote:
> On Oct 20, 10:18 am, "ollies...@googlemail.com"
> wrote:
> > Taking a brief look at extlookup()
>
> > We have a module to setup resolv.conf based on location so we have a %
> > {domain}.csv file
>
> Can you post your $extlookup_datadir and $extlookup_precedence values?
>
Sure it's set like this so host based, datacenter, country, region
based on custom fact(s) which would out the location based on a subnet
lookup to a backend system.
#
# Global configuration
#
$extlookup_datadir = "/etc/puppetlabs/puppet/envs/legacy/extdata"
$extlookup_precedence = ["%{fqdn}", "%{datacentreid}", "%{ctryid}", "%
{rgnid}", "common"]
>
> > more /etc/puppetlabs/puppet/envs/legacy/extdata/ldt.csv
> > domainname,my.domain.com
> > resolvers,10.10.10.10 10.10.10.11
> > searchpath,sub.my.domain.com sub2.my.domain.com
>
> > class resolver::setup {
> > $searchpath = extlookup("searchpath")
> > $resolvers = extlookup("resolvers")
> > $resolvdomainname = extlookup("domainname")
>
> > }
>
> This looks correct.
>
>
>
> > Each value is getting set with the “domainname” value.
>
> What happens if you change the order of the values in the csv file?
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: extlookup()
On Oct 20, 10:18 am, "ollies...@googlemail.com"
wrote:
> Taking a brief look at extlookup()
>
> We have a module to setup resolv.conf based on location so we have a %
> {domain}.csv file
Can you post your $extlookup_datadir and $extlookup_precedence values?
>
> more /etc/puppetlabs/puppet/envs/legacy/extdata/ldt.csv
> domainname,my.domain.com
> resolvers,10.10.10.10 10.10.10.11
> searchpath,sub.my.domain.com sub2.my.domain.com
>
> class resolver::setup {
> $searchpath = extlookup("searchpath")
> $resolvers = extlookup("resolvers")
> $resolvdomainname = extlookup("domainname")
>
> }
This looks correct.
>
> Each value is getting set with the “domainname” value.
What happens if you change the order of the values in the csv file?
>
> This is with Puppet v2.6.9
>
> Am I doing something wrong here ?
>
> Thanks
> Paul
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Extlookup() again.
On Wed, Aug 3, 2011 at 3:50 PM, Douglas Garstang wrote:
> Can't get the extlookup() that supports yaml to work.
>
> I did this...
>
> On server:
> mv /usr/lib/ruby/site_ruby/1.8/puppet/parser/functions/extlookup.rb
> /usr/lib/ruby/site_ruby/1.8/puppet/parser/functions/extlookup.rb.orig
> Replaced /usr/lib/ruby/site_ruby/1.8/puppet/parser/functions/extlookup.rb
> with puppet-extlookup/lib/puppet/parser/functions/extlookup.rb
> mkdir /usr/lib/ruby/site_ruby/1.8/puppet/util/extlookup/
> Added puppet-extlookup/lib/puppet/util/extlookup.rb to
> /usr/lib/ruby/site_ruby/1.8/puppet/util
> Added puppet-extlookup/lib/puppet/util/extlookup/csv_parser.rb to
> /usr/lib/ruby/site_ruby/1.8/puppet/util/extlookup
> Added puppet-extlookup/lib/puppet/util/extlookup/yaml_parser.rb to
> /usr/lib/ruby/site_ruby/1.8/puppet/util/extlookup
> Added puppet-extlookup/lib/puppet/util/extlookup/puppet_parser.rb to
> /usr/lib/ruby/site_ruby/1.8/puppet/util/extlookup
>
> Created /etc/puppet/extlookup.yaml:
> ---
> :parser: YAML
> :precedence:
> - environment_%{environment}
> - common
> :yaml:
> :datadir: /etc/puppet/extdata
>
> Created /etc/puppet/extdata/settings.yaml:
> ---
> testkey: testval
>
> Calling with:
> $test_var = extlookup('testkey', '---', 'settings.yaml')
> notice("TEST VAR=$test_var")
>
> And $test_var is always '---'. Replacing settings.yaml with a bogus file
> name also returns '---', which tells me that the file isn't being found.
> What am I missing here? Quite possibly a lot given the really bad docs.
>
> What is 'precedence' for? Not documented.
>
> Doug.
>
>
>
Actually, when I run the puppetmaster in debug mode on the server, I see:
debug: Automatically imported testmodule::test01 from testmodule/test01 into
prod2
debug: extlookup/yaml: looking for key=testkey with default=---
debug: extlookup/yaml: Looking for data in /etc/puppet/extdata
debug: extlookup/yaml: Looking for data in
/etc/puppet/extdata/settings.yaml.yaml
debug: extlookup/yaml: Looking for data in
/etc/puppet/extdata/environment_prod2.yaml
debug: extlookup/yaml: Looking for data in /etc/puppet/extdata/common.yaml
notice: Scope(Class[Testmodule::Test01]): TEST VAR=---
debug: File[/tmp/y]: Adding default for backup
so, it's looking in the right place for the right key it just
doesn't get it...
Doug.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: extlookup == bad?
jcbollinger: > 1) The "include" statement expresses a requirement that the specified > class be included in the resulting catalog, but it says nothing about > the class's parameters. > > 2) If a class is named in at least one "include" statement that is > executed while compiling a catalog, then the effect depends on whether > the named class is "declared" elsewhere in the manifest: > > 2a) If the class is declared, then include statements referencing it > have no additional effect. > > 2b) If the class is not declared, then it is treated as if the class > were declared, without any explicit parameters, at the point where the > first include statement appears. (Fixing the class declaration to the > first include statement is intended to provide backwards > compatibility.) If such a class declares a parameter for which it > does not define a default value, then an error results. I agree with the chorus of +1s for this proposal, largely because I feel that this is how people intuitively *expect* this interaction to work right now. This is the least surprising outcome, in general. I just hope it doesn't require too many extra passes over the AST to ensure declaration status! -- "As I soared high into the tag cloud Xeni Jardin carefully put up for me, I couldn't help but wonder how high we were above the blogosphere." -- Carlos Laviola -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: extlookup == bad?
On Thu, Apr 14, 2011 at 12:24 AM, Felix Frank < felix.fr...@alumni.tu-berlin.de> wrote: > Hi John, > > On 04/13/2011 05:47 PM, jcbollinger wrote: > > Given my position that the good use cases for parameterized classes > > are specialized and few, I tend to agree about different uses. That > > parameterized and ordinary classes are different concepts appears to > > be more a de facto result than a design decision, however. Other > > aspects of Puppet tend to obscure such a distinction. Consider: > > > < snip > > > You raise very good points that I won't argue about. > > > I think often what you might like to do is declare a class's > > parameters once, and elsewhere allow other classes to include that > > class without redeclaring the parameters. For instance, perhaps you > > have a user::virtual class that on some nodes declares virtual LDAP > > users, but on other nodes delares the same virtual users as local. > > Such a class might provide a parameter by which the User provider can > > be specified. Other classes that want to realize users should include > > user::virtual, but they don't care which User provider is in play. > > You get something similar by having some resources in each type of node > require (by metaparameter, not by function) the class in question. > No, it won't get auto-included with default parameter values, but you do > get a meaningful error message. > > > Now that I came up with that example, I realize that it points to an > > improvement that might be doable right away: allow the include > > statement to be used with parameterized classes. That could be > > achieved by slightly (but backwards-compatibly) changing the semantics > > of "include": > > > > 1) The "include" statement expresses a requirement that the specified > > class be included in the resulting catalog, but it says nothing about > > the class's parameters. > > > > 2) If a class is named in at least one "include" statement that is > > executed while compiling a catalog, then the effect depends on whether > > the named class is "declared" elsewhere in the manifest: > > > > 2a) If the class is declared, then include statements referencing it > > have no additional effect. > > > > 2b) If the class is not declared, then it is treated as if the class > > were declared, without any explicit parameters, at the point where the > > first include statement appears. (Fixing the class declaration to the > > first include statement is intended to provide backwards > > compatibility.) If such a class declares a parameter for which it > > does not define a default value, then an error results. > +1 > +1 ! > > Thanks for laying out your position so carefully. Speaking for myself, I > learned quite a bit from your posting. > > Nigel: It would be nice to get "the developers'" position on these > proposals. > We started having this conversation back in October, but no design decision was ever made. Here are some links to tickets related to how include and param classes working together: http://projects.puppetlabs.com/issues/5046 Perhaps with the pending changes to dynamic scoping its worth revisiting this ticket and seeing how include and param classes can work together. http://projects.puppetlabs.com/issues/5089 http://projects.puppetlabs.com/issues/5074 > Cheers, > Felix > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: extlookup == bad?
On Apr 14, 2:24 am, Felix Frank wrote: > Hi John, > > On 04/13/2011 05:47 PM, jcbollinger wrote: [...] > > For instance, perhaps you > > have a user::virtual class that on some nodes declares virtual LDAP > > users, but on other nodes delares the same virtual users as local. > > Such a class might provide a parameter by which the User provider can > > be specified. Other classes that want to realize users should include > > user::virtual, but they don't care which User provider is in play. > > You get something similar by having some resources in each type of node > require (by metaparameter, not by function) the class in question. > No, it won't get auto-included with default parameter values, but you do > get a meaningful error message. Yes, but what I would actually do right now is use extlookup(). It solves the problem cleanly, and the resulting manifests just work. It looks like the problem could as easily be solved with PDL, or with a mechansim for allowing multiple inclusion of parameterized classes. But addressing it via today's version of parameterized classes opens you up to needless errors and therefore makes your manifests harder to maintain. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: extlookup == bad?
Hi John, On 04/13/2011 05:47 PM, jcbollinger wrote: > Given my position that the good use cases for parameterized classes > are specialized and few, I tend to agree about different uses. That > parameterized and ordinary classes are different concepts appears to > be more a de facto result than a design decision, however. Other > aspects of Puppet tend to obscure such a distinction. Consider: > < snip > You raise very good points that I won't argue about. > I think often what you might like to do is declare a class's > parameters once, and elsewhere allow other classes to include that > class without redeclaring the parameters. For instance, perhaps you > have a user::virtual class that on some nodes declares virtual LDAP > users, but on other nodes delares the same virtual users as local. > Such a class might provide a parameter by which the User provider can > be specified. Other classes that want to realize users should include > user::virtual, but they don't care which User provider is in play. You get something similar by having some resources in each type of node require (by metaparameter, not by function) the class in question. No, it won't get auto-included with default parameter values, but you do get a meaningful error message. > Now that I came up with that example, I realize that it points to an > improvement that might be doable right away: allow the include > statement to be used with parameterized classes. That could be > achieved by slightly (but backwards-compatibly) changing the semantics > of "include": > > 1) The "include" statement expresses a requirement that the specified > class be included in the resulting catalog, but it says nothing about > the class's parameters. > > 2) If a class is named in at least one "include" statement that is > executed while compiling a catalog, then the effect depends on whether > the named class is "declared" elsewhere in the manifest: > > 2a) If the class is declared, then include statements referencing it > have no additional effect. > > 2b) If the class is not declared, then it is treated as if the class > were declared, without any explicit parameters, at the point where the > first include statement appears. (Fixing the class declaration to the > first include statement is intended to provide backwards > compatibility.) If such a class declares a parameter for which it > does not define a default value, then an error results. +1 ! Thanks for laying out your position so carefully. Speaking for myself, I learned quite a bit from your posting. Nigel: It would be nice to get "the developers'" position on these proposals. Cheers, Felix -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: extlookup == bad?
On Wed, Apr 13, 2011 at 9:40 AM, R.I.Pienaar wrote:
> Soon puppet will start warning
> if you use dynamic scoping and after that dynamic scoping will stop working
> this means 'include' as we know it will stop working and might as well
> be removed from the language.
Just to address this point... we are throwing deprecation warnings for
dynamic scoping in Statler, e.g. this manifest:
$foo = 'foo_value'
class a {
$bar = 'bar_value'
include b
}
class b inherits c {
notify { $baz: } # should not generate a warning -- inherited from class c
notify { $bar: } # should generate a warning -- uses dynamic scoping
notify { $foo: } # should not generate a warning -- comes from top scope
}
class c {
$baz = 'baz_value'
}
include a
will throw a warning for $bar unless you refer to it as $a::bar. I'm
not seeing the logical jump from there to "include as we know it will
stop working" though RI? Can you elaborate? I believe we achieved
consensus here on the list quite a while ago that it was desirable to
move towards lexical scoping.
We do have work to do to rehabilitate include vs param class
declaration, and more feedback on
http://projects.puppetlabs.com/issues/5046 would be great.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: extlookup == bad?
On Apr 12, 2:13 pm, Dan Bode wrote: > I completely agree that this recommendation is a little lacking without an > existing ENC to back it up. It is lacking to the extent that it depends on an ENC at all, even if PL provided an outstanding one. Allowing an ENC to specify data is good. Requiring an ENC to be used is bad. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: extlookup == bad?
> > > Background is that we are still investing a somewhat > > disproportionate amount of time into puppet development (when using puppet > > for > > managing customers' systems). Nobody's going to pay us for refactoring work. > > A style guide that recommends practices that are prone to need > > refactoring in a year or two would be a huge no. > > > I would account a desire to avoiding future refactoring as an > excellent reason to avoid parameterized classes, at least for now. I > like many of the recommendations in the new style guide, but not so > much the ones related to using parameterized classes (and effectively > making all classes parameterized). > > Were I you, I would look hard at the idea that it is better to allow > manifest compilation to fail than to explicitly include needed > classes: following that advice makes manifests substantially less > resilient and thus harder to maintain. I would also consider the > added work that will be involved if ever a need to change a class's > parameterization arises. The param node classes basically insist that class declaration happens as close to the node scope as possible. Soon puppet will start warning if you use dynamic scoping and after that dynamic scoping will stop working this means 'include' as we know it will stop working and might as well be removed from the language. This means every "node" block is now heavy with lots of parameters and because you have to declare the classes there you should also declare dependencies at that level for it to be clear. It also means each node block is full of data - the parameters - and the data is duplicated between nodes So in order to get DRY node blocks where adding a new node isnt a nightmare or where changing a property on 10s or 100s of nodes isnt a problem I see users using extlookup to lookup data that they then pass into param class declarations due to the lack of PDL. The solution being proposed is so dissfunctional that users are forced to combine the new functionality with that 'hack' you are replacing just to get work done. This discourages code separation into multiple classes - think a class for the package, service and config each. Instead promotes big classes that does lots of tings because writing multi class modules has become MUCH more complex since declaring those classes now requires node data passed between them. The answer here to prevent this duplication and complexity is use an ENC, if you use an ENC to declare your classes the duplication happens outside of Puppet in a place people dont feel the pain. This leads us down a road where an ENC is now a dependency, to use Puppet you need and ENC because Puppet Labs is making language decisions that make any other use impossible or untenable. This is a huge concern, how long till the only way to get complex modeled data support into Puppet do you have only 2 options, code in another language yourself or pay for a commercial ENC? By pushing more and more and more into ENC's aren't we admitting that we've given up the struggle with the Puppet DSL? Aren't we just saying we cant solve these problems with Puppet and its declarative language therefore we go out to a full language like Ruby where we can query databases to access paramters of nodes and other nodes because Puppet doesnt provide a DSL level search functionality? Where we can use full procedural loop constructs to declare classes, where we are creating resources based on hashes passed in from an entirely different language. You might as well just not use Puppet once you've reached this level of complexity in your ENC. -- R.I.Pienaar -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: extlookup == bad?
On Apr 13, 3:17 am, Felix Frank wrote: > On 04/12/2011 08:24 PM, R.I.Pienaar wrote: > > > This demonstrates the problem. You have had to spend time working > > with customers to explain how to build these. It's a very complex > > subject, something our target audience in many cases dont get. They > > do understand extlookup though and it does solve a high % of their > > problems *today*. So why make their life harder. > > RI, > > I strongly disagree. When I ask Puppet Labs for help on good manifest > writing practices, I would not want to be told what crude workarounds > (no offense) are usable today. I want somewhat future-proof manifests > (if at all possible). I am not persuaded that the current style guide achieves that result, nor that extlookup() constitutes a workaround, crude or otherwise. Indeed, I have grave concerns about the maintainability of manifests that make heavy use of parameterized classes (including otherwise ordinary classes "declared" via parameterized-class syntax). With extlookup() in the core in 2.6, I see no reason to think it any less future-proof than the new, raw implementation of parameterized classes. > Background is that we are still investing a somewhat disproportionate > amount of time into puppet development (when using puppet for managing > customers' systems). Nobody's going to pay us for refactoring work. A > style guide that recommends practices that are prone to need refactoring > in a year or two would be a huge no. I would account a desire to avoiding future refactoring as an excellent reason to avoid parameterized classes, at least for now. I like many of the recommendations in the new style guide, but not so much the ones related to using parameterized classes (and effectively making all classes parameterized). Were I you, I would look hard at the idea that it is better to allow manifest compilation to fail than to explicitly include needed classes: following that advice makes manifests substantially less resilient and thus harder to maintain. I would also consider the added work that will be involved if ever a need to change a class's parameterization arises. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: extlookup == bad?
On Apr 13, 2:42 am, Felix Frank
wrote:
> On 04/13/2011 12:34 AM, jcbollinger wrote:
>
> > You can include unparameterized classes any number of times, and it
> > means the same thing whether you include a class once or a hundred
> > times. This is one of the fundamental characteristics of Puppet
> > classes, and a feature that I personally put to good use. There is no
> > reason in principle why you should not be able to include
> > parameterized classes multiple times as well, provided that the
> > parameters are everywhere the same for a given node. My expectation
> > for such a case would be that the effect is the same as including the
> > class only once, just as for unparameterized classes.
>
> John,
>
> I used to think along the same lines. Not anymore. As a matter of fact,
> I don't think non-parameterized classes should be considered equal to
> their parameterized counterparts. They are different concepts with
> different uses (which are unlike defines', see below).
Given my position that the good use cases for parameterized classes
are specialized and few, I tend to agree about different uses. That
parameterized and ordinary classes are different concepts appears to
be more a de facto result than a design decision, however. Other
aspects of Puppet tend to obscure such a distinction. Consider:
* Puppet uses the same term ("class") and largely the same
infrastructure for ordinary and parameterized classes.
* The Language Guide explicitly comingles the concepts. It says "In
Puppet 2.6.0 and later, classes are extended to allow the passing of
parameters into classes." That's a far cry from "a new kind of
resource collection is available", or even from "some classes are
special."
* You can include an ordinary class via the parameterized-class
mechanism (thereby causing Puppet to prevent its inclusion elsewhere),
but you cannot include a parameterized class via 'include', even if it
has default values defined for all its parameters.
* The only way to assign an ordinary class to a run stage is via the
parameterized-class mechanism (which causes Puppet to treat it as a
parameterized class).
* Puppet Labs's official Puppet Style Guide promotes parameterized
classes and largely pushes ordinary ones under the rug.
* Inasmuch as all classes have at least one *meta*-parameter (stage),
Puppet blurs the line between ordinary and parameterized classes.
> It can be useful to be able to include things in multiple places and
> leverage the "singleton" qualities of non-parameterized classes.
> For the things you *do* parameterize, you will typically want to be sure
> it won't get included anywhere far from node level.
I agree, but the Style Guide doesn't appear to do. It advises against
using "include" at all, leaving only the parameterized class syntax
for including classes (for which it uses the term "declare" instead of
"include"). Following that advice effectively makes *all* classes be
treated as if they were parameterized. In that context it makes sense
that the Guide recommends that *all* classes be "declared" as close to
node scope as possible, and that classes actively avoid including
(declaring) other classes, but that doesn't make those recommendations
-- especially the second -- good in their own right.
> Being an 0.25 person, I have to rely on dynamic scoping here and there.
> The scoping problems are horrible, so most of the time I end up
> declaring variables near node level and immediately include the class
> that uses those variables.
> Parameterized classes allow for this to be written somewhat more
> explicitly, but I don't expect them to become as flexible (wrt.
> distributed inclusion) as non-parameterized classes (anymore).
The more I consider the problem, the more I think parameterized
classes are the wrong solution. They address the problem of providing
data to classes by creating a new type of object to which you can
explicitly provide data (if we distinguish between ordinary and
parameterized classes), or by putting substantial restrictions on the
usage of classes to which you can provide data (if we don't
distinguish). That's not a good solution either way.
Extlookup() is a better solution in several ways, but especially
because it is applicable to any class, and because it doesn't
inherently impose limitations on the classes that use it. It's in the
Puppet 2.6 core, so anyone who can use parameterized classes can also
use extlookup().
I haven't studied the PDL proposal carefully; perhaps it would be an
even better solution, but I have reservations about it because it is
based on parameterized classes.
> Technically, you can replace each and every parameterized class with a
> define that bears a label "use this only once per node!" (which I've
> done in places with 0.25, as a matter of fact). Yes, it works. Yes, it
> has largely the same effect as a paramerized class.
>
> But that's horrible design practice! Parameterized classes represent
> collections
Re: [Puppet Users] Re: extlookup == bad?
On 04/13/2011 12:34 AM, jcbollinger wrote: > You can include unparameterized classes any number of times, and it > means the same thing whether you include a class once or a hundred > times. This is one of the fundamental characteristics of Puppet > classes, and a feature that I personally put to good use. There is no > reason in principle why you should not be able to include > parameterized classes multiple times as well, provided that the > parameters are everywhere the same for a given node. My expectation > for such a case would be that the effect is the same as including the > class only once, just as for unparameterized classes. John, I used to think along the same lines. Not anymore. As a matter of fact, I don't think non-parameterized classes should be considered equal to their parameterized counterparts. They are different concepts with different uses (which are unlike defines', see below). It can be useful to be able to include things in multiple places and leverage the "singleton" qualities of non-parameterized classes. For the things you *do* parameterize, you will typically want to be sure it won't get included anywhere far from node level. Being an 0.25 person, I have to rely on dynamic scoping here and there. The scoping problems are horrible, so most of the time I end up declaring variables near node level and immediately include the class that uses those variables. Parameterized classes allow for this to be written somewhat more explicitly, but I don't expect them to become as flexible (wrt. distributed inclusion) as non-parameterized classes (anymore). Don't even get me started on distributed inclusions of classes that do expect dynamically scoped variables. > Defines are altogether different: they are resource types written in > Puppet DSL, much more akin to File or User than to any class. Whereas > it must be an error to include a class two or more times with > inconsistent parameter sets, multiple instantiations of a define > *must* differ, at least in their titles. Whereas multiple inclusions > of the same parameterized class (with consistent parameters) should > mean the same as a single inclusion of that class, multiple > instantiations of a define model separate resources to manage. I *think* Patrick knew that ;) > Defines are nevertheless relevant to the discussion, because they can > indeed serve in most of the roles for which parameterized classes are > suggested. Since parameterized classes can only be included once > anyway, the most significant distinction between a parameterized class > and a define is largely mooted in practice. This then begs the > question: for what purpose are parameterized classes uniquely useful? > I may be missing something, but the only things I come up with are Technically, you can replace each and every parameterized class with a define that bears a label "use this only once per node!" (which I've done in places with 0.25, as a matter of fact). Yes, it works. Yes, it has largely the same effect as a paramerized class. But that's horrible design practice! Parameterized classes represent collections of unique resources. Drawing on the define paradigm to get something similar is ugly. I've talked a to Dan about this during the Berlin Meetup, and it was my impression that the vision is that future manifests should 1) declare most used classes near node level and 2) have interdepending classes produce failures when dependencies are missing (whereas in the 0.25 age, you would include dependencies and be done with it) This sounds very sane to me, but I haven't had a change to really dabble in that type of design myself. For this discussion's sake, I'd be honestly thankful to see some examples of classes that should both be parameterized and multiple-includable. (Yes, I am making up terms as I go ;-) Regards, Felix -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: extlookup == bad?
On Apr 12, 10:49 am, Patrick wrote: > On Apr 12, 2011, at 8:03 AM, jcbollinger wrote: > > > 1) parameterized classes can be included only once, unlike > > unparameterized ones. This tends to require modules to have an > > hierarchical structure, which isn't very suitable in many > > circumstances. (More on that below.) It does, however, sort of > > address > > My understanding is that there's no difference between "parameterized classes > you can include more than once" and defines. If you disagree, please > enlighten me. Stand by for enlightenment. You can include unparameterized classes any number of times, and it means the same thing whether you include a class once or a hundred times. This is one of the fundamental characteristics of Puppet classes, and a feature that I personally put to good use. There is no reason in principle why you should not be able to include parameterized classes multiple times as well, provided that the parameters are everywhere the same for a given node. My expectation for such a case would be that the effect is the same as including the class only once, just as for unparameterized classes. Defines are altogether different: they are resource types written in Puppet DSL, much more akin to File or User than to any class. Whereas it must be an error to include a class two or more times with inconsistent parameter sets, multiple instantiations of a define *must* differ, at least in their titles. Whereas multiple inclusions of the same parameterized class (with consistent parameters) should mean the same as a single inclusion of that class, multiple instantiations of a define model separate resources to manage. Defines are nevertheless relevant to the discussion, because they can indeed serve in most of the roles for which parameterized classes are suggested. Since parameterized classes can only be included once anyway, the most significant distinction between a parameterized class and a define is largely mooted in practice. This then begs the question: for what purpose are parameterized classes uniquely useful? I may be missing something, but the only things I come up with are 1) Run stages. You cannot assign individual resources to stages, only classes, and you can only assign classes to stages via the parameter list syntax. That's rather arbitrary, though, and I wonder whether you can get around it by using "before => Stage['foo']" on ordinary resources instead of "stage => foo" on classes. 2) External node classifiers. I think providing for ENCs to set class data is the main reason we now have parameterized classes. There is no particular reason, though, why parameterized classes needed to be provided to give ENCs such a capability. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: extlookup == bad?
On Apr 12, 2011, at 8:03 AM, jcbollinger wrote: > 1) parameterized classes can be included only once, unlike > unparameterized ones. This tends to require modules to have an > hierarchical structure, which isn't very suitable in many > circumstances. (More on that below.) It does, however, sort of > address My understanding is that there's no difference between "parameterized classes you can include more than once" and defines. If you disagree, please enlighten me. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: extlookup == bad?
On Apr 12, 12:45 am, Dan Bode wrote: > On Mon, Apr 11, 2011 at 9:25 PM, John Warburton wrote: > > > OK, I'll bite > > > In the newly published Style Guide ( > >http://docs.puppetlabs.com/guides/style_guide.html), right at the end it > > says > > > Modules should avoid the use of extlookup() in favor of ENCs or other > > alternatives > > For clarity, this should read: > > in favor of ENCs in combination with parameterized classes > > But there is no reason as to why. > > > > that is partly my fault. > > PuppetLabs, although I can be pretty persuasive :) > > > Extlookup provided some necessary pre-2.6.x functionality, namely, a sane > way to get around dynamic scoping. > > given the option between parameterized class combined with an ENC vs. > extlookup, I choose parameterized classes+ENC. The reason is readability and > encapsulation. I seem to be the person here to most vocally express reservations about parameterized classes in general, but am I really the only one who isn't buying in? I do understand that Puppet Labs put considerable effort into developing the feature, and that they now tout it as one of the key improvements in 2.6, and that it furthermore serves as the foundation for run stages. But none of that makes it technically superior in its current form. Among parameterized classes' problems are 1) parameterized classes can be included only once, unlike unparameterized ones. This tends to require modules to have an hierarchical structure, which isn't very suitable in many circumstances. (More on that below.) It does, however, sort of address 2) parameterized classes present the opportunity to include the same class with different parameters in different places. It's quite reasonable that doing so constitutes an error, but it is unfortunate that the DSL provides the opportunity to commit such an error in the first place. On the other hand, 3) parameterized classes not only permit, but tend to require this: > With param classes, you can build a layered architecture of classes that > pass data to each other through their explicit interfaces. Talk about losing encapsulation! Every parameter a class exposes constitutes a reduction in the encapsulation it provides. That's even more the case when classes expose parameters solely to pass them on to other classes they include. That sort of parameter forwarding is likely to become necessary because the only sane way to manage an architecture wherein parameterized classes include other parameterized classes is to structure it hierarchically. A hierarchical manifest architecture isn't particularly suitable, however, because real system configurations aren't hierarchical. The Style Guide seems to poo-poo that last problem when it says "classes which require other classes should not directly declare them and should instead allow the system to fail if they are not declared by some other means." (Or at least I think that's what it's doing -- I find the Guide's usage of the term "declare" to be a bit ambiguous.) Really, the Guide's recommendation here is a practical necessity, not a stylistic issue, once you're committed to a hierarchy of parameterized classes. But why is it presented as if it were a good approach? It creates points of failure that shouldn't be needed, and it constitutes another encapsulation violation wherever it applies. Furthermore, 4) parameterized classes make configurations harder to maintain because changes to a class's parameterization require corresponding changes wherever that class is included. This is a symptom of the loss of encapsulation inherent in parameterization (3, above), but I call it out specifically because it's more concrete. Additionally, 5) parameterized classes, and especially the term "parameter passing" have a bit of code smell in declarative Puppet DSL. I'm satisfied that the implementation is appropriately declarative, but the concept evidently influences people toward thinking along imperative lines, which typically doesn't fit well with Puppet, and which is already a big enough trap for the unwary. As I understand it, the main objectives of parameterized classes are (a) to provide an alternative to dynamically-scoped variables for influencing classes, and (b) to better document classes' data dependencies. Indeed, the feature does achieve those objectives. I tend to discount the documentation aspect, however, for I don't find class parameters adequate without ordinary text documentation, whereas text documentation can be adequate without parameterization. As for feeding data to classes, parameters certainly aren't the only way, and with all their disadvantages, they usually aren't the best way. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more option
