[Puppet Users] Includes and parametrized class redefinition
Hello I have some code that works like the simplified clase shown below. The idea is to have a define foo that includes a class foo::pre which contains resources that need to be executed before the define is called. The define can be called multiple times but the initialization has to be done only once, which is why it's implemented as a class: class foo::pre { notice(foo::pre) } define foo() { include 'foo::pre' notice(foo) } class x { notice(x) foo { 'x foo': } } class y { notice(y) foo { 'y foo': } } include x include y The issue is that now I need to parametrize foo::pre so that its behavior depends on a variable that exists in foo: class foo::pre($blah) { notice(foo::pre) } define foo() { class { 'foo::pre': blah = 1, } notice(foo) } class x { notice(x) foo { 'x foo': } } class y { notice(y) foo { 'y foo': } } include x include y With this code I get Duplicate definition: Class[Foo::Pre] is already defined. This seems weird to me because I thought the class { 'myclass': } syntax was semantically equivalent to include myclass. Puppet, however, complains that it's being defined twice, even though there's no definition happening there, just inclusion. So, is there a way to redesign this to match the original behavior? I know the current trend is to keep this kind of thing in hiera but this is already a fairly large code base that can't be changed quickly... Thanks in advance, Andre -- You received this message because you are subscribed to the Google Groups Puppet Users group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/dll5ilVD_60J. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Includes and parametrized class redefinition
On Apr 27, 6:40 am, Andre Nathan andre...@gmail.com wrote: Hello I have some code that works like the simplified clase shown below. The idea is to have a define foo that includes a class foo::pre which contains resources that need to be executed before the define is called. The define can be called multiple times but the initialization has to be done only once, which is why it's implemented as a class: class foo::pre { notice(foo::pre)} define foo() { include 'foo::pre' notice(foo)} class x { notice(x) foo { 'x foo': }} class y { notice(y) foo { 'y foo': }} include x include y Great, no problem. The issue is that now I need to parametrize foo::pre so that its behavior depends on a variable that exists in foo: class foo::pre($blah) { notice(foo::pre)} define foo() { class { 'foo::pre': blah = 1, } notice(foo)} class x { notice(x) foo { 'x foo': }} class y { notice(y) foo { 'y foo': }} include x include y With this code I get Duplicate definition: Class[Foo::Pre] is already defined. This seems weird to me because I thought the class { 'myclass':} syntax was semantically equivalent to include myclass. Puppet, however, complains that it's being defined twice, even though there's no definition happening there, just inclusion. This is one of the several drawbacks of Puppet's implementation of parameterized classes. It may be that class { 'myclass':} is semantically equivalent to include 'myclass', but that's beside the point. Unlike ordinary classes, parameterized classes can only be included / declared once. It's not necessarily the declaration syntax that makes the difference, but rather the nature of the class being declared. So, is there a way to redesign this to match the original behavior? I know the current trend is to keep this kind of thing in hiera but this is already a fairly large code base that can't be changed quickly... I know it's not what you want to hear, but Hiera is your best bet. I don't think the code base size is relevant, because I don't think the time and effort to implement an Hiera-based class data source is likely to differ much from what parameterizing all the same classes would require. Or to put it a different way, solving your problem via class parameterization would require *at least* as much shakeup of your code base as would implementing an hiera-based solution. If you insist on using parameterized classes, then you have to come up with a way to ensure that they are declared exactly once if they are needed. If it is harmless to declare them when they are unneeded, then you could declare them unconditionally for every node; otherwise you have a mess to sort out. It is my impression that people with that sort of mess usually end up relying on a complex ENC to deal with it. John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Includes and parametrized class redefinition
Hello John On Friday, April 27, 2012 9:58:09 AM UTC-3, jcbollinger wrote: I know it's not what you want to hear, but Hiera is your best bet. I don't think the code base size is relevant, because I don't think the time and effort to implement an Hiera-based class data source is likely to differ much from what parameterizing all the same classes would require. Or to put it a different way, solving your problem via class parameterization would require *at least* as much shakeup of your code base as would implementing an hiera-based solution. Well, the thing is that we're in the middle of a transition process that is moving from everything is a global variable in the node to parametrized classes, and while they are not perfect, I found that our new code is much saner and easier to debug when using them. So the question is, are parametrized classes now considered deprecated? I remember reading somewhere that improvements were being made for Puppet 2.8... While I can see the advantages of Hiera, it seems to me that it's another instance of the global variable problem if it's used to load values inside some class, and I'd rather not lose the benefit of being able to check a class signature to see immediately what variables it needs, and having the code fail if any is not provided. Thanks, Andre -- You received this message because you are subscribed to the Google Groups Puppet Users group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/eA93Ge_3z2kJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: Includes and parametrized class redefinition
to me that it's another instance of the global variable problem if it's used to load values inside some class, and I'd rather not lose the benefit of being able to check a class signature to see immediately what variables it needs, and having the code fail if any is not provided. If I'm understanding you right, then I get around that problem using the %{calling_module} variable passed from hiera-puppet. For example class foo::data ( $somevar = hiera(bar) ) { When used with a hierarchy similar to %{environment}/%{calling_module} in my Means I can store all my foo variables up into, for example, dev/foo.yaml It's a nice way to be able to see at-a-glance how foo is configured in my environment, and also helps with ambiguous variable names (eg: $port) as grouping them this way offers a kind of scope. Craig -- Craig Dunn | http://www.craigdunn.org Yahoo/Skype: craigrdunn | Twitter: @crayfishX -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Creating configuration for dhcpd
On Apr 26, 6:51 am, Jiří Červenka jiricerven...@gmail.com wrote: Hi, I am trying to set up puppet for creating configuration files for automatic installation. I have a node which runs both puppet master and dhcp server. On this node I wanna create configuration file for dhcpd which will contain static ip address definition for each host. Like this : ... host node1 { hardware ethernet 08:00:27:ed:8f:ea; fixed-address 192.168.2.1; } host node2 { hardware ethernet 08:00:27:df:b4:0c; fixed-address 192.168.2.2; } host node3 { hardware ethernet 78:e7:d1:24:5c:10; fixed-address 192.168.2.3; } ... My configuration: /etc/puppet/manifests/site.pp: import nodes/puppet.pp /etc/puppet/manifests/nodes/puppet.pp: import /etc/puppet/manifests/roles/puppetserver.pp node 'puppet' inherits puppetserver { } #node puppet /etc/puppet/manifests/roles/puppetserver.pp: import /etc/puppet/manifests/roles/basenode.pp node 'puppetserver' inherits basenode { preseed { node1: mac1 = 44-1e-a1-39-05-a0, mac2 = 44:1e:a1:39:05:a0, ip = 192.168.2.1, netmask = 255.255.0.0, node2: mac1 = 78-e7-d1-24-5f-d0, mac2 = 78:e7:d1:24:5f:d0, ip = 192.168.2.2, netmask = 255.255.0.0, node3: mac1 = 78-e7-d1-24-5c-10, mac2 = 78:e7:d1:24:5c:10, ip = 192.168.2.3, netmask = 255.255.0.0, } #preseed include dhcpd } # node This is puppet module for preseed and dhcpd. /etc/puppet/modules/preseed/manifests/init.pp: class dhcpd { package { isc-dhcp-server: ensure = installed, } #package service { isc-dhcp-server: ensure = $ensure, enable = true, require = Package[isc-dhcp-server], } #service file { /etc/default/isc-dhcp-server: ensure = present, path = /etc/default/isc-dhcp-server, mode = 644, content = template(preseed/isc-dhcp-server.erb), require = Package[isc-dhcp-server], notify = Service[isc-dhcp-server], } #file } #class define preseed ($mac1 = dflt, $mac2 = undef, $ip, $netmask) { include dhcpd file { /var/www/preseed_$name.cfg: ensure = present, mode = 644, content = template(preseed/preseed_web.cfg.erb), } #file file { /var/lib/tftpboot/pxelinux.cfg/01-$mac1: ensure = present, mode = 644, content = template(preseed/pxe-config.erb), require = File[/var/www/preseed_$name.cfg], } #file file { /etc/dhcp/dhcpd.conf-$name: ensure = present, path = /etc/dhcp/dhcpd.conf, mode = 644, content = template(preseed/dhcpd.conf.erb), require = Package[isc-dhcp-server], notify = Service[isc-dhcp-server], } #file } #define dhcpd.conf.erb: ... % name.each do |val| -% host %= val % { hardware ethernet %= mac2 %; fixed-address %= ip %; } % end -% When I try to apply this configuration, I receive this message: root@puppet:/# puppet agent --test --server=puppet.domain.com info: Caching catalog for puppet.domain.com err: Could not run Puppet configuration client: Cannot alias File[/etc/ dhcp/dhcpd.conf-node1] to [/etc/dhcp/dhcpd.conf]; resource [File, [/etc/dhcp/dhcpd.conf]] already exists What is the best way to create a configuration that applies only to one node, but have to contain different parameters for each node. Puppet is complaining that you have declared multiple file resources with the same path (/etc/dhcp/dhcpd.conf), which you have. Your manifest is therefore inconsistent: Puppet cannot satisfy all your declarations simultaneously. The problem in this case is this declaration inside your preseed definition: file { /etc/dhcp/dhcpd.conf-$name: ... path= /etc/dhcp/dhcpd.conf, ... } #file Note that you are giving a literal path to a resource declared within a defined type. That would work if you only declared one instance of the defined type, but it creates a conflict if you declare more than one. The solution looks easy in this case, because the problematic declaration doesn't really depend on its context (that should have been your first clue that it didn't belong there). Pull it out and make it a sibling of the preseed { ... } declaration block: preseed { node1: ... } # preseed file { /etc/dhcp/dhcpd.conf: ... } #file John -- You received this message because you are subscribed to the
[Puppet Users] err: Could not send report: Error 400 on SERVER: execution expired
Hey there! All my clients have gone out of sync are have the following line in the puppet.out log file... err: Could not send report: Error 400 on SERVER: execution expired The clients appear to not be updating either... I have restarted the Puppet Master to no avail... Any help appreciated! -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Parametrized classes, definitions and structure
Thanks John for yet another very useful reply, it is much appreciated. I'm going to do some more reading on Hiera and look at reworking my modules. And thank you Ramin for some examples of Hiera in practice, looks very interesting! Going to have a play around with it next week :) Thanks! On Apr 25, 7:01 pm, Ramin K ramin-l...@badapple.net wrote: On 4/24/2012 10:31 AM,AndyTaylorwrote: Thank you very much for your detailed reply John. I hadn't looked at Hiera before, it looks very interesting. On the point of it's not good to include node/site data in manifests, it's kind of essential in my eyes to the setup I am trying to implement. At my company, one server may serve a site which has very different requirements to another, so I really needed something that was very flexible. To expand on the structure I outlined in my previous post: each server has it's own node.pp file. If you need to change something (e.g. MySQL settings, which will probably the most frequent one that needs changing), you can simply edit that server's node manifest file. The great advantage here is that each site's node manifest serves as documentation of what changes have been made to it. It's also very easy to read and understand for someone new to Puppet. From some quick reading of the Hiera posts on the Puppet Labs blog, I guess I could simply have a yaml file for each server with overrides instead of doing it via the node manifest. However, I don't really see the advantage of this approach, except for it would cut out some of the clutter in my Puppet modules. The idea of a hierarchical set of data being considered in line with Facter facts is very interesting though, especially if I combined it with some custom facts of my own; I will do some more reading on that. With the class/definition point, I am wherever possible using classes, as usually I'm dealing with the nature of a node (e.g. you are a webserver, database server etc.) The only case I have used definitions is for these configuration file overrides. So if I'm understanding your comments on class/definition use properly, I think I'm sort of taking the right approach to it... The key concern for me at the moment is flexibility and scalability... none of my stuff is live yet, and I want to make sure I'm using best practices before I put it all in place. :) Mr Bollinger did his usual excellent job of thoroughly explaining the problem space in other parts of this thread. I'd simply add that using a richer data store, such as Hiera, can keep you from doing complicated gymnastics in your module code. I'm using a hiera.yaml file similar to the following with fqdn at the top and common at the bottom. I've added a role fact which is reading /etc/role on the client. You can use any Facter fact such as %{domain} or write your own. %{location} or %{datacenter} seems to be a common one for admins with multiple sites. :hierarchy: - %{fqdn} - %{role}_%{environment} - %{role} - %{environment} - common In my mysql::data class I add a variable and give it a default of 256M. $innodb_buffer_pool_size = hiera('mysql_innodb_buffer_pool_size', '256M') I can then set the following to deal with the specifics of my environment. 512M in my prod environment, production.yaml 4G in role db, db.yaml 1.5G on role db in environment stage, db_stage.yaml 8G on server db01.main.sfo, db01.main.sfo.mydomain.com.yaml Going one step further you can use hiera_array to merge data. For example you might allow a set of servers to monitor all servers and then allow the local monitors as well. Assuming a location fact and the following data files common.yaml monitoring_hosts: - 'mon01.ord' - 'mon02.ord' lax.yaml monitoring_hosts: - 'mon01.lax' class someclass::data { $hosts = hiera_array('monitoring_hosts') } someclass::data::hosts = ['mon01.lax','mon01.ord','mon02.ord'] Hopefully this should give you some ideas of how to use Hiera in your new system. Ramin -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] puppet way of handling rdist and triggers
We have an existing management system of sorts, based on rdist. I'd like to know the best way to migrate it to using puppet. Currently, we have a local binaries tree, rdisted out nightly. We also make use of rdist's extra capability to trigger scripts when and if named files are updated. I'm not sure what the best method would be, of converting this to puppet. I havent found any puppet method that seems clearly designed for, replicate this large tree of files out to clients. The local tree is 260megs, in 5800 files. Converting all that stuff to be package based, would be a chore. It would also meet with a great deal of pushback from admins who are used to making changes by just logging on to the rdist master, changing the tree, and then being done. Building a new package for every change, woud be very unpopular. Even if I limited the puppet involvement, to just if file /xx/yy changes, do z triggers... doesnt that require that it has some master copy of /xx/yy somewhere to compare to? Or is it that the local puppet demon takes timestamps the first time it runs? but then what about when the demon restarts? -- You received this message because you are subscribed to the Google Groups Puppet Users group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/wLuOI6no0JEJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] puppet way of handling rdist and triggers
Hi Philip, I've never used rdist before, but I've just checked the man page quickly... How many servers have you got that's you've got a 260 MiB and 5800 file repository? Is this a raw file count or are some of those files redundant (like ldap.conf going out to every single server and being counted 100+ times)? -Luke On 27/04/12 16:04, Philip Brown wrote: We have an existing management system of sorts, based on rdist. I'd like to know the best way to migrate it to using puppet. Currently, we have a local binaries tree, rdisted out nightly. We also make use of rdist's extra capability to trigger scripts when and if named files are updated. I'm not sure what the best method would be, of converting this to puppet. I havent found any puppet method that seems clearly designed for, replicate this large tree of files out to clients. The local tree is 260megs, in 5800 files. Converting all that stuff to be package based, would be a chore. It would also meet with a great deal of pushback from admins who are used to making changes by just logging on to the rdist master, changing the tree, and then being done. Building a new package for every change, woud be very unpopular. Even if I limited the puppet involvement, to just if file /xx/yy changes, do z triggers... doesnt that require that it has some master copy of /xx/yy somewhere to compare to? Or is it that the local puppet demon takes timestamps the first time it runs? but then what about when the demon restarts? -- You received this message because you are subscribed to the Google Groups Puppet Users group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/wLuOI6no0JEJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. -- Luke Bigum Information Systems Ph: +44 (0) 20 3192 2520 luke.bi...@lmax.com | http://www.lmax.com LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN FX and CFDs are leveraged products that can result in losses exceeding your deposit. They are not suitable for everyone so please ensure you fully understand the risks involved. The information in this email is not directed at residents of the United States of America or any other jurisdiction where trading in CFDs and/or FX is restricted or prohibited by local laws or regulations. The information in this email and any attachment is confidential and is intended only for the named recipient(s). The email may not be disclosed or used by any person other than the addressee, nor may it be copied in any way. If you are not the intended recipient please notify the sender immediately and delete any copies of this message. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden. LMAX operates a multilateral trading facility. Authorised and regulated by the Financial Services Authority (firm registration number 509778) and is registered in England and Wales (number 06505809). Our registered address is Yellow Building, 1A Nicholas Road, London, W11 4AN. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] puppet way of handling rdist and triggers
On Friday, April 27, 2012 8:11:38 AM UTC-7, Luke Bigum wrote: Hi Philip, I've never used rdist before, but I've just checked the man page quickly... How many servers have you got that's you've got a 260 MiB and 5800 file repository? Is this a raw file count or are some of those files redundant (like ldap.conf going out to every single server and being counted 100+ times)? 300+ servers. 5800 files go out,(or more specifically, get synced) identically, to all machines, every night. Probably 200 of those are config files. the rest are specially compiled binaries and util scripts. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/JZsUZ6MP2yoJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] puppet way of handling rdist and triggers
Ok... Not a small job ;-) For config files where it's exactly the same on all machines, that's super easy. It's just stored once on the Puppet Master and file is managed on all Puppet Agents. I'm not sure if rdist has some templating system in it for machine specific config files, but Puppet has a Ruby based templating system that can handle lots of cases. Other more complex files can be done with what are called Concat files - one file is assembled from different fragments. Scripts and especially compile binaries are tricky. Puppet's not the best File Server. The more files you manage as actual Puppet Files the more work is involved. By default the Puppet Master MD5 sums files and the Agent does the same to work out whether a file needs to be copied down. There are some improvements you can make, like changing the checksum method, but 260MiB to every 200 servers? Sounds like a bad idea before you start. What problem are you trying to solve that you think you need Puppet? The recommended Puppet way would be to package your binaries and use Puppet to enforce new versions of the package. You said your Admins are used to just getting on the rdist master, make changes and then practically an rsync? If that's the way you work and the way you want to continue to work then I don't think Puppet's going to beat rdist for this use case. -Luke On 27/04/12 16:15, Philip Brown wrote: On Friday, April 27, 2012 8:11:38 AM UTC-7, Luke Bigum wrote: 300+ servers. 5800 files go out,(or more specifically, get synced) identically, to all machines, every night. Probably 200 of those are config files. the rest are specially compiled binaries and util scripts. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/JZsUZ6MP2yoJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. -- Luke Bigum Information Systems Ph: +44 (0) 20 3192 2520 luke.bi...@lmax.com | http://www.lmax.com LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN FX and CFDs are leveraged products that can result in losses exceeding your deposit. They are not suitable for everyone so please ensure you fully understand the risks involved. The information in this email is not directed at residents of the United States of America or any other jurisdiction where trading in CFDs and/or FX is restricted or prohibited by local laws or regulations. The information in this email and any attachment is confidential and is intended only for the named recipient(s). The email may not be disclosed or used by any person other than the addressee, nor may it be copied in any way. If you are not the intended recipient please notify the sender immediately and delete any copies of this message. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden. LMAX operates a multilateral trading facility. Authorised and regulated by the Financial Services Authority (firm registration number 509778) and is registered in England and Wales (number 06505809). Our registered address is Yellow Building, 1A Nicholas Road, London, W11 4AN. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] puppet way of handling rdist and triggers
On Friday, April 27, 2012 8:28:32 AM UTC-7, Luke Bigum wrote: What problem are you trying to solve that you think you need Puppet? The recommended Puppet way would be to package your binaries and use Puppet to enforce new versions of the package. You said your Admins are used to just getting on the rdist master, make changes and then practically an rsync? If that's the way you work and the way you want to continue to work then I don't think Puppet's going to beat rdist for this use case. I was afraid of that. Well, even if we continue doing rdisted binary distribs, the additional run if changed hooks, I think might be better served by puppet. yes? The current triggers are a bit quirky. And, we do many configs by symlinks on individual machines, to the standard configs in the rdisted common tree. I'd rather have that stuff handled by puppet configs. There are only about 15 triggers, and 10-ish symlinks per machine. For symlinks, I mean stuff like /etc/resolv.conf - /shared/path/resolv/resolv.conf-machinetype I'd rather puppet do actual COPIES of files. That works better with solaris patches. So I'm thinking some kind of puppet class, that autocopies /shared/path/resolv/resolv.conf-machinetypeto /etc/resolv.conf whenever the /shared/path version gets changed by rdist. Is that going to work reliably? Triggers do things like, if config file target has changed, restart demon. So, perfect puppetness there. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/s3m4tvQXRpMJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] scaling puppet, skipping puppetmaster?
I've heard that after (some # of machines) x (some size of manifests) puppet does not scale well, due to bottlenecking on the puppetmaster. Anyone doing large scale use by some other methods? For example, running puppet on each machine individually using cron or something? With the manifest distributed (via NFS? http? something else?) I'd be interested to hear alternatives. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/ucp6vuDFbjYJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] trouble w/ Foreman as ENC, agents and environments
Hi, I have a Puppet Enterprise (2.0) puppetmaster running with Foreman, and having trouble getting it working as an External Node Classifier. In short, if I manually add a test client into site.pp, it will retrieve the proper catalog / classes, but I would like to avoid having to do that. I'm also using environments (production/testing/ development) as well, but think I have most of this configured correctly - on the puppetmaster's puppet.conf - I do have: node_terminus = exec external_nodes = /etc/puppetlabs/puppet/node.rb node.rb does exist and is owned by pe-puppet, executable, etc. I've verified I can view the yaml for a test client using the script. and I have stanzas for each environment in the master's puppet.conf - [production] modulepath=/etc/puppetlabs/puppet/production/modules [testing] ...etc.. and the client's puppet.conf has : environment = testing under the agent stanza. Running: puppet agent --test --environment testing will run, but doesn't apply any classes. I'm not sure what else I may be missing? Thanks in advance. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] trouble w/ Foreman as ENC, agents and environments
On Fri, Apr 27, 2012 at 9:15 PM, droog72 steve@gmail.com wrote: Hi, I have a Puppet Enterprise (2.0) puppetmaster running with Foreman, and having trouble getting it working as an External Node Classifier. In short, if I manually add a test client into site.pp, it will retrieve the proper catalog / classes, but I would like to avoid having to do that. I'm also using environments (production/testing/ development) as well, but think I have most of this configured correctly - on the puppetmaster's puppet.conf - I do have: node_terminus = exec external_nodes = /etc/puppetlabs/puppet/node.rb node.rb does exist and is owned by pe-puppet, executable, etc. I've verified I can view the yaml for a test client using the script. and I have stanzas for each environment in the master's puppet.conf - [production] modulepath=/etc/puppetlabs/puppet/production/modules [testing] ...etc.. and the client's puppet.conf has : environment = testing under the agent stanza. Running: puppet agent --test --environment testing will run, but doesn't apply any classes. I'm not sure what else I may be missing? Thanks in advance. beside checking your logs, you should also make sure you can run that script as the puppetmaster user and not root. Ohad -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: trouble w/ Foreman as ENC, agents and environments
Hi Ohad, Sorry, I'm not really seeing anything useful in the logs, I can see about turning up verbosity, etc. node.rb is owned by puppetmaster user and not root - The ownership / permissions are : -r-xr-xr-x. 1 pe-puppet pe-puppet 705 Apr 27 13:24 node.rb and I was able to run /etc/puppetmaster/puppet/node.rb as the pe- puppet user and retrieve the yaml for a client. thanks again On Apr 27, 2:56 pm, Ohad Levy ohadl...@gmail.com wrote: On Fri, Apr 27, 2012 at 9:15 PM, droog72 steve@gmail.com wrote: Hi, I have a Puppet Enterprise (2.0) puppetmaster running with Foreman, and having trouble getting it working as an External Node Classifier. In short, if I manually add a test client into site.pp, it will retrieve the proper catalog / classes, but I would like to avoid having to do that. I'm also using environments (production/testing/ development) as well, but think I have most of this configured correctly - on the puppetmaster's puppet.conf - I do have: node_terminus = exec external_nodes = /etc/puppetlabs/puppet/node.rb node.rb does exist and is owned by pe-puppet, executable, etc. I've verified I can view the yaml for a test client using the script. and I have stanzas for each environment in the master's puppet.conf - [production] modulepath=/etc/puppetlabs/puppet/production/modules [testing] ...etc.. and the client's puppet.conf has : environment = testing under the agent stanza. Running: puppet agent --test --environment testing will run, but doesn't apply any classes. I'm not sure what else I may be missing? Thanks in advance. beside checking your logs, you should also make sure you can run that script as the puppetmaster user and not root. Ohad -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] scaling puppet, skipping puppetmaster?
Note that using multiple masters is one way to solve this. On Apr 27, 2012 10:46 AM, Philip Brown p...@bolthole.com wrote: I've heard that after (some # of machines) x (some size of manifests) puppet does not scale well, due to bottlenecking on the puppetmaster. Anyone doing large scale use by some other methods? For example, running puppet on each machine individually using cron or something? With the manifest distributed (via NFS? http? something else?) I'd be interested to hear alternatives. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/ucp6vuDFbjYJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Puppet Certificate's
Sorry to resurrect an old(er) thread, but: http://projects.puppetlabs.com/issues/3360#note-33 leads me to believe none of those workarounds are necessary, just allow_duplicate_cert However, https://gist.github.com/0c76fb5b28abfcb2f9d6 That's a proof of concept that I started testing on the DeployStudio side, and will probably fire up some python (once conference extravaganza passes) to iterate over a csv of serial numbers and therefore generate a bunch of certs at once. Allister On Apr 11, 12:32 pm, Gary Larizza g...@puppetlabs.com wrote: Hey Sean, First - congrats on wrangling your Macs with Puppet! Next, I understand and have shared your pain regarding timely imaging of workstations and Puppet cert-wrangling. Generally, I've seen folks do one of a couple of things: 1. Autosign 2. Utilize a CGI script to sign/revoke certs on the master (which can largely be replaced through the use of the `puppet cert` face) 3. Use the same private key everywhere and change the individual node_name Numbers 1 and 2 are largely process around signing individual certs for every node. You COULD even backup the $ssldir on your clients, image the machine, install puppet, restore the $ssldir, and then run Puppet again and Puppet will work fine for your clients. Number 3 is a bit different. With #3, you would have the SAME private cert for EVERY node in your infrastructure. Because of this, the certname must be THE SAME for every node. When you do this, however, Puppet treats every node as if it were the SAME node - so you need a way to de-couple the name of the node as Puppet knows it with the name of the node as the Certificate knows it. The solution is the 'node_name_fact' and 'node_name_value' configuration item in puppet.conf --http://docs.puppetlabs.com/references/stable/configuration.html#noden... You would essentially ship the private cert around to EVERY node, set the node_name_{fact,value} in puppet.conf, and then Puppet would treat each machine as a separate node (even though the certificate is the same everywhere). Obviously there are security implications for this, but some people prefer it to Autosigning. Hopefully, this should help you on your way. On Wed, Apr 11, 2012 at 8:31 AM, Sean McGrath seanc.mcgr...@gmail.comwrote: Firstly my apologies for posting this if it has been answered elsewhere and I missed it while looking. I'm starting to look at using Puppet to manage our fleet of Mac's running OS X in our lab environment and I'm quite impressed with it from my testing so far. I have tested the functionality of the autosign.conf file with the hostnames of the trusted clients in it. However, if I re-image one of the Mac's as we occasionally do that destroys the client certificate that it uses for the puppetca request. Thus the puppet master see's a request with a different certificate from a node with a hostname that has had its trust relationship established with a different certificate. This is probably a noob question but I haven't been able to figure it out. How do I get around this in an automated manner. I don't want to have to revoke certificates each time I re-image a Mac so they can be re-trusted by the puppet master. Is there something like a root certificate I could build into the image to establish the trust relationship easily and securely each time a Mac is re-imaged? many thanks Sean -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. -- Gary Larizza Professional Services Engineer Puppet Labs -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Agent thundering herd solutions?
I'm hoping to have all the puppet agents run on a 20 minute interval with 480sec on either side. I don't want to launch the agent from cron in order to achieve a level of randomness. Is there a splay/random option that can be included in the sysconfig file or config file? I want to continue running the agent as a daemon. Thanks, Ryan -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: scaling puppet, skipping puppetmaster?
Currently we have two puppetmaster servers with ALL requests being load balanced. I use unison to keep the ssl directory in sync between hosts. Each server runs keepalived and requests go to a VIP that exists on one of the servers. The server with the VIP load balances the requests (mod_proxy) between both servers. It's working relatively fine, though it would be ideal to have the agents connect at semi-random intervals in order to reduce thundering herd issues. We are over 500 without any real issues. Also, the decentralized approach works fine but there are caveats related to the use of custom functions that rely on a central server, virtual resources(?), etc. I would try to scale your masters as it's not that hard. -Ryan -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Agent thundering herd solutions?
On Fri, Apr 27, 2012 at 3:54 PM, Ryan Bowlby rbowlb...@gmail.com wrote: I'm hoping to have all the puppet agents run on a 20 minute interval with 480sec on either side. I don't want to launch the agent from cron in order to achieve a level of randomness. Is there a splay/random option that can be included in the sysconfig file or config file? I want to continue running the agent as a daemon. The puppet options related to this are runinterval, splay, and splaylimit. Nan -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Having trouble getting puppet to set users/groups to a defined state
On Apr 18, 6:46 am, jcbollinger john.bollin...@stjude.org wrote: On Apr 17, 5:04 pm,SteveRobertsstrob...@strobe.net wrote: On Apr 17, 6:25 am, jcbollinger john.bollin...@stjude.org wrote: well, allowdupe doesn't fix the issue only masks it. I knew about taht attribute but it just adds a duped group instead of making right the user/group. Indeed, but that's what you asked about. actually I was asking if there was an attribute that would override the other group with the same GID with the new group name. would require a provider that didn't just try to use 'useradd' but it could be written. something like a 'authoritative' attribute that would check for an existing user/group witht he spec'd UID/GID and replace it if it exists. the allowdupe would still be around if you want to just have an additional name with the same UID/GID. Well, I'm not worried about when the human has been told they can make changes but rather when a human (or bad tool) makes a change nad it slips through the cracks initially and goes boom later. I don't understand what you're looking for. You started off dissatisfied that Puppet goes boom immediately, yet you don't want the system to go boom later, either. You understandably don't want to create groups with duplicate gids. What behavior would you actually like to see? Like I mentioned above a 'authoritative' type flag. thinking the .pp would have something like this in it: user { 'tuser': ensure = 'present', comment = 'Test User', gid = '400', home = '/home/tuser', password = '!!', password_max_age = '9', password_min_age = '0', shell= '/bin/sh', uid = '400', authoritative = 'true', } and then when applied on a machine would check for uid,gid 400 and if either were not 'tuser' it would get replaced. Puppet gives the appearance of a lot of intelligence, but it has a long way to go before it can pass a Turing test. Until then, it's unreasonable to expect DWIM. :) No not looking for DWIM. just a User class (and more importantly an underlying provider) that has the options to do what I would like it to do. Puppet also has a mechanism by which you can ensure that otherwise- unmanaged resources of some types are all ensured absent. That's both very powerful and very dangerous, and I advise you to avoid it at least until you have more experience with Puppet. To that end, I'm leaving it as an exercise to determine just what the mechanism is. I may have to poke on how puppet does that. we actually do an absolute /etc/passwd (and friends) in our current conf man system. and yes it does have its pitfalls too. You can do that with Puppet as well, if you prefer, but you cannot safely mix that approach with using User and Group resources. understood. we will be migrating from our current configuration management to puppet. problem one area at a time. don't worry we won't be trying to have two different confman systems trying to manage the same resources. ick that would be a mess. Steve -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: Puppet Certificate's
On Fri, Apr 27, 2012 at 2:22 PM, Allister Banks a...@aru-b.com wrote: Sorry to resurrect an old(er) thread, but: http://projects.puppetlabs.com/issues/3360#note-33 leads me to believe none of those workarounds are necessary, just allow_duplicate_cert Have you tried this out? I've personally never used the allow_duplicate_cert feature, but if it's not working properly then reporting that in the bug is totally the best way to go? However, https://gist.github.com/0c76fb5b28abfcb2f9d6 That's a proof of concept that I started testing on the DeployStudio side, and will probably fire up some python (once conference extravaganza passes) to iterate over a csv of serial numbers and therefore generate a bunch of certs at once. Commented on the gist - that should work as long as you generate the certs ON the puppet master and use the certname of the node you're wanting to provision. Allister On Apr 11, 12:32 pm, Gary Larizza g...@puppetlabs.com wrote: Hey Sean, First - congrats on wrangling your Macs with Puppet! Next, I understand and have shared your pain regarding timely imaging of workstations and Puppet cert-wrangling. Generally, I've seen folks do one of a couple of things: 1. Autosign 2. Utilize a CGI script to sign/revoke certs on the master (which can largely be replaced through the use of the `puppet cert` face) 3. Use the same private key everywhere and change the individual node_name Numbers 1 and 2 are largely process around signing individual certs for every node. You COULD even backup the $ssldir on your clients, image the machine, install puppet, restore the $ssldir, and then run Puppet again and Puppet will work fine for your clients. Number 3 is a bit different. With #3, you would have the SAME private cert for EVERY node in your infrastructure. Because of this, the certname must be THE SAME for every node. When you do this, however, Puppet treats every node as if it were the SAME node - so you need a way to de-couple the name of the node as Puppet knows it with the name of the node as the Certificate knows it. The solution is the 'node_name_fact' and 'node_name_value' configuration item in puppet.conf -- http://docs.puppetlabs.com/references/stable/configuration.html#noden... You would essentially ship the private cert around to EVERY node, set the node_name_{fact,value} in puppet.conf, and then Puppet would treat each machine as a separate node (even though the certificate is the same everywhere). Obviously there are security implications for this, but some people prefer it to Autosigning. Hopefully, this should help you on your way. On Wed, Apr 11, 2012 at 8:31 AM, Sean McGrath seanc.mcgr...@gmail.com wrote: Firstly my apologies for posting this if it has been answered elsewhere and I missed it while looking. I'm starting to look at using Puppet to manage our fleet of Mac's running OS X in our lab environment and I'm quite impressed with it from my testing so far. I have tested the functionality of the autosign.conf file with the hostnames of the trusted clients in it. However, if I re-image one of the Mac's as we occasionally do that destroys the client certificate that it uses for the puppetca request. Thus the puppet master see's a request with a different certificate from a node with a hostname that has had its trust relationship established with a different certificate. This is probably a noob question but I haven't been able to figure it out. How do I get around this in an automated manner. I don't want to have to revoke certificates each time I re-image a Mac so they can be re-trusted by the puppet master. Is there something like a root certificate I could build into the image to establish the trust relationship easily and securely each time a Mac is re-imaged? many thanks Sean -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. -- Gary Larizza Professional Services Engineer Puppet Labs -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. -- Gary Larizza Professional Services Engineer Puppet Labs -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to
Re: [Puppet Users] Re: Puppet Certificate's
Gah, should not end with a question mark -- I've personally never used the allow_duplicate_cert feature, but if it's not working properly then reporting that in the bug report is totally the best way to go? On Fri, Apr 27, 2012 at 6:35 PM, Gary Larizza g...@puppetlabs.com wrote: On Fri, Apr 27, 2012 at 2:22 PM, Allister Banks a...@aru-b.com wrote: Sorry to resurrect an old(er) thread, but: http://projects.puppetlabs.com/issues/3360#note-33 leads me to believe none of those workarounds are necessary, just allow_duplicate_cert Have you tried this out? I've personally never used the allow_duplicate_cert feature, but if it's not working properly then reporting that in the bug is totally the best way to go? However, https://gist.github.com/0c76fb5b28abfcb2f9d6 That's a proof of concept that I started testing on the DeployStudio side, and will probably fire up some python (once conference extravaganza passes) to iterate over a csv of serial numbers and therefore generate a bunch of certs at once. Commented on the gist - that should work as long as you generate the certs ON the puppet master and use the certname of the node you're wanting to provision. Allister On Apr 11, 12:32 pm, Gary Larizza g...@puppetlabs.com wrote: Hey Sean, First - congrats on wrangling your Macs with Puppet! Next, I understand and have shared your pain regarding timely imaging of workstations and Puppet cert-wrangling. Generally, I've seen folks do one of a couple of things: 1. Autosign 2. Utilize a CGI script to sign/revoke certs on the master (which can largely be replaced through the use of the `puppet cert` face) 3. Use the same private key everywhere and change the individual node_name Numbers 1 and 2 are largely process around signing individual certs for every node. You COULD even backup the $ssldir on your clients, image the machine, install puppet, restore the $ssldir, and then run Puppet again and Puppet will work fine for your clients. Number 3 is a bit different. With #3, you would have the SAME private cert for EVERY node in your infrastructure. Because of this, the certname must be THE SAME for every node. When you do this, however, Puppet treats every node as if it were the SAME node - so you need a way to de-couple the name of the node as Puppet knows it with the name of the node as the Certificate knows it. The solution is the 'node_name_fact' and 'node_name_value' configuration item in puppet.conf -- http://docs.puppetlabs.com/references/stable/configuration.html#noden... You would essentially ship the private cert around to EVERY node, set the node_name_{fact,value} in puppet.conf, and then Puppet would treat each machine as a separate node (even though the certificate is the same everywhere). Obviously there are security implications for this, but some people prefer it to Autosigning. Hopefully, this should help you on your way. On Wed, Apr 11, 2012 at 8:31 AM, Sean McGrath seanc.mcgr...@gmail.com wrote: Firstly my apologies for posting this if it has been answered elsewhere and I missed it while looking. I'm starting to look at using Puppet to manage our fleet of Mac's running OS X in our lab environment and I'm quite impressed with it from my testing so far. I have tested the functionality of the autosign.conf file with the hostnames of the trusted clients in it. However, if I re-image one of the Mac's as we occasionally do that destroys the client certificate that it uses for the puppetca request. Thus the puppet master see's a request with a different certificate from a node with a hostname that has had its trust relationship established with a different certificate. This is probably a noob question but I haven't been able to figure it out. How do I get around this in an automated manner. I don't want to have to revoke certificates each time I re-image a Mac so they can be re-trusted by the puppet master. Is there something like a root certificate I could build into the image to establish the trust relationship easily and securely each time a Mac is re-imaged? many thanks Sean -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. -- Gary Larizza Professional Services Engineer Puppet Labs -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options,
[Puppet Users] Puppet stdlib - Is there a validate_in function
Hi List, As part of my module pre-checks I would like to confirm that a value passed in for the state of a package is part of two valid values. Currently I am doing this by: $valid_ensure_values = [ present, absent ] if ! ($::{ensure} in $::{valid_ensure_values}) { $test_ = inline_template(%= ($::apt-cacher-ng::params::valid_ensure_values).join(', ') %) fail(${module_name}::server - Invalid ensure value [currently - ${ensure}], valid values are [$::{valid_ensure_values}]) } I was hoping that I could just simply do a: validate_in($::{ensure}, $::{valid_ensure_values}) I have checked the documentation at https://github.com/puppetlabs/puppetlabs-stdlib and looked in the lib/puppet/parser directory and cannot see anything. Thanks, Peter. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.