Re: [Puppet Users] Beaker - what’s your perspective?
On 4/21/20 3:33 AM, Bram Vogelaar wrote: I too find it excruciating to use, I have described it in the past as having a rage inducing cli, and the lack of documentation makes it hard to use. Which has driven me to use test-kitchen for doing puppet acceptance testing. (with the added benefit of being able to use inspec profiles). In my world view the uptake has been very small outside the vox pupuli and puppet supported modules because of this. After Litmus was released i have never considered Beaker again for any real testing needs, since test-kitchen is very very single node only by design. My experience was similar. I tried to figure out beaker but was unable to make progress due to the lack of documentation and examples (almost two years ago now). With test-kitchen and kitchen-puppet I was able to do my testing and it is my preferred runtime testing framework. I also found the integration with InSpec very useful. -- Konrad Scherer, MTS, Linux Products Group, Wind River -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/e3571a4a-036c-5cea-7ae7-c92e8c3d08eb%40windriver.com.
Re: [Puppet Users] Re: Announce: Facter 2.2.0
On 08/26/2014 04:42 PM, Will Hopper wrote: Hi, Mark! Thanks for raising your concerns on this. This change was actually intentional, as we have been reporting the Ubuntu major release incorrectly for some time in Facter. In most platforms, splitting on the first ‘.’ of an X.Y.Z release would be a sane way of determining the major release, but Ubuntu does its versioning a bit differently. Given the Ubuntu release 10.04, the major version isn't actually 10; it's 10.04 and 10.10 isn't a patch release to 10.04. When Ubuntu does do a minor release for a distribution, they add it as the Z part of the X.Y.Z - for example, 14.04.1 should have a major release of 14.04 and a minor release of 1, not 4. Thus, our inclination here is to correct the long-standing, incorrect version reporting we’ve historically had for Ubuntu. A simple, backwards-compatible way to work with this value in your existing manifests would be to use an approximate regex match on the fact value, i.e: `if $lsbmajdistrelease =~ /^12/ …` I also agree with the rationale of the fix, but I have lsbmajdistrelease in my hiera config and this workaround does not work there. A quick grep through my puppet manifest repo shows several modules like postgres broken by this change [1]. Some warnings, deprecations, clear release notes and work to make sure modules are compatible with the change would have made this transition much less painful for everyone. A backwards compatible release shouldn't require so many code changes to maintain behavior. [1]: https://github.com/puppetlabs/puppetlabs-postgresql/blob/master/manifests/params.pp#L121 -- Konrad Scherer, MTS, Linux Products Group, Wind River -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/53FDD3D6.8020908%40windriver.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] Re: Puppet 3.6.0... and scaling?
On 05/27/2014 05:39 PM, Andy Parker wrote: On Tuesday, May 27, 2014 5:53:29 AM UTC-7, Konrad Scherer wrote: On 14-05-22 03:21 PM, Daniele Sluijters wrote: The environment caching is already there, use the environment_timeout setting. Mine is set to unlimited and I reload at deploy time by touching tmp/restart.txt. This so far seems to work really well. Thanks for the suggestion. I have also been dealing with high CPU load on my puppet masters since 3.5.0. Triggering the puppet master restart makes a lot of sense. I am using a git post commit hook to reload the puppet configs on my three puppet masters and I have added the code to restart the puppet rack app after changes have been detected. I will report back once I have had some time to analyze the results. By puppet configs do you mean the puppet manifest files? Under rack the puppet master doesn't watch nor reload the puppet.conf file. That wasn't clear, sorry. I mean puppet manifest *.pp files, not the conf files. This seems like a major change from previous puppet versions. I have been using Puppet since 2.6 and any changes to puppet configs on the master were always picked up immediately. Is this because the puppet master was not doing any caching or is the puppet master watching the puppet configs for changes? Has this behavior now changed? Will changes to puppet manifests on the master only be detected after the environment_timeout has expired? The caching behavior for directory environments is a bit different from the previous system. I've been working on a blog post about this, but haven't finished it yet :( First off, what is being cached? When we talk about caching environments we are talking (mostly) about caching the parsed and validated form of the manifest files. This saves the cost of disk access (stat to find files, reads to list directory contents, reads to fetch manifest file contents) as well as a certain amount of CPU use (lexing, parsing, building an AST, validating the AST). This is what has been part of the cache for quite a while now. What has changed is the cache eviction mechanism that is used. The directory environments employ a different eviction and caching system that the legacy environments. The legacy environments had singleton instances that the master would never get rid of to track each individual environment. The environments have references to the AST objects as well as to WatchedFile objects, which are used to track changes to the mtime of the manifest files. The WatchedFile instances would stat the file that they are supposed to watch, but limit the stat calls to happen no more often than the filetimeout setting specified. Before Puppet 3.4 (? 3.5? I lose track of what version had what change) the WatchedFile instances would get interrogated throughout the compilation process. In fact, every time it asked if one file had changed it ended up asking if *any* files had changed. There were a lot of side effects of that, but I won't derail the conversation to go in to that. In 3.4 (or was it 3.5) the legacy environment system was changed to only check if files had changed at the beginning of a compile. This, however, meant that it would still in the worst case issue a stat call for every manifest file, in the best case (depending on your viewpoint) issue no stat calls because the filetimeout had not expired, or it would be some in-between number of stats. The in-between number of stats is possible because each WatchedFile instance had its own timer for the filetimeout and so they can drift apart over time, which allowed it to detect changes to some files but not others. For the directory environments we chose a different system for managing the caches. The watch word here was KISS. Under the new system there isn't any file watching involved (right now, that is. There is a PR open to introduce a 'manual' environment_timeout system), instead once an environment has loaded a file it simply holds onto the result. All of the caching now comes down to holding onto just the environment instance. Cache eviction is just about when puppet should throw away that environment instance and re-create it. There are a few options here: * environment_timeout = 0 : Good to use in an development setup where you are editing manifest and running an agent to see what happens. Nothing will be cached and so the full lex, parse, validate overhead is incurred on every agent catalog request. * environment_timeout = some number : If you have spikey agent requests. For instance, if you don't run agents continually and instead only trigger them as needed with mco. In that case you know that from the first agent checking in to the last agent checking in it is 20 minutes and you do this kind of on demand deploy once a day, then just set the timeout to 30m (20 minutes + some extra time to deal with variance). This way the cache will last through the whole run
Re: [Puppet Users] Re: Puppet 3.6.0... and scaling?
On 14-05-22 03:21 PM, Daniele Sluijters wrote: The environment caching is already there, use the environment_timeout setting. Mine is set to unlimited and I reload at deploy time by touching tmp/restart.txt. This so far seems to work really well. Thanks for the suggestion. I have also been dealing with high CPU load on my puppet masters since 3.5.0. Triggering the puppet master restart makes a lot of sense. I am using a git post commit hook to reload the puppet configs on my three puppet masters and I have added the code to restart the puppet rack app after changes have been detected. I will report back once I have had some time to analyze the results. This seems like a major change from previous puppet versions. I have been using Puppet since 2.6 and any changes to puppet configs on the master were always picked up immediately. Is this because the puppet master was not doing any caching or is the puppet master watching the puppet configs for changes? Has this behavior now changed? Will changes to puppet manifests on the master only be detected after the environment_timeout has expired? Thank you in advance for any insight. -- Konrad Scherer, MTS, Linux Products Group, Wind River -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/53848ABE.8040806%40windriver.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] Mirroring apt.puppetlabs.com creates extra directories
On 13-12-19 09:12 PM, Aaron Hicks wrote: Hi there, Using the recommended rsync command: rsync -av --copy-links --del rsync://apt.puppetlabs.com/packages/apt/ http://www.google.com/url?q=http%3A%2F%2Fapt.puppetlabs.com%2Fpackages%2Fapt%2Fsa=Dsntz=1usg=AFQjCNHk-fZHUKWgwdEHx7_08rxVGvJoEw apt/ this mirrors the repository, but seems to create a lot of extra directories in the dist directory. These generally take the form of a distribution name followed by a timestamp number, e.g. lucid-20131219145258860605811 are these directories real? If they do exist on the puppetlabs apt server are they made invisible with .htaccess or directives in the vhost configuration? (if they are could we see that please) The suggested rsync command works but the --copy-links option uses a lot of disk space and bandwidth unnecessarily. I use rsync to mirror puppetlabs, Fedora, EPEL, CentOS, Mint, OpenSuSE and I modified the fantastic mirror-fedora [1] script to handle multiple rsync repos. If you use the rsync options from the mirror-fedora script what you get is symlinks from the distribution name to the current distribution-timestamp directory. ll mirror/puppetlabs/apt/dists Dec 20 00:09 lucid - lucid-20131219145258860605811 Mar 1 2013 lucid-20130301133454158727269 Jul 29 12:31 lucid-20130729093049741633793 Dec 11 00:05 lucid-20131209174434251944132 Dec 19 17:56 lucid-20131219145258860605811 Dec 20 00:09 precise - precise-20131219151742127528147 Dec 11 00:05 precise-20131209175838499943714 Dec 19 18:22 precise-20131219151742127528147 snip This allows Puppetlabs to atomically update the repo and it has been working well for me for over a year now. I have uploaded my modifications to mirror-fedora [2], but I will warn you that it is rough and I did not make any effort to make it usable by others. That said I hope it is useful to you. Puppetlabs should probably update the apt repo README to update the rsync options. Next time I am in Jira I will create a bug report. [1]: https://github.com/dlbewley/mirror-fedora/blob/master/mirror-fedora [2]: https://github.com/kscherer/mirror-fedora -- Konrad Scherer, MTS, Linux Products Group, Wind River -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/52B45A28.6060404%40windriver.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Puppet-dashboard work well but can't see any node
On 13-06-17 04:04 AM, Rajat Patel wrote: Hi Guys, We have Cent OS 6.4 server which is puppet master server, its take all mix environment(fedora/redhat/centos/windows). Right now we have add 2 node one from ubuntu 12.10 and fedora 18. If the dashboard is using ruby 1.8.x, it cannot process reports coming from puppet using ruby 1.9.x. I know F18 is ruby 1.9 and CentOS 6.4 is ruby 1.8, so that may be the problem. http://projects.puppetlabs.com/issues/10422 Do you see the error messages in Dashboard? -- Konrad Scherer, Sr. Engineer, Linux Products Group, Wind River direct 613-963-1342 fax 613-592-2283 -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Puppet in Novell Zenworks ZCM 11
On 03/27/2013 04:47 PM, stewart wrote: Hello.. I'm in charge of setting up a ZCM server to replace an older ZLM instance. One of the reasons given for sticking with Novell and the new version is that puppet is used as the linux engine for applying policies. Unfortunately when that decision was made it wasn't realised that the version of puppet supplied with ZCM is 0.28.. and therefore it's missing a lot of the functionality we kind of expected we'd have. (file_line being the really useful omission) Is there anyone on this list who happens to have some experience with the way Novell has crippl... I mean customised the puppet setup in ZCM? Out of the box the puppet binary doesn't even work from the command line, although I've managed to get that going with a few symlinks for the ruby libraries into /usr/lib, there's a heap of differences to standard puppet (the ZCM processes do all the work of puppet master, so there's none of that) and I'm new to puppet as it is, so it's turning out to be a hard slog. Novell's documentation on the matter doesn't help, it basically says see puppetlabs.com for more information I have managed to get a recent puppet running on SLED 11 SP2. Much of what I did should also work with Zenworks, but I can't guarantee it. The first step is to get access to newer packages and here OBS has been very helpful. Here is a list of OBS repos that I am using. http://download.opensuse.org/repositories/devel:/tools:/scm/SLE_11_SP2/ http://download.opensuse.org/repositories/devel:/languages:/ruby/SLE_11_SP2/ http://download.opensuse.org/repositories/devel:/languages:/ruby:/extensions/SLE_11_SP2/ http://download.opensuse.org/repositories/systemsmanagement:/puppet/SLE_11_SP2 http://download.opensuse.org/repositories/server:/monitoring/SLE_11_SP2/ http://download.opensuse.org/repositories/devel:/tools:/scm:/svn/SLE_11/ http://download.opensuse.org/repositories/devel:/tools/SLE_11_SP2/ http://download.opensuse.org/repositories/Apache/SLE_11_SP2/ Replace http://download.opensuse.org/repositories/ with rsync://rsync.opensuse.org:/buildservice-repos/ to access over rsync. This should allow you to get a more recent version of ruby, puppet and facter installed. I also forced the package provider to use zypper. I hope that helps. Good luck. -- Konrad Scherer, Sr. Engineer, Linux Products Group, Wind River direct 613-963-1342 fax 613-592-2283 -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Broken rsync mirroring for PuppetLabs APT repo
On 12/28/2012 12:48 PM, Matthaus Owens wrote: Arnaud, What flags are you passing to rsync? The README at http://apt.puppetlabs.com/README.txt offers a sample rsync command which includes the --copy-links flag. When I rsync using copy links, my dists directory looks like: total 0 drwxrwsr-x 7 matthaus staff 238 Dec 20 16:45 hardy drwxrwsr-x 7 matthaus staff 238 Dec 27 11:27 lucid drwxrwsr-x 7 matthaus staff 238 Dec 27 11:28 natty drwxrwsr-x 7 matthaus staff 238 Dec 27 11:30 oneiric drwxrwsr-x 7 matthaus staff 238 Dec 27 11:37 precise drwxrwsr-x 7 matthaus staff 238 Dec 27 11:38 quantal drwxrwsr-x 7 matthaus staff 238 Dec 27 11:33 sid drwxrwsr-x 7 matthaus staff 238 Dec 27 11:34 squeeze drwxrwsr-x 7 matthaus staff 238 Dec 27 11:39 stable drwxrwsr-x 7 matthaus staff 238 Dec 27 11:39 testing drwxrwsr-x 7 matthaus staff 238 Dec 27 11:31 unstable drwxrwsr-x 7 matthaus staff 238 Dec 27 11:36 wheezy Without --copy-links, the dists directory looks like what you have described. The requirement for the --copy-links option is why I submitted Issue #18055 [1]. I am maintaining a few other mirrors like CentOS and Fedora EPEL and using this flag on those mirrors would greatly increase the disk space and bandwidth required by my mirrors. Both these mirrors use symlinks like 5 - 5.8. Even the puppetlabs mirror uses symlinks in the yum/el directory! The result is that the puppetlabs yum mirror with --copy-links uses far more disk space and bandwidth than necessary when used as recommended. Initially I had mirrored puppetlabs using the same script that I used for my other rsync mirrors, but due to the --copy-links requirement I have had to split out just the puppetlabs apt mirror into a separate script. I am still using my regular rsync script without --copy-links to mirror the puppetlabs/yum portion to avoid wasting disk space and bandwidth. [1] https://projects.puppetlabs.com/issues/18055 -- Konrad Scherer, Sr. Engineer, Linux Products Group, Wind River direct 613-963-1342 fax 613-592-2283 -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: Unnecessary changes when creating facts.yaml
On 12/05/2012 10:22 AM, Andrew Beresford wrote:
I've ended up separating the template into an external file and then doing
something horrible;
out=scope.to_hash.reject { |k,v| k.to_s =~
/(uptime.*|timestamp|.*free|macaddress.*|ipaddress)/ }.to_yaml.split \n
header = out.shift
header = header+\n
sorted = out.sort
sorted_string = sorted.join \n
header sorted_string
This only works because the structure of the facts.yaml content is very simple,
and I'd much rather replace it with something more elegant.
I was originally trying to override the Hash to_yaml method to output the keys
in alphabetical order, but I ended up fighting with it too much and gave up as I
had other more important stuff to work on.
I had the same problem and tried an equally ugly hack which worked until the
yaml started containing perfectly legal yaml anchors and aliases (id001 and
*id001)[1]. My current solution is:
cron {
'generate_facts':
command = '/usr/bin/facter --puppet --yaml
/etc/mcollective/facter.yaml 2 /dev/null',
minute = '0';
}
This does miss the puppet variables in the current scope (move to custom fact?)
and potentially quickly changing facts (handled with facter_dot_d?). But this
works for my needs. If anyone has a better idea I would be very interested.
[1]: http://yaml.org/spec/1.2/spec.html#id2760395
--
Konrad Scherer, Sr. Engineer, Linux Products Group, Wind River
direct 613-963-1342 fax 613-592-2283
--
You received this message because you are subscribed to the Google Groups Puppet
Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
