Re: [Puppet Users] Slides TDD with puppet - puppetcamp paris yesterday
On Tue, Apr 15, 2014 at 06:46:21AM -0700, Alexander Fortin wrote: On Tuesday, April 15, 2014 1:10:05 PM UTC+2, Felix.Frank wrote: On 04/15/2014 09:25 AM, Nikola Petrov wrote: I haven't used catalog-diff but I am suspecting that it might be a substitute for the rspec tests. Tell me No. rspec allows you to implement unit tests to catch certain regression bugs. catalog-diff is rather concerned with integration testing (although you can use rspec for that as well). I think with catalog-diff we're catching regression bugs too, meaning that if the module interface is broken in the proposed commit, it will show up as an error (unless no node is actually including the refactored module). Hmm I am suspecting that sadly we are talking about sort of different things here. In my company we have reusable puppet modules between different teams - those are build by experts and require major testing because we don't want to break things in incremental versions. The thing is that those modules are not applied as is but are mostly reused in different project specific modules(i.e. site modules) I can see catalog-diff valuable for our site modules because it is actually testing the end result - what will be applied to a given host. In most cases the reusable modules are used in different ways with different parameters - let's say a tomcat module that is responsible for deploying war files and what not but cannot be used standalone(you have to wrap it in some way) does it provide the following: * Automatic runs through guard/rake/whatever on every commit to verify that I am not breaking things like permissions on files You could do that, but compiling each of your nodes' catalog on every commit would be excessive. I don't see it as such a big problem. You can always put more resources on your CI server to make that possible which leads me to my next question - have you guys automated this in some way. I would be glad to catch regressions from catalog-diff on all my hosts but automatically - let's say through rspec or some other srcipt. Shameless plug: I have been doing a more mature approval port for java lately https://github.com/nikolavp/approval and although it is not ready, there is a good looking thing for ruby here https://github.com/kytrinyx/approvals. The idea is that you approve the output of master compile the first time the test is run and then(on further runs) if there are differences(detected through catalog-diff) you are prompted(let's say in your editor). This allows you to approve the differences if you know that they won't do any harm but in a more automatic way. I agree it can be overkill for many environments, I guess depends on what you actually want to test, or better, what kind of confidence you want to have that your committed code is not actually breaking anything. To me seems that the catalog-diff approach is catching a broad variety of problems, also some that (I think) can't be easily catch by expressions in rspec. For example, somewhere in a random manifest I'm just changing declaration style for a class, from 'include myclass' to class {'myclass': }, leading to possible declaration dependency order / duplicated declarations problems: Node A (which I am currently working on) is including only one declaration, but node B was evaluating that class declaration two times, hence the duplicate declaration only on node B. Can rspec-puppet catch also this kind of problems? More in general, I'd like to know if rspec-puppet can give me the confidence that, given a node resource, the new version of the catalog will compile for that node. Actually no - that's what we use staging for. You push your modules up there and see if they run properly on real machines. We also use the modules through vagrant on every developer machines - so they are constantly tested. Of course we want to start using beaker for those kind of tests so we don't push crap on the bear metal(although it is not metal most of the time) -- Alex -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/826f278e-1b69-4721-9e35-7028627c44fa%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/20140417070617.GD24790%40nikolavp-desktop. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] Slides TDD with puppet - puppetcamp paris yesterday
On 04/17/2014 09:06 AM, Nikola Petrov wrote: Of course we want to start using beaker for those kind of tests so we don't push crap on the bear metal(although it is not metal most of the time) ...but usually just bears. ;-D (Yeah I know. Sorry. It is just such a cute typo that I couldn't resist.) -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/534F86CA.2030408%40alumni.tu-berlin.de. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] Slides TDD with puppet - puppetcamp paris yesterday
:D. Too fast for da bears. On Thu, Apr 17, 2014 at 09:46:18AM +0200, Felix Frank wrote: On 04/17/2014 09:06 AM, Nikola Petrov wrote: Of course we want to start using beaker for those kind of tests so we don't push crap on the bear metal(although it is not metal most of the time) ...but usually just bears. ;-D (Yeah I know. Sorry. It is just such a cute typo that I couldn't resist.) -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/534F86CA.2030408%40alumni.tu-berlin.de. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/20140417075931.GE24790%40nikolavp-desktop. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] Re: Change to issue tracking for Forge PL Module projects
Is there any update on the timeline for Module issue migration to Jira? I opened https://github.com/puppetlabs/puppetlabs-mcollective/issues/124 at the beginning of March. This has now become more important to me, so I was going to look up my issue description and try to submit a PR (it should be a relatively trivial fix), but the link to the issue no longer exists, so I can't seem to find the details that I originally put there. If the migration isn't imminent, does anyone know how to find the content of the issue that I opened, which seems to have now fallen off the face of the earth? Thanks, Jason Antman On Fri, Mar 21, 2014 at 2:16 PM, Heidi Pio heidi@puppetlabs.com wrote: Hi Everyone, Quick introduction: I'm the Engineering Project Manager of the Puppet Labs Forge. And a quick update to let you know that the Puppet Labs Forge and Module issue migrations from Redmine and GitHub are now complete. GitHub issues for each module repo have been turned off. Puppet Labs Forge issues can now be found here: https://tickets.puppetlabs.com/browse/FORGE And Puppet Labs Module issues can be found here: https://tickets.puppetlabs.com/browse/MODULES The pull request process will not change, however, the Puppet Forge Community Pull Request Review meeting has moved to Thursdays at 10am PST. Ashley Penney has graciously offered to send out weekly updates for those meetings via the Puppet-Dev list. Please feel free to contact me if you have any questions about this migration or the pull request process. Thanks and have a great weekend! On Monday, December 16, 2013 7:48:34 AM UTC-8, Ryan Coleman wrote: As Eric Sorenson noted earlier [1], issue tracking for most projects at Puppet Labs are moving to JIRA. This includes the Puppet Forge and the Puppet Labs modules on the Forge. As of this morning, Forge issues in Redmine have been set to read-only and have been migrated into the FORGE project in tickets.puppetlabs.com. Each Redmine ticket will point you to its companion JIRA ticket. Here's an example http://projects.puppetlabs.com/issues/5033 - https://tickets.puppetlabs.com/browse/FORGE-27 Issue tracking for Puppet Labs Forge modules are also moving. Most are moving from GitHub Issues but some were still being tracked in Redmine. This morning, we will update the 'Report Issues' link on each module page to point to JIRA. Please start filing new issues here: https://tickets.puppetlabs.com/browse/MODULES. We're still sorting out the software to migrate existing issues without losing critical information. Once that's ready, it'll work much like the Forge migration. I'll be spending some of my holiday curled up with hot chocolate, caring for each of the newly migrated issues. I'll update their states, ensure they're properly linked to internal work and try to give you some idea where they fit into 2014. If you have an issue you care deeply about, I suggest you follow it into JIRA and start watching it. I hope you find this transition fairly painless. Please let me know if you have any questions, concerns or suggestions. [1] https://groups.google.com/d/topic/puppet-users/4lV1cT6Li-M/discussion -- Ryan Coleman | Modules Forge | ryanycoleman on twitter #puppet IRC -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/969b4116-4558-4766-9d91-772a27d054c6%40googlegroups.comhttps://groups.google.com/d/msgid/puppet-users/969b4116-4558-4766-9d91-772a27d054c6%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAFt4V4kO%2BfXogw3D%2B4wLjY4JD47v3%2BuQWRUQfpoGK%3DjEbpM2Rw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: Hiera arry into ERB template
Thanks guys for the explanation; worked just fine. Is there any other good hiera documentation apart from the one in Puppetlab doc? -San On Wednesday, April 16, 2014 6:59:27 PM UTC+1, jcbollinger wrote: On Wednesday, April 16, 2014 11:31:53 AM UTC-5, Sans wrote: Hi there, Trying to figure out how to use hiera-array in my template. So, this is what I have in .yaml file: No, you are just trying to use an array. Your template can't tell where the array came from or how it was constructed. my_coll_list: - mon502.local - mon522.local and in my nodes.pp, I have this: $my_colloctors = hiera_array('my_coll_list') You probably do not want to use hiera_array() for that. Use plain hiera() instead. The former traverses your entire hierarchy, returning an array whose elements are the values matching the specified key at each level. The latter returns the value associated with the requested key at the highest-priority level of your hierarchy that provides one, as whatever data type is modeled by the data (an array, in your example). then in one of my ERB templates, I using that like this: COLLECTORS = %= @my_colloctors % if i understood correctly, according the Puppet documentation, using *array merge lookup,* I was expecting a result like this: COLLECTORS = ['mon502.local', 'mon522.local'] but what I'm getting is: COLLECTORS = mon502.localmon522.local 1. You are confusing the Puppet DSL representation of an array with its (Ruby) stringification. The latter is what your template will produce, and it is simply the concatenation of the string values of the array elements. 2. The Puppet DSL form of the value of $my_colloctors that you actually have is in all likelihood [['mon502.local', 'mon522.local']]. That is, an array whose only element is an array of your colloctors (whatever those are). See my previous comments about hiera_array() vs. hiera(). What am I missing here? If you really do want to collect collectors from multiple levels of your hierarchy, then you will need to flatten the result. Additionally, you will need to format it into your template yourself. There are many ways to do that, but one would be something like this: COLLECTORS = [%= @my_collectors.map { |c| '#{c.to_s}'}.join(', ') %] John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/eff94c7e-1651-4c2f-a850-b723ed7406ee%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Serious performance issues with Puppet 3.5.x when using Directory Environments
Dear Puppet Users community, I would like to report an issue I experience since the first release of Puppet 3.5, and possibly get some input from people who face the same problem (if any). I run a Puppet environment in which all 48 nodes run CentOS 6.5. I have been using config file-based environments so far without any trouble. When Puppet 3.5.0 was officially released I decided to give give it a try, along with the new Directory Environments feature. I disabled my old-style environment config while making sure my directory tree was correctly set, as described in the Directory Environments documentation pagehttp://docs.puppetlabs.com/puppet/latest/reference/environments.html . The problem shows up immediately after I restart httpd: Passenger starts spawning more and more processes which all consume 25% of my machine CPU, reaching a load average of 10+ in less than a minute as soon as the first node requests a catalog. This makes the master unusable, it takes between 200 and 300sec to compile any catalog. What I have tried: 1. Switching back from Passenger to WEBrick: same performance issue 2. Disabling the puppet service on all nodes, and triggering *one*single manual Puppet run on a random node: same performance issue 3. Disabling my external node classifier (Foreman): same performance issue 4. Upgrading to Puppet 3.5.1: same performance issue 5. Switching back to config-based environments and re-enabling all nodes: no issue, my master behaves normally and the load-average remains under 0.25 constantly As you might have noticed, this is definitely not a sizing issue. My master has 2GB of RAM and 2 allocated vCPU, but the Puppet master rack process starts consuming all resources as soon as I re-enable Directory Environments. When I want to use Directory Environments, the only thing I do is commenting the [development] and [production] sections in my puppet.conf file, and adding this to the [main] section: environmentpath = $confdir/environments Let me describe my configuration: *-puppet.conf--* [main] logdir = /var/log/puppet. rundir = /var/run/puppet ssldir = $vardir/ssl privatekeydir = $ssldir/private_keys { group = service } hostprivkey = $privatekeydir/$certname.pem { mode = 640 } autosign = $confdir/autosign.conf { mode = 664 } [master] autosign = $confdir/autosign.conf { mode = 664 } reports= foreman external_nodes = /etc/puppet/node.rb node_terminus = exec ca = true ssldir = /var/lib/puppet/ssl storeconfigs = true storeconfigs_backend = puppetdb [development] modulepath = /etc/puppet/environments/development/modules:/usr/share/puppet/modules manifest = /etc/puppet/environments/development/manifests/site.pp config_version = [production] modulepath = /etc/puppet/environments/production/modules:/usr/share/puppet/modules manifest = /etc/puppet/environments/production/manifests/site.pp config_version = *--* *--directory tree--*/etc/puppet/environments/ |-- development | |-- manifests | | `-- site.pp | `-- modules | |-- activemq | |-- apache | |-- apt | |-- concat | |-- datacat | |-- environment | |-- epel | |-- erlang | |-- firewall | |-- java | |-- java_ks | |-- mcollective | |-- postfix | |-- profiles | |-- rabbitmq | |-- repos | |-- stdlib | `-- tomcat `-- production |-- manifests | `-- site.pp `-- modules |-- activemq |-- apache |-- apt |-- concat |-- datacat |-- environment |-- epel |-- erlang |-- firewall |-- inifile |-- java |-- java_ks |-- mcollective |-- mysql |-- ntp |-- postfix |-- postgresql |-- profiles |-- puppetdb |-- rabbitmq |-- repos |-- stdlib |-- tomcat `-- vsftpd *--* Can anybody reproduce this it on any platform? Thank you in advance for your help! Toni -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/8036c0fa-57ee-401b-b9d7-03c43dda0fd2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Nodes in Environments can't post reports
Hi, I am employing a puppetmaster version 3.x and I have not been able to receive reports from nodes in environments other than production. All other aspects of environments function to my satisfaction. All affected clients report: Error: Could not send report: Error 400 on SERVER: Could not intern from pson: undefined method `intern' for nil:NilClass The server only reports: [2014-04-17 14:16:36] 93.184.216.119 - - [17/Apr/2014:14:16:36 CEST] PUT /test/report/hostname.example.com HTTP/1.1 400 70 I have not found any post regarding this problem so I assume it is a local problem with my setup and not a bug (or maybe it's just the way of things?). I could not debug this further as there does not seem to be a way to enable more detailed logging in this respect. Is there anybody who could point out something I might have overlooked? I will post config files if necessary of course but I wanted to keep the first post concise. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/e0efa942-6c4d-4580-9e6a-4810d16fa73c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: Different modes for directory, contents?
On Wednesday, April 16, 2014 7:06:53 AM UTC-5, Bret Wortman wrote: Is there a simple way to enforce a different mode for a directory and its contents when the contents of the directory are highly variable? What I mean is that I've got a case where some developers want a directory /var/log/httpd to be protected 755 but the contents they want at 644. Is there a simple, Puppet-ish way to make this happen, or are we basically stuck with: file { '/var/log/httpd': ensure = directory, mode = '0644', recurse = true, } I'm missing something. That declaration ought to do exactly what you say you want (Puppet will add add search permission to directories wherever there is read permission; see http://docs.puppetlabs.com/references/3.4.stable/type.html#file-attribute-mode). Where do you imagine simplifying it or making it more puppet-ish? John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/90f5de75-6672-41a4-81b0-e95eab6354ab%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: file: square bracket in template
Le mercredi 16 avril 2014 19:42:40 UTC+2, jcbollinger a écrit : To the best of my knowledge, square brackets appearing in ERB template text have no special meaning to ERB. They are just copied verbatim to the output. I suspect that you have written something like file { '/etc/my.cnf'': source = template('my.cnf.erb') } where what you want is file { '/etc/my.cnf'': content = template('my.cnf.erb') } The distinction is quite important: the 'source' parameter is expected to specify a URL from which the agent get retrieve the desired file content, whereas the 'content' property gives the content itself. (The two are mutually exclusive.) You're right about the fact that I've used source instead of content ! Thanks for your help. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/a9affa58-42b4-43e5-bd45-fe50698f4ba6%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] Re: Different modes for directory, contents?
Doesn't this also make the directory itself 644 instead of 755? Maybe I need to play around with it a bit more. *Bret Wortman* http://about.me/wortmanbret On Thu, Apr 17, 2014 at 9:17 AM, jcbollinger john.bollin...@stjude.orgwrote: On Wednesday, April 16, 2014 7:06:53 AM UTC-5, Bret Wortman wrote: Is there a simple way to enforce a different mode for a directory and its contents when the contents of the directory are highly variable? What I mean is that I've got a case where some developers want a directory /var/log/httpd to be protected 755 but the contents they want at 644. Is there a simple, Puppet-ish way to make this happen, or are we basically stuck with: file { '/var/log/httpd': ensure = directory, mode = '0644', recurse = true, } I'm missing something. That declaration ought to do exactly what you say you want (Puppet will add add search permission to directories wherever there is read permission; see http://docs.puppetlabs.com/references/3.4.stable/type.html#file-attribute-mode). Where do you imagine simplifying it or making it more puppet-ish? John -- You received this message because you are subscribed to a topic in the Google Groups Puppet Users group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/YvyV72vReyc/unsubscribe. To unsubscribe from this group and all its topics, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/90f5de75-6672-41a4-81b0-e95eab6354ab%40googlegroups.comhttps://groups.google.com/d/msgid/puppet-users/90f5de75-6672-41a4-81b0-e95eab6354ab%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAN9oxgR7-%2BqtcOCrwRNOu%2B6AbDUHJ78rMMb9Ovqzw_A1aXU6bQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: Referencing a value from one level to another.
John, Thanks so much; this worked as expected. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/763eff2b-2dd4-4845-aa34-437b5c687714%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] API
Hello Do you know why I get 404 erro on GET? curl -u user:password -k -H 'Accept: s' https://puppet.mydomain.com.br/certificate/ca auth.config file contains path / auth any allow * Thanks -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/851acad4-f1f0-4632-b385-9cdbb2631322%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] How to use facter variable in node definition?
Hi, I want to use facter scope variable to use in my manifest files for puppet for now i have something like this: node 'web-server' { .. .. } I want to be able to use if/case definition to define nodes root@ip-10-187-40-216:~# facter -p system_role app-server root@ip-10-187-40-216:~# so in my manifest node definition i would like to use something like: node if $::system_role == app-server { .. ... include default; } or something similar to it. I want to use facter variables because i want to use with ec2tags for each server and build each server based on ec2 tag How I can approach this? thanks -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/3f7a87c2-cdec-42b8-88d5-25a57267ee46%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] How to use facter variable in node definition?
Remove the node word and you have it if $::system_role == app-server { .. ... include default; } Regards, El 17/04/2014 20:46, Pawel Kilian pawelkil...@gmail.com escribió: Hi, I want to use facter scope variable to use in my manifest files for puppet for now i have something like this: node 'web-server' { .. .. } I want to be able to use if/case definition to define nodes root@ip-10-187-40-216:~# facter -p system_role app-server root@ip-10-187-40-216:~# so in my manifest node definition i would like to use something like: node if $::system_role == app-server { .. ... include default; } or something similar to it. I want to use facter variables because i want to use with ec2tags for each server and build each server based on ec2 tag How I can approach this? thanks -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/3f7a87c2-cdec-42b8-88d5-25a57267ee46%40googlegroups.comhttps://groups.google.com/d/msgid/puppet-users/3f7a87c2-cdec-42b8-88d5-25a57267ee46%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAF_B3dduy%2B9M5pg4sTCNqC%3DW0yoFX0JxyBbAdR72e3QuMuDXQg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: Serious performance issues with Puppet 3.5.x when using Directory Environments
Hi Toni, this is an interesting report -- can you please run a client against the server with profiling enabled and post the output from the master? Instructions are here: http://docs.puppetlabs.com/pe/latest/trouble_puppet.html#improving-profiling-and-debugging-of-slow-catalog-compilations On Thursday, April 17, 2014 2:23:24 AM UTC-7, Antoine Cotten wrote: Dear Puppet Users community, I would like to report an issue I experience since the first release of Puppet 3.5, and possibly get some input from people who face the same problem (if any). I run a Puppet environment in which all 48 nodes run CentOS 6.5. I have been using config file-based environments so far without any trouble. When Puppet 3.5.0 was officially released I decided to give give it a try, along with the new Directory Environments feature. I disabled my old-style environment config while making sure my directory tree was correctly set, as described in the Directory Environments documentation pagehttp://docs.puppetlabs.com/puppet/latest/reference/environments.html . The problem shows up immediately after I restart httpd: Passenger starts spawning more and more processes which all consume 25% of my machine CPU, reaching a load average of 10+ in less than a minute as soon as the first node requests a catalog. This makes the master unusable, it takes between 200 and 300sec to compile any catalog. What I have tried: 1. Switching back from Passenger to WEBrick: same performance issue 2. Disabling the puppet service on all nodes, and triggering *one*single manual Puppet run on a random node: same performance issue 3. Disabling my external node classifier (Foreman): same performance issue 4. Upgrading to Puppet 3.5.1: same performance issue 5. Switching back to config-based environments and re-enabling all nodes: no issue, my master behaves normally and the load-average remains under 0.25 constantly As you might have noticed, this is definitely not a sizing issue. My master has 2GB of RAM and 2 allocated vCPU, but the Puppet master rack process starts consuming all resources as soon as I re-enable Directory Environments. When I want to use Directory Environments, the only thing I do is commenting the [development] and [production] sections in my puppet.conf file, and adding this to the [main] section: environmentpath = $confdir/environments Let me describe my configuration: *-puppet.conf--* [main] logdir = /var/log/puppet. rundir = /var/run/puppet ssldir = $vardir/ssl privatekeydir = $ssldir/private_keys { group = service } hostprivkey = $privatekeydir/$certname.pem { mode = 640 } autosign = $confdir/autosign.conf { mode = 664 } [master] autosign = $confdir/autosign.conf { mode = 664 } reports= foreman external_nodes = /etc/puppet/node.rb node_terminus = exec ca = true ssldir = /var/lib/puppet/ssl storeconfigs = true storeconfigs_backend = puppetdb [development] modulepath = /etc/puppet/environments/development/modules:/usr/share/puppet/modules manifest = /etc/puppet/environments/development/manifests/site.pp config_version = [production] modulepath = /etc/puppet/environments/production/modules:/usr/share/puppet/modules manifest = /etc/puppet/environments/production/manifests/site.pp config_version = *--* *--directory tree--*/etc/puppet/environments/ |-- development | |-- manifests | | `-- site.pp | `-- modules | |-- activemq | |-- apache | |-- apt | |-- concat | |-- datacat | |-- environment | |-- epel | |-- erlang | |-- firewall | |-- java | |-- java_ks | |-- mcollective | |-- postfix | |-- profiles | |-- rabbitmq | |-- repos | |-- stdlib | `-- tomcat `-- production |-- manifests | `-- site.pp `-- modules |-- activemq |-- apache |-- apt |-- concat |-- datacat |-- environment |-- epel |-- erlang |-- firewall |-- inifile |-- java |-- java_ks |-- mcollective |-- mysql |-- ntp |-- postfix |-- postgresql |-- profiles |-- puppetdb |-- rabbitmq |-- repos |-- stdlib |-- tomcat `-- vsftpd *--* Can anybody reproduce this it on any platform? Thank you in advance for your help! Toni -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to
Re: [Puppet Users] Setting validproperties fields in provider modules
Hi, On 04/16/2014 06:10 PM, Jim Perry wrote: Thanks. I am still learning Ruby so I will look into accessors in more detail. Sure, but note that those aren't the important point. Sticking to @resource should work just as well. What I am trying to do is call the HPUX provider and use it to set the min and max age that gets passed back to the parent. Then I also need to be able to verify the right commands are used to reset the expiration when the user password is updated if it is a trusted setup. Hmm, that sounds as though there was some kind of basic problem in your approach. The agent does not explicitly send messages to single providers. Instead, ou would expect the agent to select the HPUX provider on its own and see to it that the necessary overrides are in place. I realize that I probably read like the metaphorical broken record, but if you could share at least some pseudo code that depicts what you are currently trying, we could discuss in a much more effective fashion. We're about two layers too meta :) Thanks, Felix -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/53503BA9.1060600%40Alumni.TU-Berlin.de. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] Same methods in two providers - how to optimize it?
Hi, you received an answer from Nan on the dev list, and that was pretty much spot on I think (yay Nan:) I would only add that for the types, you may want to take a look at how the ensure property and the ensurable method are implemented. If it's worthwile, you could try and duplicate that for your own shared properties. Although off the top of my head, I'm not sure how one would lodge the implementation in between your types and the Type base class. Be sure to share your code if you manage to whip something up :) HTH, Felix On 04/16/2014 07:15 PM, Jakov Sosic wrote: Hi, I'm developing some of my custom types, and some of them share same params, with exactly the same methods... For example, I have these params in three of my types and they are same: lib/puppet/type/mytype1.rb lib/puppet/type/mytype2.rb lib/puppet/type/mytype3.rb newproperty(:comment) do defaultto '' end Is there a way I can maybe extract that code into some external library and reuse it in all of my three types? Now, this definition is not a problem, but when you start to include various checks, and override methods like insync? and your param code grows to 30-40 lines, and you use it in multiple types, it gets usefull to extract that code to some external class and include it into these types. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/53503D79.3030502%40Alumni.TU-Berlin.de. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Announcing our new quest based Learning VM
Hi, We just rolled out the new and improved Learning VM — now with fun quests to learn Puppet at your own pace. If you've been meaning to explore the technology in depth, now you can do so with a series of detailed, multi-step quests. The Learning VM covers topics such as: accessing the Puppet Enterprise console, writing and using Puppet modules, and delving into Puppet resources, manifests, facts and more. Check out the new Learning VM: http://puppetlabs.com/download-learning-vm Happy learning. - Ben -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/c43a3c51-cdd0-4ea6-96c7-e38e35352d4d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Type 0 is not supported error running puppet resource package with 3.4.3 agent on Server 2003
After right at a hundred successful agent installs on a very heterogeneous mix of Server 2003 and Server 2008, I've encountered one machine that doesn't want to cooperate. The problem agent is the open source Puppet 3.4.3 MSI running on Server 2003. I run puppet resource package and I get Error: Could not run: Type 0 is not supported. I add --debug --trace --verbose and the first entry after the error is C:/Program Files/Puppet Labs/Puppet/sys/ruby/lib/ruby/1.9.1/win32/registry.rb:605:in 'read'. Invoking puppet resource service works just fine. This is the same MSI I've used on the hundred other machines. I ran the md5 sum just to make sure it wasn't corrupted in transfer. I uninstalled and reinstalled. Same behavior. (The agent is configured to communicate with a master, and that's where I first encountered the problem. I thought maybe reproducing the problem this way would take master interactions and catalog issues out of the equation.) The machine itself is remote and I'm limited on what I can do to it. Anything else I ought to try? -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/5228eb33-f500-4498-8c98-af1865723790%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] Same methods in two providers - how to optimize it?
On 04/17/2014 10:45 PM, Felix Frank wrote: Hi, you received an answer from Nan on the dev list, and that was pretty much spot on I think (yay Nan:) I would only add that for the types, you may want to take a look at how the ensure property and the ensurable method are implemented. If it's worthwile, you could try and duplicate that for your own shared properties. Although off the top of my head, I'm not sure how one would lodge the implementation in between your types and the Type base class. Be sure to share your code if you manage to whip something up :) Another question regarding custom types: is it possible to use a value of another param in 'defaultto'? For example, I have resource like this: mycustomtype { 'web.example.com': ensure = present, hostname = 'web.example.com', } I would like to be able to set 'defaultto' to the title of the resource, and still allow user to change it if he wishes to. I have tried something along these lines: newproperty(:hostname) do desc 'The hostname of the system, can be equal to name' defaultto @resource[:name] end but it throws: undefined method `[]' for nil:NilClass Is this possible at all? -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/535075F8.1080706%40gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] Failed running the puppet tests: bundle exec rake spec
On Tue, Apr 15, 2014 at 9:20 AM, David Portabella david.portabe...@gmail.com wrote: I've added more memory to the virtual machine, and now it works. 19753 examples, 0 failures, 86 pending (although I still get one Cannot allocate memory error) If you run 'bundle exec rake parallel:spec[1]' instead of 'bundle exec rake spec' it should use less memory. That number in square brackets is the number of processes to run in parallel, so if you've given your vm multiple processors, bump that number and specs should run faster as well. Kylo -- Kylo Ginsberg k...@puppetlabs.com *Join us at PuppetConf 2014**, September 22-24 in San Francisco - http://puppetconf.com http://puppetconf.com/* *Register by May 30th to take advantage of the Early Adopter discount http://links.puppetlabs.com/puppetconf-early-adopter **—**save $349!* -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CALsUZFHjy4ftTeS1mwZcNQMCV_vA9Kq2qmR_eM2Z_mfc4mrXvQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] API
On Thu, Apr 17, 2014 at 6:30 AM, Marcio Ordoñez marcio.ordo...@gmail.comwrote: Do you know why I get 404 erro on GET? curl -u user:password -k -H 'Accept: s' https://puppet.mydomain.com.br/certificate/ca By default the puppetmaster listens on port 8140. So you might try: curl -k -H 'Accept: s' https://puppet.mydomain.com.br:8140/certificate/cahttps://puppet.mydomain.com.br/certificate/ca HTH Kylo -- Kylo Ginsberg k...@puppetlabs.com *Join us at PuppetConf 2014**, September 23-24 in San Francisco - http://puppetconf.com http://puppetconf.com/* *Register by May 30th to take advantage of the Early Adopter discount http://links.puppetlabs.com/puppetconf-early-adopter **—**save $349!* -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CALsUZFEeBCvRAzAbB-KsiK-zsHPSwci8b9JjXOah0FrQCYMTog%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.