Re: [Puppet Users] puppet 3.6.2, parse time error about storeconfigs
Thanks, I failed to find it in Jira. Glad to know it's something already in the roadmap. On Wed, Nov 5, 2014 at 5:10 PM, Henrik Lindberg henrik.lindb...@cloudsmith.com wrote: On 2014-06-11 1:30, Tristan Smith wrote: So, I'm trying to be a good citizen and use more forge code. I run puppet parser validate across all manifests as part of this - which results in much hilarity and tears. The latest one was when I pulled in puppetlabs/apache. .../modules/apache/manifests$ puppet parser validate balancer.pp --parser=future Warning: You cannot collect exported resources without storeconfigs being set; the collection will be ignored at .../modules/apache/manifests/balancer.pp:65:5 .../modules/apache/manifests$ puppet parser validate balancer.pp --parser=future --storeconfigs=true Error: Could not parse application options: needless argument: --storeconfigs=true In general, warnings have been about things that I should care about, and I enforce them, but this one's just plain improper. The warning is issues because the code contains collection of exported resources. It is a known issue that this should not be a parse time warning since the environment where parsing/validation takes place does not have to have storeconfigs set at all (the result is the same anyway, the difference is when running that code). The future parser has the same warning to be compatible. We will fix this during the 4x series if it does not make it in time for Puppet 4.0 If you really want to not see the error, you have to configure storeconfigs: http://projects.puppetlabs.com/projects/1/wiki/using_ stored_configuration - which as you can see in that information is deprecated if you want to use Rails and ActiveRecord, in favor of PuppetDB. This means you have to have a PuppetDb set up when you are validating... (and the lines of yaks requiring a haircut keeps piling up). Am I off base, or should this be a bug report against puppet's parser? (btw: future or not future, same result.) No need, it is a known issue (if you are referring to the pointless warning about storeconfigs when there is a collection of exported resources in the code). Regards - henrik -- Visit my Blog Puppet on the Edge http://puppet-on-the-edge.blogspot.se/ -- You received this message because you are subscribed to a topic in the Google Groups Puppet Users group. To unsubscribe from this topic, visit https://groups.google.com/d/ topic/puppet-users/EKlG7O_S6PE/unsubscribe. To unsubscribe from this group and all its topics, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/ msgid/puppet-users/m3ehqq%24hev%241%40ger.gmane.org. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAKdO7cd9qVSdAYmBcuYzdiVTiZsrP%3DaPG2ovxH8hXdxPe%3DST4A%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] puppet 3.6.2, parse time error about storeconfigs
So, I'm trying to be a good citizen and use more forge code. I run puppet parser validate across all manifests as part of this - which results in much hilarity and tears. The latest one was when I pulled in puppetlabs/apache. .../modules/apache/manifests$ puppet parser validate balancer.pp --parser=future Warning: You cannot collect exported resources without storeconfigs being set; the collection will be ignored at .../modules/apache/manifests/balancer.pp:65:5 .../modules/apache/manifests$ puppet parser validate balancer.pp --parser=future --storeconfigs=true Error: Could not parse application options: needless argument: --storeconfigs=true In general, warnings have been about things that I should care about, and I enforce them, but this one's just plain improper. Am I off base, or should this be a bug report against puppet's parser? (btw: future or not future, same result.) --Triss -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/62a3eeec-925a-4d6b-85fe-8b4a5ca83332%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Puppet 3.7.1 parse problems with resource overrides.
So, new behavior that I'm not sure is intentional. File['blah'] works fine, File ['blah'] is now illegal - the additional whitespace causes the below stupendously readable error on puppet parser validate. Error: Could not parse for environment production: Expression is not valid as a resource, resource-default, or resource-override at (slice () 'file'):(slice () 'line'):(slice ([] (cat '' (str $myvar) '')) 'pos') Is this a known bug I've failed to find on the ticket system, or intended behavior I've just missed in the notes? If not, I'll go chase it as a bug. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/8c2ca98f-55a2-4be1-9504-e24607f48656%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: Puppet 3.7.1 parse problems with resource overrides.
One more, mostly just a 'wouldn't it be nice if this were clearly handled?'
Having a comma at the end of an assignment like the below appears to cause
confusion; mostly at this point my gripe is that the error line is reported
as the enclosing class/define/whatever instead of at the line with the
errant comma.
puppet parser validate --parser=future init.pp
Error: Could not parse for environment production: This Array Expression is
not productive. A Host Class Definition can not end with a non productive
construct at init.pp:1:1
class a {
$foo = bar,
$banana = apple
}
On Monday, September 22, 2014 11:48:20 AM UTC-7, Tristan Smith wrote:
So, new behavior that I'm not sure is intentional. File['blah'] works
fine, File ['blah'] is now illegal - the additional whitespace causes the
below stupendously readable error on puppet parser validate.
Error: Could not parse for environment production: Expression is not valid
as a resource, resource-default, or resource-override at (slice ()
'file'):(slice () 'line'):(slice ([] (cat '' (str $myvar) '')) 'pos')
Is this a known bug I've failed to find on the ticket system, or intended
behavior I've just missed in the notes? If not, I'll go chase it as a bug.
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/27b39605-a929-4271-8a4b-b693337b7207%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: Each loop issue
Hayo.
Other than s/notice/notify/ for best effect, that works a treat when I
invoke it with puppet apply --parser future test.pp (added an 'include
mytestclass' for invocation, but that's it).
The parser=future is required on whatever node is doing the catalog
compilation, and needs to go in [main] for best effect, I believe
(certainly won't do anything in [agent], might be sufficient in [master])
On Monday, June 30, 2014 2:58:19 PM UTC-7, martin...@gmail.com wrote:
Hi Puppet users!
I'm having an issue with each loop on puppet, and I just can't figure
out what am I doing wrong, so any help will be much appreciated.
Here is my test.pp example:
class mytestclass {
$stuff = [1, 2, 3]
each($stuff) |$x| {
notice { $x: }
}
}
And here is the output when trying to apply it: *Error: Could not parse
for environment production: Could not match |$x| at /tmp/test.pp:10*
I've parser = future inside my puppet.conf, ruby-rgen is installed in
both client and server. I'm also running it with --parser future on the
client, but no joy.
I've been playing with variants like $stuff.each, using hashes, and a
few more desperate ideas.
OS: Ubuntu 12.04
Puppet: 3.5.1-1puppetlabs1
ruby-rgen: 0.6.5-1puppetlabs1
I'm clearly missing something, but can't find what.
Any suggestion is welcomed!
Cheers,
Martin
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/a365e094-1d17-42c0-88c5-e135f24bd191%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] Re: Puppet 3.6.0... and scaling?
A followup for closure; I've now turned environment_timeout to 'unlimited' and am doing a graceful restart of httpd to reset passenger. This seems to work. The resultant performance is _much_ more sane; I'm still seeing mebbe 20% increase in server CPU requirements, but that's quite manageable. Overall runtime is palpably better (about 10%); haven't identified where I'm getting my savings, though since I'm in parser=future land that may be a fair chunk of it. Upshot: Oh, god, 5s default timeout is nightmarishly bad for large systems, made it look like Puppet suddenly didn't know how to scale. The reality is that this has minimal real effect once you tune and I'm rocking 3.6 in the wild. On Sun, May 25, 2014 at 2:53 PM, Tristan Smith tr...@dreamlibrarian.com wrote: I've been using the default (which if I read correctly is 5s?); seems likely that a good answer here for me would be to push it out to $something_long and use restart.txt or somesuch to uptake changes. I may try that this week. My MaxRequests was already 10k. I'm guessing that 5 seconds of cache was approximately equivalent to jack and squat. I'm not sure if my attempt to tune by granting more workers would've done me any good. On Fri, May 23, 2014 at 5:25 PM, Chuck cssc...@gmail.com wrote: I had it set to 3 minutes for this test. I am going to work on trying some different settings. So far the best result has been with the following settings PassengerMaxRequests 5000 environment_timeout 15 minutes service httpd graceful whenever my svn update script detects a change. I will be working on getting more information. On Friday, May 23, 2014 3:24:43 PM UTC-5, Josh Partlow wrote: Hi Tristan, Chuck, What environment_timeout do you have set currently? On Friday, May 23, 2014 12:20:33 PM UTC-7, Tristan Smith wrote: Whuf. I'm already at 1, so that's not it for me. Seems like the doubled run time was just murdering me in the face. Trying to run 1000 clients in 30m was just too much. Guess I could make it an hour splay, but the overall resource requirement change is sufficiently large I'm probably just going to see about opting out of directory environments for a couple revisions, wait for them to tune up a bit. On Fri, May 23, 2014 at 10:23 AM, Chuck css...@gmail.com wrote: I am noticing increase CPU and Memory requirements from Puppet 3.4.3 I had to set the following passenger config so that my server would not run out of memory: PassengerMaxRequets 1 CPU idle 3.4.3 = 70% - 80% Memory utilization = 52% CPU idle 3.6.1 = 60% - 73% Memory utilization = 85% -- You received this message because you are subscribed to a topic in the Google Groups Puppet Users group. To unsubscribe from this topic, visit https://groups.google.com/d/ topic/puppet-users/arLckU-mPhw/unsubscribe. To unsubscribe from this group and all its topics, send an email to puppet-users...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/ msgid/puppet-users/2c9172e1-6129-46d1-94f1-38c078d72226% 40googlegroups.com https://groups.google.com/d/msgid/puppet-users/2c9172e1-6129-46d1-94f1-38c078d72226%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to a topic in the Google Groups Puppet Users group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/arLckU-mPhw/unsubscribe. To unsubscribe from this group and all its topics, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/490ad0ec-a35a-4ab3-a0a9-88e52e752083%40googlegroups.com https://groups.google.com/d/msgid/puppet-users/490ad0ec-a35a-4ab3-a0a9-88e52e752083%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAKdO7ceOBXU8wQ4gcY%2BLrWD_FrK4SyVTvb9K%2BU73P1MGAnKSQg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] Re: Puppet 3.6.0... and scaling?
I've plenty of possible mechanisms to experiment with for lessening the load, but I honestly don't _have_ to now that I've fixed the caching problem. 20% higher load == the system peaks out at 40%idle, now. I've got some headroom. With the 5s cache, it was bringing the server to its knees. That's no longer an issue. On Sat, May 31, 2014 at 11:41 AM, Brian Wilkins bwilk...@gmail.com wrote: What about staggering your runs? It seems trivial but at least it would reduce your load I think. -- You received this message because you are subscribed to a topic in the Google Groups Puppet Users group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/arLckU-mPhw/unsubscribe. To unsubscribe from this group and all its topics, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/97a9d7b0-45a0-468c-b981-34ac9221aa40%40googlegroups.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAKdO7ceQzCd0bBiG86ztT7Zspqq-GEXCv%2B_9kD9pNreYd6MoYA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] Re: Puppet 3.6.0... and scaling?
I've been using the default (which if I read correctly is 5s?); seems likely that a good answer here for me would be to push it out to $something_long and use restart.txt or somesuch to uptake changes. I may try that this week. My MaxRequests was already 10k. I'm guessing that 5 seconds of cache was approximately equivalent to jack and squat. I'm not sure if my attempt to tune by granting more workers would've done me any good. On Fri, May 23, 2014 at 5:25 PM, Chuck cssc...@gmail.com wrote: I had it set to 3 minutes for this test. I am going to work on trying some different settings. So far the best result has been with the following settings PassengerMaxRequests 5000 environment_timeout 15 minutes service httpd graceful whenever my svn update script detects a change. I will be working on getting more information. On Friday, May 23, 2014 3:24:43 PM UTC-5, Josh Partlow wrote: Hi Tristan, Chuck, What environment_timeout do you have set currently? On Friday, May 23, 2014 12:20:33 PM UTC-7, Tristan Smith wrote: Whuf. I'm already at 1, so that's not it for me. Seems like the doubled run time was just murdering me in the face. Trying to run 1000 clients in 30m was just too much. Guess I could make it an hour splay, but the overall resource requirement change is sufficiently large I'm probably just going to see about opting out of directory environments for a couple revisions, wait for them to tune up a bit. On Fri, May 23, 2014 at 10:23 AM, Chuck css...@gmail.com wrote: I am noticing increase CPU and Memory requirements from Puppet 3.4.3 I had to set the following passenger config so that my server would not run out of memory: PassengerMaxRequets 1 CPU idle 3.4.3 = 70% - 80% Memory utilization = 52% CPU idle 3.6.1 = 60% - 73% Memory utilization = 85% -- You received this message because you are subscribed to a topic in the Google Groups Puppet Users group. To unsubscribe from this topic, visit https://groups.google.com/d/ topic/puppet-users/arLckU-mPhw/unsubscribe. To unsubscribe from this group and all its topics, send an email to puppet-users...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/ msgid/puppet-users/2c9172e1-6129-46d1-94f1-38c078d72226% 40googlegroups.comhttps://groups.google.com/d/msgid/puppet-users/2c9172e1-6129-46d1-94f1-38c078d72226%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to a topic in the Google Groups Puppet Users group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/arLckU-mPhw/unsubscribe. To unsubscribe from this group and all its topics, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/490ad0ec-a35a-4ab3-a0a9-88e52e752083%40googlegroups.comhttps://groups.google.com/d/msgid/puppet-users/490ad0ec-a35a-4ab3-a0a9-88e52e752083%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAKdO7cc%2BYe1-LSSk5_DBrMdD5YaqCdcRwuGKL6VqJ3AwGNE4Zg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] Re: Puppet 3.6.0... and scaling?
Whuf. I'm already at 1, so that's not it for me. Seems like the doubled run time was just murdering me in the face. Trying to run 1000 clients in 30m was just too much. Guess I could make it an hour splay, but the overall resource requirement change is sufficiently large I'm probably just going to see about opting out of directory environments for a couple revisions, wait for them to tune up a bit. On Fri, May 23, 2014 at 10:23 AM, Chuck cssc...@gmail.com wrote: I am noticing increase CPU and Memory requirements from Puppet 3.4.3 I had to set the following passenger config so that my server would not run out of memory: PassengerMaxRequets 1 CPU idle 3.4.3 = 70% - 80% Memory utilization = 52% CPU idle 3.6.1 = 60% - 73% Memory utilization = 85% -- You received this message because you are subscribed to a topic in the Google Groups Puppet Users group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/arLckU-mPhw/unsubscribe. To unsubscribe from this group and all its topics, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/2c9172e1-6129-46d1-94f1-38c078d72226%40googlegroups.comhttps://groups.google.com/d/msgid/puppet-users/2c9172e1-6129-46d1-94f1-38c078d72226%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAKdO7cd9NwwnTL2cg6NwKL-zrsGi9Cs4-fdv0LzH3HZuQYch5A%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Puppet 3.6.0... and scaling?
After much hacking to get directory environments settled and the manifest directory in place, I rolled Puppet 3.6 to our puppetmasters last night. One of our puppetmasters has nearabouts 1000 clients, runs passenger under apache 2.2 (ruby 1.8.7, sadly, thanks CentOS), and normally doesn't really notice puppet running - basically peaks out at 20% CPU usage. Under 3.6, even doubling the passenger worker count, it couldn't keep up with the load - I started running out of apache procs due to workers stuck in waiting mode and they were all just hanging waiting for a passenger worker to free up. CPU usage on the system capped out. Strace -c on the passenger workers had them spending 30% of their time in clone() and 60% in wait4(), fwiw. I'm going to be digging to figure out what in hell changed to cause this, but has anyone else experienced a significant change in performance under 3.6? --Triss -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/035a95e0-243e-46f2-9092-fe547a0580f9%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] 3.6, file resources, and warnings
So I'm testing out puppet 3.6 with our existing environment and in general
i'm pleased.
There's one oddity that I'm tripping on and I'm not sure how to address it.
We have a number of places where we leverage the 'file' resource with an
array of sources - using fileservers to pick the first match of a file for
a host, for an operating system, or a fallthrough file, like, say:
file { $sudoersd/005_$hostname:
require = File[$sudoersd],
ensure = file,
owner = root,
group = 0,
mode = 0440,
source = [puppet:///modules/sudo/etc/sudoers.hosts/$hostname,
puppet:///modules/sudo/etc/sudoers.hosts/no_unique_entries]
}
In previous versions, this has worked fine - and it still _works_ today.
However, I'm seeing warning messages regarding the hostname entry, the one
that fails. That seems excessive when the resource doesn't fail or even do
something improper.
My question then: Should I
a) consider another idiom for these needs?
Or
b) take advantage of the new log_level directive (thank you, by the way)
and set log_level=err and not worry about warnings except when debugging
failures?
--Triss
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/70070de5-ea62-4406-94c6-12632e3e0c7c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
