[Puppet Users] Re: puppet dashboard group and class

[Nicolas Aizier]

Hi Christian,

Thanks a lot buddy, that helped me a lot. I was able today to change
all my puppet configuration with the external_node feature using
dashboard.
This is really a good thing but definitely need to be documented so
people can use it. I'm considering writing such a doc for the wiki to
help the project.
The more people use puppet the more puppet will be loved, this project
really deserve it 

Thx again !

On Dec 15, 5:42 pm, Christian 
wrote:
> Hi Nicolas,
>
> i had exactly the same problem when i start using puppet dashboard
> some months ago. There is no documentation about it on the page and i
> would say that the names 'Classes' and 'Groups' are used unlucky. It
> turns out that classes acutally are the puppet modules (why the hell
> they dont use the same expression in puppet dashboard like in the rest
> puppet) and groups can be used to model a kind of inheritance. If you
> use classes and groups you have to use puppet dashboard as an external
> node classifier means you are not using node.pp anymore. The modeling
> which module belongs to which node will be done then in puppet
> dashboard. Your class name have of course be exactly the same name as
> the module name you have in your folder structure in the puppet module
> folder.
>
> Hope that helps a bit
>
> Christian
>
> On 15 Dez., 02:17, Nicolas Aizier 
> wrote:
>
>
>
>
>
>
>
> > Hi everyone,
>
> > I'm actually kind of new in puppet but I'm doing good in progressing.
> > I have read lot of docs and how to to understand the whole behaviour
> > of puppet.
> > Installed a puppet master server, deployed 15 clients to test it on
> > some of our testing servers.
> > Written modules to get exactly what we want, and then installed puppet
> > dashboard which work really fine.
>
> > My question might sound a bit noobish but I'm stuck on that point and
> > I really don't like to don't understand every part of a tool.
> > What is the use of adding "groups" and "class" in the dashboard ?
> > It seems that the dashboard don't see your class from the modules (saw
> > that it will be corrected soon), but what is the point to creating
> > class in the GUI ? Can you do anything with that ? and if yes will
> > that not spread the config through dashboard + puppet config files,
> > it'll be messy to maintain 
> > Same question with the groups, I understand that you can assign class
> > to groups so it's easyer to add a server in a global behavior but it
> > only uses class from dashboard . And is there a way to create such
> > group in puppet config files (if yes I didn't manage to find it ...) ?
>
> > Thx a lot for your time and to all the puppet users !

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Prompting fact refresh from Puppet Event

[Nigel Kersten]

On Wed, Dec 15, 2010 at 7:53 PM, Trevor Vaughan wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> No, because I'm not picking the 64 bit version.
>
> Someone else is picking the version and I have to figure out what it is
> and "do the right thing", making for extra fun times.
>

Are their version choices sane? You can't look at the arch of the machine
and work out which version got installed?



>
> Thanks though,
>
> Trevor
>
> On 12/15/2010 10:12 AM, Felix Frank wrote:
> > On 12/15/2010 03:46 PM, Trevor Vaughan wrote:
> >> Is is possible to spawn a puppet fact refresh from the completion of an
> event?
> >>
> >> Here's the scenario:
> >>
> >> 64-bit system, either a 64 or 32 bit application 'foo' could be
> installed.
> >>
> >> If the 64 bit version is installed, foo.conf needs to be configured
> >> with a prepended lib64 path. If not, then the 32bit default is fine.
> >>
> >> The only thing that I could come up with is a custom fact that returns
> >> the architecture of the installed package and then adjust the path
> >> based on that.
> >>
> >> However, this takes *two* puppet runs to complete and I would like to
> >> get this down to one run by prompting a fact refresh after the package
> >> is installed.
> >>
> >> Does anyone have any suggestions/techniques?
> >
> > Hrm, your manifest should be able to handle this one on its own.
> >
> > Assuming your foo.conf file is part of the foo::config class, add this
> > subclass:
> > foo::config::arch64bit {
> >   File["/etc/foo.conf"] { source => "..." }
> > }
> >
> > Then, at the place in the manifest where you pick the 64bit version of
> > the foo application, add an "include foo::config::arch64bit".
> >
> > Will this work for your manifest?
> >
> > Cheers,
> > Felix
> >
>
> - --
> Trevor Vaughan
>  Vice President, Onyx Point, Inc.
>  email: tvaug...@onyxpoint.com
>  phone: 410-541-ONYX (6699)
>  pgp: 0x6C701E94
>
> - -- This account not approved for unencrypted sensitive information --
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.11 (GNU/Linux)
>
> iQEcBAEBAgAGBQJNCY1LAAoJECNCGV1OLcypICYIAKwF9TvZNVt8LGY+F4Cw6gzD
> dbQrpVORDoJfSxCY7Yxwc9YH8NqBWaQY+A9pMvl13lReHnv3UfJGcjY9ryza6BkP
> sFUTuSuhOsKxoMxa5vYh5eny8QXaWSiVren46y7bis1J7qEYtgXJWXntNltNWf1M
> e+jZ8A2gUDV63NoO8x4SAAMPPmxugmwQW0H5JjvbERHmtV0j8DPb+xzAMCrsq2JA
> /eF+Gl9dtnSKQ1TWwpnyvmarIu9x0SvRX9vA1wzpzAXuqT/ZJxGUNqGg6LYPh5bm
> XKtHskZGedf4Qk9CaI+8WOAZSu7iUxMyBOey4ycp0+6Mf0tMJTxL2nRHFnQT89I=
> =K3Pw
> -END PGP SIGNATURE-
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-us...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>


-- 
Nigel Kersten - Puppet Labs -  http://www.puppetlabs.com

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: Separating puppetmaster file serving and catalogs

[Nigel Kersten]

On Wed, Dec 15, 2010 at 8:03 PM, Patrick  wrote:

>
> On Dec 15, 2010, at 6:08 AM, jcbollinger wrote:
>
> > On Dec 15, 5:04 am, Patrick  wrote:
> >> I'm looking for a way to run more than one puppetmaster on the same
> server under passenger.  Most of the puppet CPU load is waiting for the
> catalogs to compile.  This also seems to be mostly what takes large amounts
> of RAM.  I have storedconfigs on.
> >>
> >> I want to be able to move the fileserver to a different pool of
> puppetmaster processes.  Is there an easy way to tell the client, either in
> the catalog or in the config file, to get the files from a different port
> than the catalog?
> >
> > You can write source => "puppet:///..." instead of
> > source => "puppet:///...".  If  resolves to the
> > same physical machine then apache can direct it to a different virtual
> > host.  (And if it resolves to a different physical machine then no
> > worries on that level.)  I'm not sure, however, whether you can run
> > separate copies of Passenger in different vhosts.  (But if not, then
> > it would be a desirable feature.)
>
> This is a backup plan, but I would like to do this automatically without
> needing to change the manifests.
>

You can use a fact here in the place of the server name, then it can be
automatic.

If you have your own conditions client-side for working out what host is
best to get files from, then you can get quite effective load distribution.
I like this sort of self-organizing principle.

>
> >> Is there a better way to do this?  What I really want is for the cheap
> file requests to stop being blocked by the expensive catalog requests and
> keep the RAM usage low on the file serving processes.
> >
> > I don't know whether it's better, but since you're serving through
> > Apache anyway, you could serve your files directly via http.  That has
> > implications on where you put said files on disk and on both client
> > and file system security management, but it ought to be very fast, and
> > it will scale as Apache itself does.
>
> Is there a good way to do this without breaking subscription and notify?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-us...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>


-- 
Nigel Kersten - Puppet Labs -  http://www.puppetlabs.com

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Separating puppetmaster file serving and catalogs

[Patrick]

On Dec 15, 2010, at 1:48 PM, Brice Figureau wrote:

> On 15/12/10 12:04, Patrick wrote:
>> I'm looking for a way to run more than one puppetmaster on the same
>> server under passenger.  Most of the puppet CPU load is waiting for
>> the catalogs to compile.  This also seems to be mostly what takes
>> large amounts of RAM.  I have storedconfigs on.
> 
> If you don't need the full storedconfigs, you can use thin_storedconfigs
> for wy better performance.

Thanks.  I'm actually doing that, and misspoke in the first post.

>> Is there a better way to do this?  What I really want is for the
>> cheap file requests to stop being blocked by the expensive catalog
>> requests and keep the RAM usage low on the file serving processes.
> 
> You can use what I called file serving offloading:
> http://www.masterzen.fr/2010/03/21/more-puppet-offloading/

The file offloading is interesting.  So if I'm reading that right, that only 
makes a difference if some of the files are not in sync?




My original error was that I didn't set:
SSLProxyEngine on

Now I'm just getting errors that say all requests are forbidden.  I assume this 
is because the puppetmaster isn't seeing the headers from apache that have the 
SSL information.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: Separating puppetmaster file serving and catalogs

[Patrick]

On Dec 15, 2010, at 6:08 AM, jcbollinger wrote:

> On Dec 15, 5:04 am, Patrick  wrote:
>> I'm looking for a way to run more than one puppetmaster on the same server 
>> under passenger.  Most of the puppet CPU load is waiting for the catalogs to 
>> compile.  This also seems to be mostly what takes large amounts of RAM.  I 
>> have storedconfigs on.
>> 
>> I want to be able to move the fileserver to a different pool of puppetmaster 
>> processes.  Is there an easy way to tell the client, either in the catalog 
>> or in the config file, to get the files from a different port than the 
>> catalog?
> 
> You can write source => "puppet:///..." instead of
> source => "puppet:///...".  If  resolves to the
> same physical machine then apache can direct it to a different virtual
> host.  (And if it resolves to a different physical machine then no
> worries on that level.)  I'm not sure, however, whether you can run
> separate copies of Passenger in different vhosts.  (But if not, then
> it would be a desirable feature.)

This is a backup plan, but I would like to do this automatically without 
needing to change the manifests.

>> Is there a better way to do this?  What I really want is for the cheap file 
>> requests to stop being blocked by the expensive catalog requests and keep 
>> the RAM usage low on the file serving processes.
> 
> I don't know whether it's better, but since you're serving through
> Apache anyway, you could serve your files directly via http.  That has
> implications on where you put said files on disk and on both client
> and file system security management, but it ought to be very fast, and
> it will scale as Apache itself does.

Is there a good way to do this without breaking subscription and notify?

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Prompting fact refresh from Puppet Event

[Trevor Vaughan]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

No, because I'm not picking the 64 bit version.

Someone else is picking the version and I have to figure out what it is
and "do the right thing", making for extra fun times.

Thanks though,

Trevor

On 12/15/2010 10:12 AM, Felix Frank wrote:
> On 12/15/2010 03:46 PM, Trevor Vaughan wrote:
>> Is is possible to spawn a puppet fact refresh from the completion of an 
>> event?
>>
>> Here's the scenario:
>>
>> 64-bit system, either a 64 or 32 bit application 'foo' could be installed.
>>
>> If the 64 bit version is installed, foo.conf needs to be configured
>> with a prepended lib64 path. If not, then the 32bit default is fine.
>>
>> The only thing that I could come up with is a custom fact that returns
>> the architecture of the installed package and then adjust the path
>> based on that.
>>
>> However, this takes *two* puppet runs to complete and I would like to
>> get this down to one run by prompting a fact refresh after the package
>> is installed.
>>
>> Does anyone have any suggestions/techniques?
> 
> Hrm, your manifest should be able to handle this one on its own.
> 
> Assuming your foo.conf file is part of the foo::config class, add this
> subclass:
> foo::config::arch64bit {
>   File["/etc/foo.conf"] { source => "..." }
> }
> 
> Then, at the place in the manifest where you pick the 64bit version of
> the foo application, add an "include foo::config::arch64bit".
> 
> Will this work for your manifest?
> 
> Cheers,
> Felix
> 

- -- 
Trevor Vaughan
 Vice President, Onyx Point, Inc.
 email: tvaug...@onyxpoint.com
 phone: 410-541-ONYX (6699)
 pgp: 0x6C701E94

- -- This account not approved for unencrypted sensitive information --
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)

iQEcBAEBAgAGBQJNCY1LAAoJECNCGV1OLcypICYIAKwF9TvZNVt8LGY+F4Cw6gzD
dbQrpVORDoJfSxCY7Yxwc9YH8NqBWaQY+A9pMvl13lReHnv3UfJGcjY9ryza6BkP
sFUTuSuhOsKxoMxa5vYh5eny8QXaWSiVren46y7bis1J7qEYtgXJWXntNltNWf1M
e+jZ8A2gUDV63NoO8x4SAAMPPmxugmwQW0H5JjvbERHmtV0j8DPb+xzAMCrsq2JA
/eF+Gl9dtnSKQ1TWwpnyvmarIu9x0SvRX9vA1wzpzAXuqT/ZJxGUNqGg6LYPh5bm
XKtHskZGedf4Qk9CaI+8WOAZSu7iUxMyBOey4ycp0+6Mf0tMJTxL2nRHFnQT89I=
=K3Pw
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

<>

[Puppet Users] err: Could not call puppetbucket.addfile: #

[Paul Stepowski]

Puppet version: 0.24.8 (Red Hat Enterprise Linux Server release 5.3 (Tikanga))

My module runs fine but when I change the file on the server side I get the 
error:

err: Could not call puppetbucket.addfile: #

Note that I'm running puppet locally, the idea being I want to fun it from cron 
so puppet scales well.

Details below:

My module:

---snip---
# cat /var/puppet-local/modules/test/manifests/init.pp
class test {
file { "/etc/puppet-test":
owner   => root,
group   => root,
mode=> 600,
source  => "puppet:///test/test",
}
}
---snip---

The dummy file I'm trying to propagate:

---snip---
# cat /var/puppet-local/modules/test/files/test
1
---snip---

The error:

---snip---
# /usr/local/site/puppet/bin/puppet --verbose --debug 
--modulepath=/var/puppet-local/modules /var/puppet-local/manifests/site.pp
debug: importing '/var/puppet-local/manifests/modules.pp'
debug: importing '/var/puppet-local/modules/test/manifests/init.pp'
debug: importing '/var/puppet-local/manifests/classes.pp'
debug: importing '/var/puppet-local/manifests/nodes.pp'
debug: File[/etc/puppet-test]: Adding default for backup
info: mount[localhost]: Mounted /
info: mount[modules]: Mounted
info: mount[plugins]: Mounted
debug: Creating default schedules
debug: Failed to load library 'ldap' for feature 'ldap'
debug: Failed to load library 'shadow' for feature 'libshadow'
debug: //Node[default]/linux/test/File[/etc/puppet-test]/checksum: Initializing 
checksum hash
debug: //Node[default]/linux/test/File[/etc/puppet-test]: Creating checksum 
{md5}b026324c6904b2a9cb4b88d6d61c81d1
debug: //Node[default]/linux/test/File[/etc/puppet-test]: Changing source
debug: //Node[default]/linux/test/File[/etc/puppet-test]: 1 change(s)
debug: Calling puppetbucket.addfile
warning: peer certificate won't be verified in this SSL session
err: Could not call puppetbucket.addfile: #
err: //Node[default]/linux/test/File[/etc/puppet-test]/source: change from 
{md5}26ab0db90d72e28ad0ba1e22ee510510 to puppet:///test/test failed: 
HTTP-Error: 500 Internal Server Error
debug: Finishing transaction -606223158 with 1 changes
---snip---

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Could not find class

[Dmytro Bablinyuk]

Hi Everybody,

I am sure that I am missing something very obvious,

I have installed rsync package

git clone https://github.com/camptocamp/puppet-rsyncd.git rsync

The path

r...@puppet-master:/etc/puppet/modules/rsync/manifests/classes# ls -l
total 4
-rw-r--r-- 1 root root 1078 2010-12-16 09:54 rsyncd.pp

The manifest file

package { ["rsync", "xinetd"]: ensure => present }
service { "xinetd": ensure => running }

include rsyncd

rsyncd::export { "etc":
path => "/etc",
chroot => true,
gid => "staff",
allow => ["172.16.50.4", "172.16.41.10"],
}

I have error on the client

r...@puppet-client-ubuntu:~# puppetd --test
info: Retrieving plugin
err: Could not retrieve catalog from remote server: Error 400 on
SERVER: Could not find class rsyncd at /etc/puppet/manifests/site.pp:
33 on node puppet-client-ubuntu.harbour
warning: Not using cache on failed catalog
err: Could not retrieve catalog; skipping run

How do I specify path to the rsyncd class?

Thank you
Dmytro

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: puppetmaster 100%cpu usage on 2.6 (not on 0.24)

[Nigel Kersten]

On Wed, Dec 15, 2010 at 4:47 PM, Ashley Penney  wrote:

> Just to reply to this - like I said earlier I can get this problem with 1
> node checking in against puppetmaster.  All the puppetmasterd processes use
> maximum CPU.  It's not a scaling issue considering serving one node is
> certainly not going to max out a newish physical server.


That is definitely a problem.

Does this happen as soon as a node checks in? Or as soon as you start the
passenger processes?

Can you post a sanitized strace somewhere?


>
>
> On Wed, Dec 15, 2010 at 4:45 PM, Brice Figureau <
> brice-pup...@daysofwonder.com> wrote:
>
>>
>> Just some math (which might be totally wrong), to give an idea of how I
>> think we can compute our optimal scaling case:
>> So with 250 nodes and a sleep time of 30 minutes, we need to overcome
>> 250 compiles in every 30 minute time spans. If we assume a concurrency
>> of 2 and all nodes evenly spaced (in time), that means we must compile
>> 125 nodes in 30 minutes. If each compilation takes about 10s, then that
>> means it'll take 1250s, which means 20 minutes so you have some room for
>> growth :)
>> Now during those 20min your 2 master processes will consume 100% CPU.
>> Since we're consuming the the CPU for only 66% of the 30 minute span,
>> you'll globally consume 66% of all your CPU available...
>>
>> Hope that helps,
>>
>> [1]: http://projects.puppetlabs.com/projects/1/wiki/Puppet_Introspection
>> --
>> Brice Figureau
>> My Blog: http://www.masterzen.fr/
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Puppet Users" group.
>> To post to this group, send email to puppet-us...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> puppet-users+unsubscr...@googlegroups.com
>> .
>> For more options, visit this group at
>> http://groups.google.com/group/puppet-users?hl=en.
>>
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-us...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>



-- 
Nigel Kersten - Puppet Labs -  http://www.puppetlabs.com

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: puppetmaster 100%cpu usage on 2.6 (not on 0.24)

[Ashley Penney]

Just to reply to this - like I said earlier I can get this problem with 1
node checking in against puppetmaster.  All the puppetmasterd processes use
maximum CPU.  It's not a scaling issue considering serving one node is
certainly not going to max out a newish physical server.

On Wed, Dec 15, 2010 at 4:45 PM, Brice Figureau <
brice-pup...@daysofwonder.com> wrote:

>
> Just some math (which might be totally wrong), to give an idea of how I
> think we can compute our optimal scaling case:
> So with 250 nodes and a sleep time of 30 minutes, we need to overcome
> 250 compiles in every 30 minute time spans. If we assume a concurrency
> of 2 and all nodes evenly spaced (in time), that means we must compile
> 125 nodes in 30 minutes. If each compilation takes about 10s, then that
> means it'll take 1250s, which means 20 minutes so you have some room for
> growth :)
> Now during those 20min your 2 master processes will consume 100% CPU.
> Since we're consuming the the CPU for only 66% of the 30 minute span,
> you'll globally consume 66% of all your CPU available...
>
> Hope that helps,
>
> [1]: http://projects.puppetlabs.com/projects/1/wiki/Puppet_Introspection
> --
> Brice Figureau
> My Blog: http://www.masterzen.fr/
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-us...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Autorequiring and dependency problem

[Stefan Schulte]

On Wed, Dec 15, 2010 at 02:27:46PM -0800, Jay N. wrote:
> Hi,
> 
> I thought I would have no problem with that but /etc/puppet is
> autorequired by /etc/puppet/files, which means a dependency cycle. So
> my first question is, is there a way to avoid that unwanted
> autorequiring?
> 

An explicit require should overwrite any autorequire and should not
cause a dependency cycle. You should see something like this when
running in debugmode

debug: /Stage[main]//Node[default]/File[/etc/puppet/files]: Skipping
automatic relationship with File[/etc/puppet]

I just tried that on my own to be sure:

  node default {
file { '/etc/puppet':
  require => File['/etc/puppet/files'],
  ensure => directory,
}
file { '/etc/puppet/files':
  ensure => directory,
}
  }

Worked for me

-Stefan


pgpC4tNwfWxDz.pgp
Description: PGP signature

[Puppet Users] Autorequiring and dependency problem

[Jay N.]

Hi,

I have a dependency problem in my configuration. I have thoses 2
objects.

file { "/etc/puppet/files":
...
}

file { "/etc/puppet":
...
require => File["/etc/puppet/files"],
}

I thought I would have no problem with that but /etc/puppet is
autorequired by /etc/puppet/files, which means a dependency cycle. So
my first question is, is there a way to avoid that unwanted
autorequiring?

Let me explain my choice : I first change the owner of /etc/puppet/
files for someone else than root being able to write. Then I commit my
svn configuration into /etc/puppet/files ( I do like that because root
doesn't have the right to checkout, only some users does ), and in the
end I update my puppet configuration ( with the /etc/puppet object )
using the checkouted files. Maybe there's a better way to do it.

Second time : when I saw there was a problem, I tried the dirty way by
commenting the /etc/puppet/files object and creating an exec object
doing just the same.

exec { "Dirty way":
command => "chown svn /etc/puppet/files",
path => "/usr/bin:/usr/sbin",
unless => "ls -la /etc/puppet/files | head -2 | grep grpsvn",
}

Always the same dependency problem because "Dirty way" autorequires /
etc/puppet ( I changed the require in /etc/puppet too ).

The weird thing I realized is that if I change my exec to :
exec { "Dirty way":
command => "chgrp grpsvn /etc/puppet/files",
path => "/usr/bin:/usr/sbin",
unless => "cd /etc; ls -la puppet/files | head -2 | grep grpsvn",
}

then I don't have any more the autorequire and the configuration is
working just fine. Is it normal that puppet parses the unless
parameter ( and maybe others ) to find autorequirements? Isn't it too
much autorequirements?

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Separating puppetmaster file serving and catalogs

[Brice Figureau]

On 15/12/10 12:04, Patrick wrote:
> I'm looking for a way to run more than one puppetmaster on the same
> server under passenger.  Most of the puppet CPU load is waiting for
> the catalogs to compile.  This also seems to be mostly what takes
> large amounts of RAM.  I have storedconfigs on.

If you don't need the full storedconfigs, you can use thin_storedconfigs
for wy better performance.

> I want to be able to move the fileserver to a different pool of
> puppetmaster processes.  Is there an easy way to tell the client,
> either in the catalog or in the config file, to get the files from a
> different port than the catalog?

In every puppet:// url you can specify a different server. You can
dedicate some masters to serve files only and others to server catalogs.

> Is there a better way to do this?  What I really want is for the
> cheap file requests to stop being blocked by the expensive catalog
> requests and keep the RAM usage low on the file serving processes.

You can use what I called file serving offloading:
http://www.masterzen.fr/2010/03/21/more-puppet-offloading/

-- 
Brice Figureau
My Blog: http://www.masterzen.fr/

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: puppetmaster 100%cpu usage on 2.6 (not on 0.24)

[Brice Figureau]

On 15/12/10 21:10, Disconnect wrote:
> As a datapoint, this exact config (with mongrel_cluster) was working
> great under 0.25.x. With fewer, slower cpus, slower storage (vm image
> files) and 2G of ram...

So I ask it again: could it be a problem with passenger more than an
issue with puppet itself?

It would really be interesting to use some ruby introspection[1] to find
exactly where the cpu is spent in those masters.

Like with passenger it reparses everything instead of just compiling?
(I simply don't know, just throwing out some ideas)

I myself use nginx + mongrel, but have only a dozen of nodes, so I don't
really qualify.

> I gave puppet-load a try, but it is throwing errors that I don't have
> time to dig into today:
> debug: reading facts from: puppet.foo.com.yaml
> /var/lib/gems/1.8/gems/em-http-request-0.2.15/lib/em-http/request.rb:72:in
> `send_request': uninitialized constant EventMachine::ConnectionError
> (NameError)
> from
> /var/lib/gems/1.8/gems/em-http-request-0.2.15/lib/em-http/request.rb:59:in
> `setup_request'
> from
> /var/lib/gems/1.8/gems/em-http-request-0.2.15/lib/em-http/request.rb:49:in
> `get'
> from ./puppet-load.rb:272:in `spawn_request'
> from ./puppet-load.rb:334:in `spawn'

Could it be that you're missing EventMachine?

> Running about 250 nodes, every 30 minutes.

Did you try to use mongrel?
Do you use splay time?

Just some math (which might be totally wrong), to give an idea of how I
think we can compute our optimal scaling case:
So with 250 nodes and a sleep time of 30 minutes, we need to overcome
250 compiles in every 30 minute time spans. If we assume a concurrency
of 2 and all nodes evenly spaced (in time), that means we must compile
125 nodes in 30 minutes. If each compilation takes about 10s, then that
means it'll take 1250s, which means 20 minutes so you have some room for
growth :)
Now during those 20min your 2 master processes will consume 100% CPU.
Since we're consuming the the CPU for only 66% of the 30 minute span,
you'll globally consume 66% of all your CPU available...

Hope that helps,

[1]: http://projects.puppetlabs.com/projects/1/wiki/Puppet_Introspection
-- 
Brice Figureau
My Blog: http://www.masterzen.fr/

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] 'Cleanup' stage in 2.6

[Morgan Haskel]

Thanks!

On Wed, Dec 15, 2010 at 4:14 PM, James Turnbull wrote:

> Morgan Haskel wrote:
> > Hi All,
> >
> > I have a script that I want to run at the end of my puppet run after
> > every run.  Is there a way to create a 'cleanup' stage that will run
> > at the end of the stage even if there are failures in previous stages?
>
> Pre and post run hooks where added in 0.25.4.  See:
>
> http://projects.puppetlabs.com/projects/puppet/wiki/Release_Notes#0.25.4
>
> Regards
>
> James Turnbull
>
> --
> Puppet Labs - http://www.puppetlabs.com
> C: 503-734-8571
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-us...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] 'Cleanup' stage in 2.6

[James Turnbull]

Morgan Haskel wrote:
> Hi All,
> 
> I have a script that I want to run at the end of my puppet run after
> every run.  Is there a way to create a 'cleanup' stage that will run
> at the end of the stage even if there are failures in previous stages?

Pre and post run hooks where added in 0.25.4.  See:

http://projects.puppetlabs.com/projects/puppet/wiki/Release_Notes#0.25.4

Regards

James Turnbull

-- 
Puppet Labs - http://www.puppetlabs.com
C: 503-734-8571

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] 'Cleanup' stage in 2.6

[Morgan Haskel]

Hi All,

I have a script that I want to run at the end of my puppet run after
every run.  Is there a way to create a 'cleanup' stage that will run
at the end of the stage even if there are failures in previous stages?

Thanks,
Morgan

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: puppetmaster 100%cpu usage on 2.6 (not on 0.24)

[Disconnect]

As a datapoint, this exact config (with mongrel_cluster) was working great
under 0.25.x. With fewer, slower cpus, slower storage (vm image files) and
2G of ram...

I gave puppet-load a try, but it is throwing errors that I don't have time
to dig into today:
debug: reading facts from: puppet.foo.com.yaml
/var/lib/gems/1.8/gems/em-http-request-0.2.15/lib/em-http/request.rb:72:in
`send_request': uninitialized constant EventMachine::ConnectionError
(NameError)
from
/var/lib/gems/1.8/gems/em-http-request-0.2.15/lib/em-http/request.rb:59:in
`setup_request'
from
/var/lib/gems/1.8/gems/em-http-request-0.2.15/lib/em-http/request.rb:49:in
`get'
from ./puppet-load.rb:272:in `spawn_request'
from ./puppet-load.rb:334:in `spawn'

Running about 250 nodes, every 30 minutes.

On Wed, Dec 15, 2010 at 2:43 PM, Brice Figureau <
brice-pup...@daysofwonder.com> wrote:

> On 15/12/10 20:24, Disconnect wrote:
> > On Wed, Dec 15, 2010 at 2:14 PM, Brice Figureau
> > mailto:brice-pup...@daysofwonder.com>>
> > wrote:
> >>
> >> Note: we were talking about the puppet master taking 100% CPU, but
> >> you're apparently looking to the puppet agent, which is a different
> story.
> >>
> >
> > The agent isn't taking cpu, it is hanging waiting for the master to do
> > anything. (The run I quoted earlier eventually ended with a timeout..)
> > The master has pegged the cpus, and it seems to be related to file
> > resources:
>
> Oh, I see.
>
> > $ ps auxw|grep master
> > puppet   31392 74.4  4.7 361720 244348 ?   R10:42 162:06 Rack:
> > /usr/share/puppet/rack/puppetmasterd
> >
> > puppet   31396 70.0  4.9 369524 250200 ?   R10:42 152:32 Rack:
> > /usr/share/puppet/rack/puppetmasterd
> >
> > puppet   31398 66.2  3.9 318828 199472 ?   R10:42 144:10 Rack:
> > /usr/share/puppet/rack/puppetmasterd
> >
> > puppet   31400 66.6  4.9 369992 250588 ?   R10:42 145:04 Rack:
> > /usr/share/puppet/rack/puppetmasterd
> >
> > puppet   31406 68.6  3.9 318292 200992 ?   R10:42 149:31 Rack:
> > /usr/share/puppet/rack/puppetmasterd
> >
> > puppet   31414 67.0  2.4 243800 124476 ?   R10:42 146:00 Rack:
> > /usr/share/puppet/rack/puppetmasterd
>
> Note that they're all running. That means there is none left to serve
> file content if they are all busy for several seconds (in our case
> around 20) while compiling catalogs.
>
> > Dec 15 13:42:23 puppet puppet-master[31406]: Compiled catalog for
> > puppet.foo.com  in environment production in
> > 30.83 seconds
> > Dec 15 13:42:49 puppet puppet-agent[10515]: Caching catalog for
> > puppet.foo.com 
> > Dec 15 14:00:18 puppet puppet-agent[10515]: Applying configuration
> > version '1292438512'
> > ...
> > Dec 15 14:14:56 puppet puppet-agent[10515]: Finished catalog run in
> > 882.43 seconds
> > Changes:
> > Total: 6
> > Events:
> >   Success: 6
> > Total: 6
> > Resources:
> >   Changed: 6
> >   Out of sync: 6
> > Total: 287
>
> That's not a big number.
>
> > Time:
> >Config retrieval: 72.20
>
> This is also suspect.
>
> >  Cron: 0.05
> >  Exec: 32.42
> >  File: 752.33
>
> Indeed.
>
> >Filebucket: 0.00
> > Mount: 0.98
> >   Package: 6.13
> >  Schedule: 0.02
> >   Service: 9.09
> >Ssh authorized key: 0.07
> >Sysctl: 0.00
> >
> > real34m56.066s
> > user1m6.030s
> > sys0m26.590s
> >
>
> That just means your master are so busy serving catalogs that they
> barely have the time to serve files. One possibility is to use file
> content offloading (see one of my blog post about this:
> http://www.masterzen.fr/2010/03/21/more-puppet-offloading/).
>
> How many nodes are you compiling at the same time? Apparently you have 6
> master processes running at high CPU usage.
>
> As I said earlier, I really advise people to try puppet-load (which can
> be found in the ext/ directory of the source tarball since puppet 2.6)
> to execise load againts a master. This will help you find your actual
> concurrency.
>
> But, if it's a bug, then could this be an issue with passenger?
> --
> Brice Figureau
> My Blog: http://www.masterzen.fr/
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-us...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: puppetmaster 100%cpu usage on 2.6 (not on 0.24)

[Brice Figureau]

On 15/12/10 20:24, Disconnect wrote:
> On Wed, Dec 15, 2010 at 2:14 PM, Brice Figureau
> mailto:brice-pup...@daysofwonder.com>>
> wrote:
>>
>> Note: we were talking about the puppet master taking 100% CPU, but
>> you're apparently looking to the puppet agent, which is a different story.
>>
> 
> The agent isn't taking cpu, it is hanging waiting for the master to do
> anything. (The run I quoted earlier eventually ended with a timeout..)
> The master has pegged the cpus, and it seems to be related to file
> resources:

Oh, I see.

> $ ps auxw|grep master
> puppet   31392 74.4  4.7 361720 244348 ?   R10:42 162:06 Rack:
> /usr/share/puppet/rack/puppetmasterd  
>  
> 
> puppet   31396 70.0  4.9 369524 250200 ?   R10:42 152:32 Rack:
> /usr/share/puppet/rack/puppetmasterd  
>  
> 
> puppet   31398 66.2  3.9 318828 199472 ?   R10:42 144:10 Rack:
> /usr/share/puppet/rack/puppetmasterd  
>  
> 
> puppet   31400 66.6  4.9 369992 250588 ?   R10:42 145:04 Rack:
> /usr/share/puppet/rack/puppetmasterd  
>  
> 
> puppet   31406 68.6  3.9 318292 200992 ?   R10:42 149:31 Rack:
> /usr/share/puppet/rack/puppetmasterd  
>  
> 
> puppet   31414 67.0  2.4 243800 124476 ?   R10:42 146:00 Rack:
> /usr/share/puppet/rack/puppetmasterd  
> 

Note that they're all running. That means there is none left to serve
file content if they are all busy for several seconds (in our case
around 20) while compiling catalogs.

> Dec 15 13:42:23 puppet puppet-master[31406]: Compiled catalog for
> puppet.foo.com  in environment production in
> 30.83 seconds
> Dec 15 13:42:49 puppet puppet-agent[10515]: Caching catalog for
> puppet.foo.com 
> Dec 15 14:00:18 puppet puppet-agent[10515]: Applying configuration
> version '1292438512'
> ...
> Dec 15 14:14:56 puppet puppet-agent[10515]: Finished catalog run in
> 882.43 seconds
> Changes:
> Total: 6
> Events:
>   Success: 6
> Total: 6
> Resources:
>   Changed: 6
>   Out of sync: 6
> Total: 287

That's not a big number.

> Time:
>Config retrieval: 72.20

This is also suspect.

>  Cron: 0.05
>  Exec: 32.42
>  File: 752.33

Indeed.

>Filebucket: 0.00
> Mount: 0.98
>   Package: 6.13
>  Schedule: 0.02
>   Service: 9.09
>Ssh authorized key: 0.07
>Sysctl: 0.00
> 
> real34m56.066s
> user1m6.030s
> sys0m26.590s
> 

That just means your master are so busy serving catalogs that they
barely have the time to serve files. One possibility is to use file
content offloading (see one of my blog post about this:
http://www.masterzen.fr/2010/03/21/more-puppet-offloading/).

How many nodes are you compiling at the same time? Apparently you have 6
master processes running at high CPU usage.

As I said earlier, I really advise people to try puppet-load (which can
be found in the ext/ directory of the source tarball since puppet 2.6)
to execise load againts a master. This will help you find your actual
concurrency.

But, if it's a bug, then could this be an issue with passenger?
-- 
Brice Figureau
My Blog: http://www.masterzen.fr/

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: puppetmaster 100%cpu usage on 2.6 (not on 0.24)

[Disconnect]

On Wed, Dec 15, 2010 at 2:14 PM, Brice Figureau <
brice-pup...@daysofwonder.com> wrote:
>
> Note: we were talking about the puppet master taking 100% CPU, but
> you're apparently looking to the puppet agent, which is a different story.
>

The agent isn't taking cpu, it is hanging waiting for the master to do
anything. (The run I quoted earlier eventually ended with a timeout..) The
master has pegged the cpus, and it seems to be related to file resources:

$ ps auxw|grep master
puppet   31392 74.4  4.7 361720 244348 ?   R10:42 162:06 Rack:
/usr/share/puppet/rack/puppetmasterd

puppet   31396 70.0  4.9 369524 250200 ?   R10:42 152:32 Rack:
/usr/share/puppet/rack/puppetmasterd

puppet   31398 66.2  3.9 318828 199472 ?   R10:42 144:10 Rack:
/usr/share/puppet/rack/puppetmasterd

puppet   31400 66.6  4.9 369992 250588 ?   R10:42 145:04 Rack:
/usr/share/puppet/rack/puppetmasterd

puppet   31406 68.6  3.9 318292 200992 ?   R10:42 149:31 Rack:
/usr/share/puppet/rack/puppetmasterd

puppet   31414 67.0  2.4 243800 124476 ?   R10:42 146:00 Rack:
/usr/share/puppet/rack/puppetmasterd


Dec 15 13:42:23 puppet puppet-master[31406]: Compiled catalog for
puppet.foo.com in environment production in 30.83 seconds
Dec 15 13:42:49 puppet puppet-agent[10515]: Caching catalog for
puppet.foo.com
Dec 15 14:00:18 puppet puppet-agent[10515]: Applying configuration version
'1292438512'
...
Dec 15 14:14:56 puppet puppet-agent[10515]: Finished catalog run in 882.43
seconds
Changes:
Total: 6
Events:
  Success: 6
Total: 6
Resources:
  Changed: 6
  Out of sync: 6
Total: 287
Time:
   Config retrieval: 72.20
 Cron: 0.05
 Exec: 32.42
 File: 752.33
   Filebucket: 0.00
Mount: 0.98
  Package: 6.13
 Schedule: 0.02
  Service: 9.09
   Ssh authorized key: 0.07
   Sysctl: 0.00

real34m56.066s
user1m6.030s
sys0m26.590s

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: puppetmaster 100%cpu usage on 2.6 (not on 0.24)

[Brice Figureau]

On 15/12/10 19:35, Disconnect wrote:
> "me too". All the logs show nice quick compilations but the actual wall
> clock to get anything done is HUGE.
> 
> Dec 15 13:10:29 puppet puppet-master[31406]: Compiled catalog for
> puppet.foo.com  in environment production in
> 21.52 seconds

This looks long.

> Dec 15 13:10:51 puppet puppet-agent[8251]: Caching catalog for
> puppet.foo.com 
> 
> That was almost 30 minutes ago. Since then, it has sat there doing
> nothing...
> $ sudo strace -p 8251
> Process 8251 attached - interrupt to quit
> select(7, [6], [], [], {866, 578560}
> 
> lsof shows:
> puppetd 8251 root6u  IPv4   11016045  0t0  TCP
> puppet.foo.com:33065->puppet.foo.com:8140 
> (ESTABLISHED)

Note: we were talking about the puppet master taking 100% CPU, but
you're apparently looking to the puppet agent, which is a different story.

-- 
Brice Figureau
My Blog: http://www.masterzen.fr/

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: puppetmaster 100%cpu usage on 2.6 (not on 0.24)

[Brice Figureau]

On 15/12/10 19:27, Ashley Penney wrote:
> This issue is definitely a problem.  I have a support ticket in with
> Puppet Labs about the same thing.  My CPU remains at 100% almost
> constantly and it slows things down significantly.  If you strace it you
> can see that very little appears to be going on.  This is absolutely not
> normal behavior.  Even when I had 1 client checking in I had all cores
> fully used.

I do agree that it's not the correct behavior. I suggest you to strace
or use any other ruby introspection techniques to find what part of the
master is taking CPU.
-- 
Brice Figureau
My Blog: http://www.masterzen.fr/

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Separating puppetmaster file serving and catalogs

[Patrick]

On Dec 15, 2010, at 3:09 AM, Felix Frank wrote:

> 
> 
> On 12/15/2010 12:04 PM, Patrick wrote:
>> I'm looking for a way to run more than one puppetmaster on the same server 
>> under passenger.  Most of the puppet CPU load is waiting for the catalogs to 
>> compile.  This also seems to be mostly what takes large amounts of RAM.  I 
>> have storedconfigs on.
>> 
>> I want to be able to move the fileserver to a different pool of puppetmaster 
>> processes.  Is there an easy way to tell the client, either in the catalog 
>> or in the config file, to get the files from a different port than the 
>> catalog?
>> 
>> Is there a better way to do this?  What I really want is for the cheap file 
>> requests to stop being blocked by the expensive catalog requests and keep 
>> the RAM usage low on the file serving processes.
>> 
> 
> Hi,
> 
> what springs to mind is a webserver with mod_proxy up front (or in fact,
> any intelligent reverse proxy), that chooses your actual webserver with
> respect to request URIs.
> 
> Fileserver requests do go to a different root directory, yes?

Technically, I decided to just pass the catalog requests through instead.  
Catalog requests are sent to "/production/catalog/" so it should be easy to do.

Do you have any advice for me before I try separating the proxy into a 
different (third) VirtualServer?

I setup a second server on 8141 and I can send puppet requests to that server 
just fine too.

Then I added these lines in my VirtualHost block:
ProxyPass /production/catalog/ 
https://localhost:8141/production/catalog/
ProxyPassReverse /production/catalog/ 
https://localhost:8141/production/catalog/

They don't seem to have any effect though.

I can see the log entries like this one (I chopped off 3 pages of facts) which 
shows the URL:
Simba.Outer:8140 192.168.2.252 - - [15/Dec/2010:10:21:07 -0800] "GET 
/production/catalog/simba.outer?facts=eNp1 HTTP/1.1" 200 95433 "-" "-"

My config file for the primary virtual server is here:
http://pastie.org/1380225

In summery, both servers work, but no redirection is taking place.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: puppetmaster 100%cpu usage on 2.6 (not on 0.24)

[Disconnect]

One addendum - the box is absolutely not io or cpu bound:
Cpu(s): 83.0%us, 13.1%sy,  0.0%ni,  2.5%id,  0.0%wa,  0.1%hi,  1.3%si,
0.0%st

(64bit kvm vm w/ 6 3.5ghz amd64 cpus, on an lvm partition - raw disk - with
5G ram, but only using 3 gigs. PLENTY of power, and monitoring supports
that..)

On Wed, Dec 15, 2010 at 1:35 PM, Disconnect  wrote:

> "me too". All the logs show nice quick compilations but the actual wall
> clock to get anything done is HUGE.
>
> Dec 15 13:10:29 puppet puppet-master[31406]: Compiled catalog for
> puppet.foo.com in environment production in 21.52 seconds
> Dec 15 13:10:51 puppet puppet-agent[8251]: Caching catalog for
> puppet.foo.com
>
> That was almost 30 minutes ago. Since then, it has sat there doing
> nothing...
> $ sudo strace -p 8251
> Process 8251 attached - interrupt to quit
> select(7, [6], [], [], {866, 578560}
>
> lsof shows:
> puppetd 8251 root6u  IPv4   11016045  0t0  TCP
> puppet.foo.com:33065->puppet.foo.com:8140 (ESTABLISHED)
>
>
>
> On Wed, Dec 15, 2010 at 1:27 PM, Ashley Penney  wrote:
>
>> This issue is definitely a problem.  I have a support ticket in with
>> Puppet Labs about the same thing.  My CPU remains at 100% almost constantly
>> and it slows things down significantly.  If you strace it you can see that
>> very little appears to be going on.  This is absolutely not normal behavior.
>>  Even when I had 1 client checking in I had all cores fully used.
>>
>>
>> On Wed, Dec 15, 2010 at 12:15 PM, Brice Figureau <
>> brice-pup...@daysofwonder.com> wrote:
>>
>>> On Wed, 2010-12-15 at 05:28 -0800, Chris wrote:
>>> >
>>> > On Dec 15, 12:42 pm, Brice Figureau 
>>> > wrote:
>>> > > On Tue, 2010-12-14 at 00:24 -0800, Chris wrote:
>>> > > > Hi
>>> > >
>>> > > > I recently upgraded my puppet masters (and clients) from 0.24.8 to
>>> > > > 2.6.4
>>> > >
>>> > > > Previously, my most busy puppet master would hover around about 0.9
>>> > > > load  average, after the upgrade, its load hovers around 5
>>> > >
>>> > > > I am running passenger and mysql based stored configs.
>>> > >
>>> > > > Checking my running processes, ruby (puppetmasterd) shoots up to
>>> 99%
>>> > > > cpu load and stays there for a few seconds before dropping again.
>>> > > > Often there are 4 of these running simultaneously, pegging each
>>> core
>>> > > > at 99% cpu.
>>> > >
>>> > > I would say it is perfectly normal. Compiling the catalog is a hard
>>> and
>>> > > complex problem and requires CPU.
>>> > >
>>> > > The difference between 0.24.8 and 2.6 (or 0.25 for what matters) is
>>> that
>>> > > some performance issues have been fixed. Those issues made the master
>>> be
>>> > > more I/O bound under 0.24, but now mostly CPU bound in later
>>> versions.
>>> >
>>> > If we were talking about only cpu usage, I would agree with you.  But
>>> > in this case, the load average of the machine has gone up over 5x.
>>> > And as high load average indicates processes not getting enough
>>> > runtime, in this case it is an indication to me that 2.6 is performing
>>> > worse than 0.24 (previously, on average, all processes got enough
>>> > runtime and did not have to wait for system resources, now processes
>>> > are sitting in the run queue, waiting to get a chance to run)
>>>
>>> Load is not necessarily an indication of an issue. It can also mean some
>>> tasks are waiting for I/O not only CPU.
>>> The only real issue under load is if service time is beyond an
>>> admissible value, otherwise you can't say it's bad or not.
>>> If you see some hosts reporting timeouts, then it's an indication that
>>> service time is not good :)
>>>
>>> BTW, do you run your mysql storedconfig instance on the same server?
>>> You can activate thin_storeconfigs to reduce the load on the mysql db.
>>>
>>> > >
>>> > > I don't really get what is the issue about using 100% of CPU?
>>> > Thats not the issue, just an indication of what is causing it
>>> >
>>> > >
>>> > > You're paying about the same price when your CPU is used and when
>>> it's
>>> > > idle, so that shouldn't make a difference :)
>>> > Generally true, but this is a on VM which is also running some of my
>>> > radius and proxy instances, amongst others.
>>> >
>>> > >
>>> > > If that's an issue, reduce the concurrency of your setup (run less
>>> > > compilation in parallel, implement splay time, etc...).
>>> > splay has been enabled since 0.24
>>> >
>>> > My apache maxclients is set to 15 to limit concurrency.
>>>
>>> I think this is too many except if you have 8 cores. As Trevor said in
>>> another e-mail in this thread, 2PM/Core is the best.
>>>
>>> Now it all depends on your number of nodes and sleeptime. I suggest you
>>> use ext/puppet-load to find your setup real concurrency.
>>> --
>>> Brice Figureau
>>> Follow the latest Puppet Community evolutions on www.planetpuppet.org!
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "Puppet Users" group.
>>> To post to this group, send email to puppet-us...@goog

Re: [Puppet Users] Re: puppetmaster 100%cpu usage on 2.6 (not on 0.24)

[Disconnect]

"me too". All the logs show nice quick compilations but the actual wall
clock to get anything done is HUGE.

Dec 15 13:10:29 puppet puppet-master[31406]: Compiled catalog for
puppet.foo.com in environment production in 21.52 seconds
Dec 15 13:10:51 puppet puppet-agent[8251]: Caching catalog for
puppet.foo.com

That was almost 30 minutes ago. Since then, it has sat there doing
nothing...
$ sudo strace -p 8251
Process 8251 attached - interrupt to quit
select(7, [6], [], [], {866, 578560}

lsof shows:
puppetd 8251 root6u  IPv4   11016045  0t0  TCP
puppet.foo.com:33065->puppet.foo.com:8140 (ESTABLISHED)


On Wed, Dec 15, 2010 at 1:27 PM, Ashley Penney  wrote:

> This issue is definitely a problem.  I have a support ticket in with Puppet
> Labs about the same thing.  My CPU remains at 100% almost constantly and it
> slows things down significantly.  If you strace it you can see that very
> little appears to be going on.  This is absolutely not normal behavior.
>  Even when I had 1 client checking in I had all cores fully used.
>
>
> On Wed, Dec 15, 2010 at 12:15 PM, Brice Figureau <
> brice-pup...@daysofwonder.com> wrote:
>
>> On Wed, 2010-12-15 at 05:28 -0800, Chris wrote:
>> >
>> > On Dec 15, 12:42 pm, Brice Figureau 
>> > wrote:
>> > > On Tue, 2010-12-14 at 00:24 -0800, Chris wrote:
>> > > > Hi
>> > >
>> > > > I recently upgraded my puppet masters (and clients) from 0.24.8 to
>> > > > 2.6.4
>> > >
>> > > > Previously, my most busy puppet master would hover around about 0.9
>> > > > load  average, after the upgrade, its load hovers around 5
>> > >
>> > > > I am running passenger and mysql based stored configs.
>> > >
>> > > > Checking my running processes, ruby (puppetmasterd) shoots up to 99%
>> > > > cpu load and stays there for a few seconds before dropping again.
>> > > > Often there are 4 of these running simultaneously, pegging each core
>> > > > at 99% cpu.
>> > >
>> > > I would say it is perfectly normal. Compiling the catalog is a hard
>> and
>> > > complex problem and requires CPU.
>> > >
>> > > The difference between 0.24.8 and 2.6 (or 0.25 for what matters) is
>> that
>> > > some performance issues have been fixed. Those issues made the master
>> be
>> > > more I/O bound under 0.24, but now mostly CPU bound in later versions.
>> >
>> > If we were talking about only cpu usage, I would agree with you.  But
>> > in this case, the load average of the machine has gone up over 5x.
>> > And as high load average indicates processes not getting enough
>> > runtime, in this case it is an indication to me that 2.6 is performing
>> > worse than 0.24 (previously, on average, all processes got enough
>> > runtime and did not have to wait for system resources, now processes
>> > are sitting in the run queue, waiting to get a chance to run)
>>
>> Load is not necessarily an indication of an issue. It can also mean some
>> tasks are waiting for I/O not only CPU.
>> The only real issue under load is if service time is beyond an
>> admissible value, otherwise you can't say it's bad or not.
>> If you see some hosts reporting timeouts, then it's an indication that
>> service time is not good :)
>>
>> BTW, do you run your mysql storedconfig instance on the same server?
>> You can activate thin_storeconfigs to reduce the load on the mysql db.
>>
>> > >
>> > > I don't really get what is the issue about using 100% of CPU?
>> > Thats not the issue, just an indication of what is causing it
>> >
>> > >
>> > > You're paying about the same price when your CPU is used and when it's
>> > > idle, so that shouldn't make a difference :)
>> > Generally true, but this is a on VM which is also running some of my
>> > radius and proxy instances, amongst others.
>> >
>> > >
>> > > If that's an issue, reduce the concurrency of your setup (run less
>> > > compilation in parallel, implement splay time, etc...).
>> > splay has been enabled since 0.24
>> >
>> > My apache maxclients is set to 15 to limit concurrency.
>>
>> I think this is too many except if you have 8 cores. As Trevor said in
>> another e-mail in this thread, 2PM/Core is the best.
>>
>> Now it all depends on your number of nodes and sleeptime. I suggest you
>> use ext/puppet-load to find your setup real concurrency.
>> --
>> Brice Figureau
>> Follow the latest Puppet Community evolutions on www.planetpuppet.org!
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Puppet Users" group.
>> To post to this group, send email to puppet-us...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> puppet-users+unsubscr...@googlegroups.com
>> .
>> For more options, visit this group at
>> http://groups.google.com/group/puppet-users?hl=en.
>>
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-us...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com
> .
> F

Re: [Puppet Users] Re: puppetmaster 100%cpu usage on 2.6 (not on 0.24)

[Ashley Penney]

This issue is definitely a problem.  I have a support ticket in with Puppet
Labs about the same thing.  My CPU remains at 100% almost constantly and it
slows things down significantly.  If you strace it you can see that very
little appears to be going on.  This is absolutely not normal behavior.
 Even when I had 1 client checking in I had all cores fully used.

On Wed, Dec 15, 2010 at 12:15 PM, Brice Figureau <
brice-pup...@daysofwonder.com> wrote:

> On Wed, 2010-12-15 at 05:28 -0800, Chris wrote:
> >
> > On Dec 15, 12:42 pm, Brice Figureau 
> > wrote:
> > > On Tue, 2010-12-14 at 00:24 -0800, Chris wrote:
> > > > Hi
> > >
> > > > I recently upgraded my puppet masters (and clients) from 0.24.8 to
> > > > 2.6.4
> > >
> > > > Previously, my most busy puppet master would hover around about 0.9
> > > > load  average, after the upgrade, its load hovers around 5
> > >
> > > > I am running passenger and mysql based stored configs.
> > >
> > > > Checking my running processes, ruby (puppetmasterd) shoots up to 99%
> > > > cpu load and stays there for a few seconds before dropping again.
> > > > Often there are 4 of these running simultaneously, pegging each core
> > > > at 99% cpu.
> > >
> > > I would say it is perfectly normal. Compiling the catalog is a hard and
> > > complex problem and requires CPU.
> > >
> > > The difference between 0.24.8 and 2.6 (or 0.25 for what matters) is
> that
> > > some performance issues have been fixed. Those issues made the master
> be
> > > more I/O bound under 0.24, but now mostly CPU bound in later versions.
> >
> > If we were talking about only cpu usage, I would agree with you.  But
> > in this case, the load average of the machine has gone up over 5x.
> > And as high load average indicates processes not getting enough
> > runtime, in this case it is an indication to me that 2.6 is performing
> > worse than 0.24 (previously, on average, all processes got enough
> > runtime and did not have to wait for system resources, now processes
> > are sitting in the run queue, waiting to get a chance to run)
>
> Load is not necessarily an indication of an issue. It can also mean some
> tasks are waiting for I/O not only CPU.
> The only real issue under load is if service time is beyond an
> admissible value, otherwise you can't say it's bad or not.
> If you see some hosts reporting timeouts, then it's an indication that
> service time is not good :)
>
> BTW, do you run your mysql storedconfig instance on the same server?
> You can activate thin_storeconfigs to reduce the load on the mysql db.
>
> > >
> > > I don't really get what is the issue about using 100% of CPU?
> > Thats not the issue, just an indication of what is causing it
> >
> > >
> > > You're paying about the same price when your CPU is used and when it's
> > > idle, so that shouldn't make a difference :)
> > Generally true, but this is a on VM which is also running some of my
> > radius and proxy instances, amongst others.
> >
> > >
> > > If that's an issue, reduce the concurrency of your setup (run less
> > > compilation in parallel, implement splay time, etc...).
> > splay has been enabled since 0.24
> >
> > My apache maxclients is set to 15 to limit concurrency.
>
> I think this is too many except if you have 8 cores. As Trevor said in
> another e-mail in this thread, 2PM/Core is the best.
>
> Now it all depends on your number of nodes and sleeptime. I suggest you
> use ext/puppet-load to find your setup real concurrency.
> --
> Brice Figureau
> Follow the latest Puppet Community evolutions on www.planetpuppet.org!
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-us...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] source /etc/profile

[Adrian Tofan]

Here is what I managed to do:

I was forced to replace package {"buildr":ensure=>installed} with :

$java_home = "/usr/lib/jvm/java-1.5.0-sun"

exec { install_buildr:  
command => "echo 'export JAVA_HOME=$java_home;gem install -y 
buildr'|/bin/bash",
path => "/usr/local/bin:/usr/bin:/usr/sbin:/bin",
require => Package["sun-java5-jdk"], 
unless => "gem list buildr|grep buildr"
}

1. it may be possible to initialize $java_home  from /etc/profile but I guess 
that is not that important
2. I am not sure how it will handle errors as the commands are piped to bash

Thank you all for the suggestions. 

>> 
>> 
>> 
>> There is a way to source /etc/profile inside puppet on demand?
>> 
>> My problem is that I install Java, afterwards I set JAVA_HOME variable
>> in /etc/profile.
>> 
>> After Java is installed, a gem which depends on Java starts to
>> install ... but as it needs JAVA_HOME environment to be set, it fails.
>> 
>> I am not sure what can I do ...


-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: puppetmaster 100%cpu usage on 2.6 (not on 0.24)

[Brice Figureau]

On Wed, 2010-12-15 at 05:28 -0800, Chris wrote:
> 
> On Dec 15, 12:42 pm, Brice Figureau 
> wrote:
> > On Tue, 2010-12-14 at 00:24 -0800, Chris wrote:
> > > Hi
> >
> > > I recently upgraded my puppet masters (and clients) from 0.24.8 to
> > > 2.6.4
> >
> > > Previously, my most busy puppet master would hover around about 0.9
> > > load  average, after the upgrade, its load hovers around 5
> >
> > > I am running passenger and mysql based stored configs.
> >
> > > Checking my running processes, ruby (puppetmasterd) shoots up to 99%
> > > cpu load and stays there for a few seconds before dropping again.
> > > Often there are 4 of these running simultaneously, pegging each core
> > > at 99% cpu.
> >
> > I would say it is perfectly normal. Compiling the catalog is a hard and
> > complex problem and requires CPU.
> >
> > The difference between 0.24.8 and 2.6 (or 0.25 for what matters) is that
> > some performance issues have been fixed. Those issues made the master be
> > more I/O bound under 0.24, but now mostly CPU bound in later versions.
> 
> If we were talking about only cpu usage, I would agree with you.  But
> in this case, the load average of the machine has gone up over 5x.
> And as high load average indicates processes not getting enough
> runtime, in this case it is an indication to me that 2.6 is performing
> worse than 0.24 (previously, on average, all processes got enough
> runtime and did not have to wait for system resources, now processes
> are sitting in the run queue, waiting to get a chance to run)

Load is not necessarily an indication of an issue. It can also mean some
tasks are waiting for I/O not only CPU. 
The only real issue under load is if service time is beyond an
admissible value, otherwise you can't say it's bad or not.
If you see some hosts reporting timeouts, then it's an indication that
service time is not good :)

BTW, do you run your mysql storedconfig instance on the same server?
You can activate thin_storeconfigs to reduce the load on the mysql db.

> >
> > I don't really get what is the issue about using 100% of CPU?
> Thats not the issue, just an indication of what is causing it
> 
> >
> > You're paying about the same price when your CPU is used and when it's
> > idle, so that shouldn't make a difference :)
> Generally true, but this is a on VM which is also running some of my
> radius and proxy instances, amongst others.
> 
> >
> > If that's an issue, reduce the concurrency of your setup (run less
> > compilation in parallel, implement splay time, etc...).
> splay has been enabled since 0.24
> 
> My apache maxclients is set to 15 to limit concurrency.

I think this is too many except if you have 8 cores. As Trevor said in
another e-mail in this thread, 2PM/Core is the best.

Now it all depends on your number of nodes and sleeptime. I suggest you
use ext/puppet-load to find your setup real concurrency.
-- 
Brice Figureau
Follow the latest Puppet Community evolutions on www.planetpuppet.org!

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] ANNOUNCE: The Marionette Collective version 1.0.0

[R.I.Pienaar]

Hello,

We am pleased to announce the release of The Marionette Collective version 
1.0.0.

This is a bug fix and minor feature release.

New Features:

 * You can now send RPC requests to a subset of discovered nodes
 * SimpleRPC custom_request can now be used to create fire and forget requests
 * Clients can now cleanly disconnect from the middleware. Bundled clients have 
   been updated. This should cause fewer exceptions in ActiveMQ logs
 * Rather than big exceptions many clients will now log errors only
 * mc-facts has been reworked to be a SimpleRPC client, this speeds it up 
significantly
 * Add get_config_item to rpcutil to retrieve a running config value from a 
server
 * YAML facts are now forced to be all strings and is thread safe
 * On Red Hat based systems the requirement for the LSB packages has been 
removed

If you expose a service redundantly on your network using MCollective you 
wouldn’t 
always want to send requests to all the nodes providing the service. You can 
now 
limit the requests to an arbitrary amount using the new –limit-nodes option 
which 
will also take a percentage. A shortcut -1 has been added that is the 
equivalent 
to –limit-nodes 1.  See the documentation for more details.[2]

Bug Fixes:

 * Settings like retry times were ignored in the Stomp connector
 * The default init script had incorrect LSB comments
 * The rpcutil DDL has better validation and will now match all facts

Please see the release notes for full details[1].

This release can be downloaded from the Puppet Labs site.[3]

Other News:

The downloads has been moved to the Puppet Labs site and added to their hosted 
Yum and
Apt repositories at http://yum.puppetlabs.com and http://apt.puppetlabs.com/

We've taken previous feedback from the community on versioning and release 
management
and have the following changes to release process planned:

This is the first in a new major version number - 1.0.x - we will maintain 1.0.x
as a stable branch that do not get new features.  We do not have a set schedule
for releases in this branch it will depend on what bug fixes go into it.

Within a few weeks there will also be the first release in 1.1.x series, this is
where new development will be done and should be used by early adopters.  We 
will
at all times aim to keep these releases as production ready as possible but they
will get some technology previews and perhaps a few changes that are not 
backwards
compatible.  Releases in this series should be roughly monthly.

The work for the 1.1.x series is shown on the Roadmap page[4], we welcome any 
new feature
requests.

[1] http://srt.ly/mcollective100
[2] http://srt.ly/62
[3] http://srt.ly/4y
[4] http://srt.ly/63

-- 
R.I.Pienaar

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] source /etc/profile

[Richard Crowley]

On Tue, Dec 14, 2010 at 9:42 AM, Adrian  wrote:
> Hello ,
>
> There is a way to source /etc/profile inside puppet on demand?
>
> My problem is that I install Java, afterwards I set JAVA_HOME variable
> in /etc/profile.
>
> After Java is installed, a gem which depends on Java starts to
> install ... but as it needs JAVA_HOME environment to be set, it fails.
>
> I am not sure what can I do ...

Make a fake new puppetd that does what you want:

$ cat /usr/local/bin/puppetd
#!/bin/sh
set -e

export JAVA_HOME=whatever
# OR
. /etc/profile

exec /usr/bin/puppetd "$@"

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: source /etc/profile

[Felix Frank]

> Next, what I did is to send SIGUSR1 to puppet after updating /etc/profile 
> with something like this 
>   
> exec { "refresh_profile" :
>   command => "/usr/bin/killall -SIGUSR1 puppetd",
>   onlyif => "pidof -x puppetd",
>   path => "/bin:/usr/bin",
>   user => "root",
>   refreshonly => true,
>   }

No good. Signaling puppet won't initialize your environment anew. You
would have to make the shell that runs puppetd source the profile, then
probably start puppet from scratch. Note that puppetd can be started by
init and not run in a shell at all.

Using puppet to initialize its own environment is tricky business at
best. As such, I found Ken's approach quite interesting.

Regards,
Felix

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Prompting fact refresh from Puppet Event

[Felix Frank]

On 12/15/2010 03:46 PM, Trevor Vaughan wrote:
> Is is possible to spawn a puppet fact refresh from the completion of an event?
> 
> Here's the scenario:
> 
> 64-bit system, either a 64 or 32 bit application 'foo' could be installed.
> 
> If the 64 bit version is installed, foo.conf needs to be configured
> with a prepended lib64 path. If not, then the 32bit default is fine.
> 
> The only thing that I could come up with is a custom fact that returns
> the architecture of the installed package and then adjust the path
> based on that.
> 
> However, this takes *two* puppet runs to complete and I would like to
> get this down to one run by prompting a fact refresh after the package
> is installed.
> 
> Does anyone have any suggestions/techniques?

Hrm, your manifest should be able to handle this one on its own.

Assuming your foo.conf file is part of the foo::config class, add this
subclass:
foo::config::arch64bit {
  File["/etc/foo.conf"] { source => "..." }
}

Then, at the place in the manifest where you pick the 64bit version of
the foo application, add an "include foo::config::arch64bit".

Will this work for your manifest?

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Prompting fact refresh from Puppet Event

[Trevor Vaughan]

Is is possible to spawn a puppet fact refresh from the completion of an event?

Here's the scenario:

64-bit system, either a 64 or 32 bit application 'foo' could be installed.

If the 64 bit version is installed, foo.conf needs to be configured
with a prepended lib64 path. If not, then the 32bit default is fine.

The only thing that I could come up with is a custom fact that returns
the architecture of the installed package and then adjust the path
based on that.

However, this takes *two* puppet runs to complete and I would like to
get this down to one run by prompting a fact refresh after the package
is installed.

Does anyone have any suggestions/techniques?

Thanks,

Trevor

-- 
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvaug...@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Re: error in execution of script file

[jcbollinger]

On Dec 15, 12:29 am, "sanjiv.singh" 
wrote:
> when i tried to fire puppet as
>
> $ sudo puppetd  --test --debug
>
> , then  "/usr/local/runprocess.sh"  has some permission issues.
> some command in this file violate to execute  due to permission error.
>
> , but i tried to execute this shelll script from terminal..as
>
> $ sudo /usr/local/runprocess.sh
>
> , then all command  in shell file executed sucessfully .
>
> what could be the reason?
> , i didnt get  the reason , why permission not granted ?

In all likelihood, the problem is related to the details of the script
you are trying to run.

Be especially wary of relying on environment variables that your
script does not set itself, for Puppet intentionally provides a rather
sparse evironment to subprocesses that it execs.  Bash does not
automatically read system or user shell initialization files when it
is started non-interactively (such as when invoked to run a script),
but it does inherit environment variables exported by the environment
from which it is launched.  Therefore, your script has a rather richer
environment when launched from an interactive shell than when launched
by Puppet.


John

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Re: error in execution of script file

[jcbollinger]

On Dec 15, 12:57 am, "sanjiv.singh" 
wrote:
> > Try putting just this in a file called test.pp.  Then try this:
> > puppet --verbose --debug test.pp
>
> thanks patrick for quick reply
>
> i didnt get U, what U tring to say...
> how this is going to help us by placing code from nodeUpdate.pp class
> to test.pp class.
>
> can U explain in more details.?

You are missing the most important points: the test.pp file goes on
the host on which it will be applied, not on the puppetmaster (if
that's different), and you apply it via the "puppet" program instead
of via "puppetd".  The name of the file has no special significance.


John

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: puppetmaster 100%cpu usage on 2.6 (not on 0.24)

[Trevor Vaughan]

What is your CPU to Puppetmaster instance ratio?

I've had decent luck with 1 CPU to 2 PM, but not much above that.

If you need dedicated resources for other tasks, you may want to
ensure that you don't have as many masters spawning as you have
processors.

Trevor

On Wed, Dec 15, 2010 at 8:28 AM, Chris
 wrote:
>
>
> On Dec 15, 12:42 pm, Brice Figureau 
> wrote:
>> On Tue, 2010-12-14 at 00:24 -0800, Chris wrote:
>> > Hi
>>
>> > I recently upgraded my puppet masters (and clients) from 0.24.8 to
>> > 2.6.4
>>
>> > Previously, my most busy puppet master would hover around about 0.9
>> > load  average, after the upgrade, its load hovers around 5
>>
>> > I am running passenger and mysql based stored configs.
>>
>> > Checking my running processes, ruby (puppetmasterd) shoots up to 99%
>> > cpu load and stays there for a few seconds before dropping again.
>> > Often there are 4 of these running simultaneously, pegging each core
>> > at 99% cpu.
>>
>> I would say it is perfectly normal. Compiling the catalog is a hard and
>> complex problem and requires CPU.
>>
>> The difference between 0.24.8 and 2.6 (or 0.25 for what matters) is that
>> some performance issues have been fixed. Those issues made the master be
>> more I/O bound under 0.24, but now mostly CPU bound in later versions.
>
> If we were talking about only cpu usage, I would agree with you.  But
> in this case, the load average of the machine has gone up over 5x.
> And as high load average indicates processes not getting enough
> runtime, in this case it is an indication to me that 2.6 is performing
> worse than 0.24 (previously, on average, all processes got enough
> runtime and did not have to wait for system resources, now processes
> are sitting in the run queue, waiting to get a chance to run)
>
>>
>> I don't really get what is the issue about using 100% of CPU?
> Thats not the issue, just an indication of what is causing it
>
>>
>> You're paying about the same price when your CPU is used and when it's
>> idle, so that shouldn't make a difference :)
> Generally true, but this is a on VM which is also running some of my
> radius and proxy instances, amongst others.
>
>>
>> If that's an issue, reduce the concurrency of your setup (run less
>> compilation in parallel, implement splay time, etc...).
> splay has been enabled since 0.24
>
> My apache maxclients is set to 15 to limit concurrency.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Puppet Users" group.
> To post to this group, send email to puppet-us...@googlegroups.com.
> To unsubscribe from this group, send email to 
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/puppet-users?hl=en.
>
>



-- 
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvaug...@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Re: Separating puppetmaster file serving and catalogs

[jcbollinger]

On Dec 15, 5:04 am, Patrick  wrote:
> I'm looking for a way to run more than one puppetmaster on the same server 
> under passenger.  Most of the puppet CPU load is waiting for the catalogs to 
> compile.  This also seems to be mostly what takes large amounts of RAM.  I 
> have storedconfigs on.
>
> I want to be able to move the fileserver to a different pool of puppetmaster 
> processes.  Is there an easy way to tell the client, either in the catalog or 
> in the config file, to get the files from a different port than the catalog?

You can write source => "puppet:///..." instead of
source => "puppet:///...".  If  resolves to the
same physical machine then apache can direct it to a different virtual
host.  (And if it resolves to a different physical machine then no
worries on that level.)  I'm not sure, however, whether you can run
separate copies of Passenger in different vhosts.  (But if not, then
it would be a desirable feature.)

> Is there a better way to do this?  What I really want is for the cheap file 
> requests to stop being blocked by the expensive catalog requests and keep the 
> RAM usage low on the file serving processes.

I don't know whether it's better, but since you're serving through
Apache anyway, you could serve your files directly via http.  That has
implications on where you put said files on disk and on both client
and file system security management, but it ought to be very fast, and
it will scale as Apache itself does.


John

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: source /etc/profile

[Adrian Tofan]

Thank you for an interesting idea ,

> Hi Adrian,
> 
> I'm not sure but I imagine by just trying to 'source' the profile within the 
> current puppet runtime the scope will be lost as it would be called within an 
> exec? 
> 
> Obviously if you are not worried about running puppet multiple times you 
> might be able to source it in your 'prerun_command' script

I didn't manage to convince puppet to source the profile...

I've tried to add in [agent] section

prerun_command=/root/somecommand 

but the command was not executed and I didn't see any log about it. 
 
I wonder if 

prerun_command = echo 'source /etc/profile'|/bin/bash

is suposed to work, because for the moment it is not. 

Next, what I did is to send SIGUSR1 to puppet after updating /etc/profile with 
something like this 
  
exec { "refresh_profile" :
  command => "/usr/bin/killall -SIGUSR1 puppetd",
  onlyif => "pidof -x puppetd",
  path => "/bin:/usr/bin",
  user => "root",
  refreshonly => true,
  }


> Otherwise I have one possible (albeit slightly hacky) option. Perhaps you can 
> drop to ruby to ensure you set the variable JAVA_HOME within the puppet 
> runtime. Let me give you an example I set-up on my machine:
> /tmp/var.sh:
> #!/bin/bash
> echo $JAVA_HOME > /tmp/java_path
> test.pp:
> $dummy = inline_template("<% ENV['JAVA_HOME'] = '/opt/java_1.6.0_20' %>")
> exec {"a":
> command => "/tmp/var.sh",
> }


My problem is that I do not use exec. What I do is something like this : 

package { "buildr": 
   ensure => installed,
   provider => "gem",
   require => Package["sun-java5-jdk"],
}

When it tries to install the gem it needs the JAVA_HOME. Apparently if 
Java_HOME was set before running puppetd, then Java_Home is available to 
installer.

I am not sure what I can do next ... 

> notice(inline_template("<%= ENV['JAVA_HOME'] %>"))
> 
> And to run it:
> 
> $ puppet test.pp 
> notice: Scope(Class[main]): /opt/java_1.6.0_20
> notice: /Stage[main]//Exec[a]/returns: executed successfully
> $ cat /tmp/java_path
> /opt/java_1.6.0_20
> $ 
> 
> This shows that the environment variable is available at the scope when ruby 
> does an exec (the exec resource) and when you try to grab the variable inline 
> later on. Hopefully the gem provider should see this, but you'll have to make 
> sure the inline_template runs before the gem does its work. Putting it in a 
> class (like your "java" puppet class or something) and doing a require on the 
> class may solve this.
> 
> Of course there is probably a nicer less complicated way of doing this :-).

This would be helpful - even if I don't totaly understand how it works - only 
if I can grab Java_Home from /etc/profile and assign it to 

$dummy = inline_template("<% ENV['JAVA_HOME'] = 'JAVA_HOME_FROM_PROFILE' %>")


-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Re: puppetmaster 100%cpu usage on 2.6 (not on 0.24)

[Chris]

On Dec 15, 12:42 pm, Brice Figureau 
wrote:
> On Tue, 2010-12-14 at 00:24 -0800, Chris wrote:
> > Hi
>
> > I recently upgraded my puppet masters (and clients) from 0.24.8 to
> > 2.6.4
>
> > Previously, my most busy puppet master would hover around about 0.9
> > load  average, after the upgrade, its load hovers around 5
>
> > I am running passenger and mysql based stored configs.
>
> > Checking my running processes, ruby (puppetmasterd) shoots up to 99%
> > cpu load and stays there for a few seconds before dropping again.
> > Often there are 4 of these running simultaneously, pegging each core
> > at 99% cpu.
>
> I would say it is perfectly normal. Compiling the catalog is a hard and
> complex problem and requires CPU.
>
> The difference between 0.24.8 and 2.6 (or 0.25 for what matters) is that
> some performance issues have been fixed. Those issues made the master be
> more I/O bound under 0.24, but now mostly CPU bound in later versions.

If we were talking about only cpu usage, I would agree with you.  But
in this case, the load average of the machine has gone up over 5x.
And as high load average indicates processes not getting enough
runtime, in this case it is an indication to me that 2.6 is performing
worse than 0.24 (previously, on average, all processes got enough
runtime and did not have to wait for system resources, now processes
are sitting in the run queue, waiting to get a chance to run)

>
> I don't really get what is the issue about using 100% of CPU?
Thats not the issue, just an indication of what is causing it

>
> You're paying about the same price when your CPU is used and when it's
> idle, so that shouldn't make a difference :)
Generally true, but this is a on VM which is also running some of my
radius and proxy instances, amongst others.

>
> If that's an issue, reduce the concurrency of your setup (run less
> compilation in parallel, implement splay time, etc...).
splay has been enabled since 0.24

My apache maxclients is set to 15 to limit concurrency.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] having trouble with puppet 0.25.5 on openbsd 4.8 on amd64

[Daniel Pittman]

On Wed, Dec 15, 2010 at 18:23, Eric Sorenson  wrote:
> On Dec 14, 2010, at 11:04 PM, Daniel Pittman wrote:
>
>> I think you will then run into the issue that $name can't be expanded
>> at that point, though, which would require writing an extra define to
>> make it available. :(
>
> I really thought that would work too and mocked it up but apparently it ain't 
> so. I tried '$name' '$namevar' '$title' to no avail.
>
> https://projects.puppetlabs.com/issues/5259 seemed germane so I updated it 
> with this discussion.

Sad but true - and thanks for updating the bug report.  It helps
communicate what is hurting folks.

You can "fix" this like so:

define foo () {
  package { $name: source => "http:/.../$name" }
}

class bar {
  foo { ["one", "two"]: }
}

Not as nice, but at least you can do the array expansion stuff - and
where you needs lots of similar parameters it makes some sense to do
it that way.

Regards,
Daniel
-- 
✣ Daniel Pittman            ✉ dan...@rimspace.net            ☎ +61 401 155 707
              ♽ made with 100 percent post-consumer electrons

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: File type: no success using array

[Arnau Bria]

On Wed, 15 Dec 2010 11:04:27 +0100
Peter Meier wrote:

Hi Pete,

> >>  file {  [ '/software/atlas' , '/software/cms' , '/software/lhcb' ,
> >> '/software/magic' , '/software/at3' , '/software/paus' ,
> >> '/software/mice' , '/software/ops.vo.ibergrid.eu' ,
> >> '/software/supernemo.vo.eu-egee.org' ]: require=>
> >> File['/software'] ; }
> 
> not that you don't need this require, as puppet automagically orders
> file resources according to their path. So File['/software/atlas']
> automatically depends on File['/software']
thanks! didn't know.
 
> ~pete
Cheers,
Arnau

PS: I love automagically word!

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Separating puppetmaster file serving and catalogs

[Felix Frank]

On 12/15/2010 12:04 PM, Patrick wrote:
> I'm looking for a way to run more than one puppetmaster on the same server 
> under passenger.  Most of the puppet CPU load is waiting for the catalogs to 
> compile.  This also seems to be mostly what takes large amounts of RAM.  I 
> have storedconfigs on.
> 
> I want to be able to move the fileserver to a different pool of puppetmaster 
> processes.  Is there an easy way to tell the client, either in the catalog or 
> in the config file, to get the files from a different port than the catalog?
> 
> Is there a better way to do this?  What I really want is for the cheap file 
> requests to stop being blocked by the expensive catalog requests and keep the 
> RAM usage low on the file serving processes.
> 

Hi,

what springs to mind is a webserver with mod_proxy up front (or in fact,
any intelligent reverse proxy), that chooses your actual webserver with
respect to request URIs.

Fileserver requests do go to a different root directory, yes?

HTH,
Felix

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Separating puppetmaster file serving and catalogs

[Patrick]

I'm looking for a way to run more than one puppetmaster on the same server 
under passenger.  Most of the puppet CPU load is waiting for the catalogs to 
compile.  This also seems to be mostly what takes large amounts of RAM.  I have 
storedconfigs on.

I want to be able to move the fileserver to a different pool of puppetmaster 
processes.  Is there an easy way to tell the client, either in the catalog or 
in the config file, to get the files from a different port than the catalog?

Is there a better way to do this?  What I really want is for the cheap file 
requests to stop being blocked by the expensive catalog requests and keep the 
RAM usage low on the file serving processes.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: n00b questions - verbosity of config????

[Felix Frank]

> #Copy a file from the Puppet master.
> #
> #A good idea sourced from http://narrabilis.com/mybook/puppet/functions.pp
> #Basically it's a little wrapper for the file type that's used to copy
> #a file from the puppet master. 'module' and 'path' are the only
> required
> #parameters.
> #
> #PARAMETERS:
> #  - pathThe path to the source file to copy as relative to
> modules//files/.
> #  - module  The name of the module to copy the file from.
> #  - modeParameter passed directly to File type.
> #  - owner   Parameter passed directly to File type.
> #  - group   Parameter passed directly to File type.
> #  - ensure  Parameter passed directly to File type.
> define remotefile($path, $module, $owner = root, $group = root, $mode
> = 0644, $ensure = present) {
>   file { $name:
> ensure  => $ensure,
> mode=> $mode,
> owner   => $owner,
> group   => $group,
> source  => "puppet:///modules/$module/$path",
>   }
> }
> 

This compells me to elaborate a bit more. The problem with the above is
that you cannot have a remotefile that doesn't care about the mode
(i.e., let puppet accept whatever mode it finds on the client machine).

If you just speicfy
remotefile { "/etc/motd": module => "all" }
the mode will be forced to 0644, which may not be what you want in edge
cases.

This is the pattern we use for this problem:

define remotefile($mode = "") {
  if $mode != "" { File { mode => $mode } }
  file { $name:
source => "puppet:///.../$name"
  }
}

Actually, our remotefile has similar support for about *all* parameters
that "file" supports, except that owner and group indeed default to root
(you would not want to not set those explicitly).

Also note that the "path" parameter is optional in our implementation.
Normally the URL uses $name, so if you need to override that behaviour,
you can fall back to specifying the path. This is handled by if-else
logic in the define.

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] puppetmaster 100%cpu usage on 2.6 (not on 0.24)

[Brice Figureau]

On Tue, 2010-12-14 at 00:24 -0800, Chris wrote:
> Hi
> 
> I recently upgraded my puppet masters (and clients) from 0.24.8 to
> 2.6.4
> 
> Previously, my most busy puppet master would hover around about 0.9
> load  average, after the upgrade, its load hovers around 5
> 
> I am running passenger and mysql based stored configs.
> 
> Checking my running processes, ruby (puppetmasterd) shoots up to 99%
> cpu load and stays there for a few seconds before dropping again.
> Often there are 4 of these running simultaneously, pegging each core
> at 99% cpu.

I would say it is perfectly normal. Compiling the catalog is a hard and
complex problem and requires CPU. 

The difference between 0.24.8 and 2.6 (or 0.25 for what matters) is that
some performance issues have been fixed. Those issues made the master be
more I/O bound under 0.24, but now mostly CPU bound in later versions.

Now compare the compilation time under 0.24.8 and 2.6 and you should see
that it reduced drastically (allowing to fit more compilation in the
same time unit). The reverse of the medal is that now your master
requires transient high CPU usage.

I don't really get what is the issue about using 100% of CPU?

You're paying about the same price when your CPU is used and when it's
idle, so that shouldn't make a difference :)

If that's an issue, reduce the concurrency of your setup (run less
compilation in parallel, implement splay time, etc...).

> It seems that there has been a serious performance regression between
> 0.24 and 2.6 for my configuration

I think it's the reverse that happened.

> I hop the following can help work out where...
> 
> I ran puppetmasterd through a profiler to find the root cause of this
> (http://boojum.homelinux.org/profile.svg).  The main problem appears
> to be in /usr/lib/ruby/site_ruby/1.8/puppet/parser/ast/resource.rb, in
> the evaluate function.
> 
> I added a few timing commands around various sections of that function
> to find the following breakdown of times spent inside it, and the two
> most intensive calls are
> ---
> paramobjects = parameters.collect { |param|
>   param.safeevaluate(scope)
> }
> ---
> 
> and
> ---
> resource_titles.flatten.collect { |resource_title|
>   exceptwrap :type => Puppet::ParseError do
> resource = Puppet::Parser::Resource.new(
>   fully_qualified_type, resource_title,
>   :parameters => paramobjects,
>   :file => self.file,
>   :line => self.line,
>   :exported => self.exported,
>   :virtual => virt,
>   :source => scope.source,
>   :scope => scope,
>   :strict => true
> )
> 
> if resource.resource_type.is_a? Puppet::Resource::Type
>   resource.resource_type.instantiate_resource(scope, resource)
> end
> scope.compiler.add_resource(scope, resource)
> scope.compiler.evaluate_classes([resource_title],scope,false)
> if fully_qualified_type == 'class'
> resource
>   end
> }.reject { |resource| resource.nil? }
> ---

Yes, this is what the compiler is doing during compilation: evaluating
resources and parameters. The more resources you use, the more the
compilation will take time and CPU.
-- 
Brice Figureau
Follow the latest Puppet Community evolutions on www.planetpuppet.org!

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: File type: no success using array

[Peter Meier]

On 12/15/2010 10:48 AM, Arnau Bria wrote:
> On Wed, 15 Dec 2010 10:44:55 +0100
> Arnau Bria wrote:
> 
>> Hi all,
> Hi again,
> 
> I said nothing.
>  
>>  file {  [ '/software/atlas' , '/software/cms' , '/software/lhcb' ,
>> '/software/magic' , '/software/at3' , '/software/paus' ,
>> '/software/mice' , '/software/ops.vo.ibergrid.eu' ,
>> '/software/supernemo.vo.eu-egee.org' ]: require=>
>> File['/software'] ; }

not that you don't need this require, as puppet automagically orders
file resources according to their path. So File['/software/atlas']
automatically depends on File['/software']

~pete

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Re: Converting to Ruby DSL

[Ross Harvey]

I can solve 5 for you.  The resource reference syntax is apparently a
magic String object of some sort on its own, and not something that
was being parsed by the original DSL.

So, you simply need to write:

  :require => 'File[test123]'

In this case I used the path parameter; test123 is just a resource
name.

On Dec 5, 4:37 pm, Ken Barber  wrote:
> Hi everyone,
>
> I'm experimenting with converting some of my live puppet content to
> Ruby DSL and have found a few gaps I wouldn't mind some advice on. I'm
> not sure how many of you have already experimented in this arena yet.
>
> I've already read Dan Bode's excellent blog article on the subject:
>
> http://www.puppetlabs.com/blog/ruby-dsl/
>
> I know the Ruby DSL is quite new - but I figured perhaps some of these
> items deserve to be tickets (or perhaps documented) and I thought I'd
> ask first :-).
>
> 1. Defaults
>
> How do I set defaults in Ruby DSL? For example what is the ruby DSL
> equivalent to this:
>
>   Service {
>     hasstatus => true
>   }
>
> 2. Top scope
>
> I notice the convenience methodology does not work at 'top scope' for
> example
> this will fail:
>
>   file "/tmp/zzz", :content => "foo"
>
> But this will not:
>
>   node "default" do
>     file "/tmp/zzz", :content => "foo"
>   end
>
> Is there a proper way to call resources without using the convenience
> methods? Or
> perhaps is there a way to define elements that reside in top scope?
>
> 3. Class resource
>
> I think Dan Bode already raised this as a bug:
>
> https://projects.puppetlabs.com/issues/5236
>
> The following returns an error 'method_missing':
>
>   hostclass :foo do
>     notify "bar"
>   end
>   node "default" do
>     hostclass "foo", :stage => "baz"
>   end
>
> This is because hostclass has not been defined as a convenience method
> it would seem. Just like issues 2 If someone knows
> a way to call a resource directly without requiring the convenience
> method that would be a decent enough work-around.
>
> 4. exec Resource
>
> The 'exec' resource as a convenience method is overlapping with the
> ruby 'exec'
> method. So if you do this:
>
>   node "default" do
>     exec "ls", :command => "/bin/ls"
>   end
>
> You get:
>
>   can't convert Hash into String on node obelisk.usr.bob.sh
>
> Returned from the Kernel.exec call.
>
> 5. Referencing other resources
>
> I'm not quite clear how to reference other resources that already
> exist. This is obviously a problem for the 'require' attribute:
>
>   node "default" do
>     package "foo", :ensure => "installed"
>     service "foo", :enable => "true", :ensure => "running", :require
> => ???
>   end
>
> 6. Chaining Resources
>
> Is there a syntactical equivalent to this in Ruby DSL yet? For
> example:
>
>   Stage[pre] -> Stage[main] -> Stage[post]
>
> Without knowing how to solve 5 I'm kind of stuck when it comes to
> defining dependencies :-).
>
> Cheers :-).
>
> ken.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] European Master Puppet Training Brussels/Belgium

[Mario Verbelen]

Hello,

Open-Future and Reductive Labs are organizing an official Puppet
Master training in Brussels.

This training is ideal for those who want a Puppet jumpstart. Newer
members at an organization already using Puppet, or experienced
sysadmins wanting to bring Puppet into their team will get everything
they need to deploy solutions.
Prerequisites:

Attendees should have at least the equivalent experience of a junior
Unix/Linux administrator.
Topics covered include:

* Configuring Puppet and Puppetmaster
* Resource Types and the Resource Abstration Layer
* Virtual Resources, Exported Resources and Stored Configs
* Meta-parameters, Dependencies and Events
* Classes, Modules and Definitions
* Tags and Environments
* Puppet Language Patterns and Best Practices

The topics are covered over 3 days. Sessions will mix theory and
practice, balancing lectures with hands-on exercises. (Each student
should bring a WiFi enabled laptop with VMWare installed to
participate in the labs).

You can subscribe via our site 
http://www.open-future.be/european-puppet-training

Best regards,

Mario Verbelen
Open-Future

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] n00b questions - verbosity of config????

[Martijn Grendelman]

Hi,

I haven't followed the entire discussion, but one small thing caught my
attention:

On 14-12-10 19:15, Rameses Mss wrote:
> In your base node (i.e., top level inheritance) set a default:
> 
> node base {
> $sshd_type="client"
> include sshd
> }
> 
> node myclient inherits base {
> }
> 
> node myserver inherits base {
> $sshd_type="server"
> }
> 
> This would require sshd_config.client. Override it on a per-node basis.
> External nodes scripts make this sort of thing simpler.

AFAIK, it doesn't work if you write it like this. Please correct me if I
am wrong.

I have tried this, and spent the better part of a day trying to find out
why $ssh_type didn't have "server" on 'myserver', but "client". The parse
order matters in this case, and it only works if you write it like this:

node base {
$sshd_type="client"
}

node myclient inherits base {
include sshd
}

node myserver inherits base {
$sshd_type="server"
include sshd
}

The obvious downside being, that you have to write the include-line for
each node separately.

Best regards,
Martijn.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Re: File type: no success using array

[Arnau Bria]

On Wed, 15 Dec 2010 10:44:55 +0100
Arnau Bria wrote:

> Hi all,
Hi again,

I said nothing.
 
>  file {  [ '/software/atlas' , '/software/cms' , '/software/lhcb' ,
> '/software/magic' , '/software/at3' , '/software/paus' ,
> '/software/mice' , '/software/ops.vo.ibergrid.eu' ,
> '/software/supernemo.vo.eu-egee.org' ]: require=>
> File['/software'] ; }

the error was in the next line, in other array.


sorry for the noise.
Cheers,
Arnau

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] File type: no success using array

[Arnau Bria]

Hi all,

from http://docs.puppetlabs.com/guides/language_tutorial.html this
syntax should be ok:

file { [ 'foo', 'bar', 'foobar' ]:
owner => root,
group => root,
mode  => 600,
}

I've this in my code:

 file {  [ '/software/atlas' , '/software/cms' , '/software/lhcb' , 
'/software/magic' , '/software/at3' , '/software/paus' , '/software/mice' , 
'/software/ops.vo.ibergrid.eu' , '/software/supernemo.vo.eu-egee.org' ]:
 require=> File['/software'] ;
}

And an error like:

err: Could not parse for environment production: Syntax error at ':'; expected 
'}' at /etc/pup...

I'm on 2.6.1, is it a bug? Anyone is using this syntax with no problems?

Cheers,
Arnau   

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] Re: n00b questions - verbosity of config????

[luke.bigum]

> I believe most people are bound to write this define sooner or later:
>
> define my_file() {
>   file { "$name":
>     source => "puppet:///files/$name",
>
> }
>
> It's a bit trickier to add support for all of file's options, but once
> that's in, it's a huge advantage.

I use a similar define I shamelessly copied from someone else. Pasted
below in case it benefits others:

#Copy a file from the Puppet master.
#
#A good idea sourced from http://narrabilis.com/mybook/puppet/functions.pp
#Basically it's a little wrapper for the file type that's used to copy
#a file from the puppet master. 'module' and 'path' are the only
required
#parameters.
#
#PARAMETERS:
#  - pathThe path to the source file to copy as relative to
modules//files/.
#  - module  The name of the module to copy the file from.
#  - modeParameter passed directly to File type.
#  - owner   Parameter passed directly to File type.
#  - group   Parameter passed directly to File type.
#  - ensure  Parameter passed directly to File type.
define remotefile($path, $module, $owner = root, $group = root, $mode
= 0644, $ensure = present) {
  file { $name:
ensure  => $ensure,
mode=> $mode,
owner   => $owner,
group   => $group,
source  => "puppet:///modules/$module/$path",
  }
}

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] Re: error in execution of script file

[Patrick]

On Dec 14, 2010, at 10:57 PM, sanjiv.singh wrote:

> 
> 
> 
>> Try putting just this in a file called test.pp.  Then try this:
>> puppet --verbose --debug test.pp
> 
> thanks patrick for quick reply
> 
> i didnt get U, what U tring to say...
> how this is going to help us by placing code from nodeUpdate.pp class
> to test.pp class.
> 
> can U explain in more details.?

Sure.  This will remove the puppetmaster and the puppetclient from this 
problem.  That means there is less code it could be going wrong in.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.