[Puppet Users] Re: puppet 3.4.0 upgrade, foreman 1.1-stable, problems that are not provisioning related
*facepalm*
For anyone else who encounters a problem like this and, like me, you are
not a puppet administrator the first thing you should do is check the
puppet master config (probably in /etc/puppet/puppet.conf) to see if it has
reverted to a package supplied file rather than the customized file
required to work with your ENC. Once you have determined this is the case,
you should find where your administrator keeps the custom config file and
replace it (tip: if it's stored as a template that will only be installed
by running puppet to get a class list from the ENC this won't help and it
needs to be done manually). Catalogs compile much happier when puppet
actually uses the ENC.
Occam's razor in action, check the most likely cause of the problem first.
On Monday, December 23, 2013 3:50:23 PM UTC-5, Luke Alford wrote:
>
> I am having problems with our puppet/foreman infrastructure that seem to
> coincide with an upgrade to 3.4.0-1, but that are persisting after a
> downgrade back to 3.3.2-1. There are no obvious errors I can find and I
> don't seem to be having any problems with puppet talking to foreman, so
> overall things look completely normal other than I have puppet classes when
> foreman responds with node information and an empty catalog after the
> puppet master compiles. I am pretty stumped by this and would appreciate
> any thoughts others may have on troubleshooting this problem.
>
> First, here's some more important information about our system. We had
> been running puppet 3.3.2-1 and foreman 1.1-stable for quite some time
> before things bumped up to 3.4.0 over the weekend.
> - centos 6.4 for puppet master and nodes
> - puppet and puppet-server 3.3.2-1 (I downgraded both puppet and
> puppet-server back to this version since we'd been running successfully
> with it for some time now)
> - puppetdb and puppetdb-terminus 1.5.0-1
> - foreman 1.1-stable (we've also been running this for some time, we don't
> do any provisioning with it but we use it as an ENC for storing data and
> classifying nodes into roles)
> - puppet and foreman running behind nginx/passenger
>
> If the upgrade to 3.4.0 was the sole source of my problem I would expect
> things to have returned back to normal after the downgrade. Since the
> catalog is still empty though, there's got to be something I'm missing.
>
> Here are a few things I've tried while digging into this.
>
> 1) puppet master --compile
> The node is found, but there are no classes identified. Therein lies the
> problem.
> ...
> "version": 1387830624,
> "classes": [
> "settings",
> "default"
> ],
> "environment": "production"
> ...
>
> 2) Firing up irb, I manually checked what foreman returns for . I
> won't post the whole output including parameters, but the important part is
> that the keyset for the classes key in the yaml document matches what I
> would expect for this particular monolithic test box.
> ...
> {"play"=>nil, "sudo"=>nil, "mongo"=>nil, "rsyslog"=>nil,
> "mysql::server"=>nil, "jmx"=>nil, "nginx::loadbalancer"=>nil, "ssh"=>nil,
> "play::install"=>nil, "mongo::seed"=>nil, "users"=>nil,
> "mysql::server::master"=>nil, "logstash"=>nil, "common"=>nil,
> "puppet"=>nil, "zabbix"=>nil, "motd"=>nil, "logrotate"=>nil, "java"=>nil,
> "yum"=>nil, "ntp"=>nil}
>
> 3) I noticed that facter also updated to version 1.7.4 during this time,
> but at least from what I've tried so far this doesn't seem to be a problem
> of any kind.
>
> 4) Yes, I did try turning it off and on again. :)
>
> So it seems that somewhere between talking to the ENC and actually
> compiling the catalog that my puppet classes disappear into the ether that
> I have not yet identified. I would appreciate any suggestions people have
> on further troubleshooting, or why the upgrade might have gotten things
> into an odd state that rolling back the package somehow doesn't fix.
>
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/a4d7e044-2210-4d5f-8543-873331360f09%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[Puppet Users] puppet 3.4.0 upgrade, foreman 1.1-stable, problems that are not provisioning related
I am having problems with our puppet/foreman infrastructure that seem to
coincide with an upgrade to 3.4.0-1, but that are persisting after a
downgrade back to 3.3.2-1. There are no obvious errors I can find and I
don't seem to be having any problems with puppet talking to foreman, so
overall things look completely normal other than I have puppet classes when
foreman responds with node information and an empty catalog after the
puppet master compiles. I am pretty stumped by this and would appreciate
any thoughts others may have on troubleshooting this problem.
First, here's some more important information about our system. We had been
running puppet 3.3.2-1 and foreman 1.1-stable for quite some time before
things bumped up to 3.4.0 over the weekend.
- centos 6.4 for puppet master and nodes
- puppet and puppet-server 3.3.2-1 (I downgraded both puppet and
puppet-server back to this version since we'd been running successfully
with it for some time now)
- puppetdb and puppetdb-terminus 1.5.0-1
- foreman 1.1-stable (we've also been running this for some time, we don't
do any provisioning with it but we use it as an ENC for storing data and
classifying nodes into roles)
- puppet and foreman running behind nginx/passenger
If the upgrade to 3.4.0 was the sole source of my problem I would expect
things to have returned back to normal after the downgrade. Since the
catalog is still empty though, there's got to be something I'm missing.
Here are a few things I've tried while digging into this.
1) puppet master --compile
The node is found, but there are no classes identified. Therein lies the
problem.
...
"version": 1387830624,
"classes": [
"settings",
"default"
],
"environment": "production"
...
2) Firing up irb, I manually checked what foreman returns for . I
won't post the whole output including parameters, but the important part is
that the keyset for the classes key in the yaml document matches what I
would expect for this particular monolithic test box.
...
{"play"=>nil, "sudo"=>nil, "mongo"=>nil, "rsyslog"=>nil,
"mysql::server"=>nil, "jmx"=>nil, "nginx::loadbalancer"=>nil, "ssh"=>nil,
"play::install"=>nil, "mongo::seed"=>nil, "users"=>nil,
"mysql::server::master"=>nil, "logstash"=>nil, "common"=>nil,
"puppet"=>nil, "zabbix"=>nil, "motd"=>nil, "logrotate"=>nil, "java"=>nil,
"yum"=>nil, "ntp"=>nil}
3) I noticed that facter also updated to version 1.7.4 during this time,
but at least from what I've tried so far this doesn't seem to be a problem
of any kind.
4) Yes, I did try turning it off and on again. :)
So it seems that somewhere between talking to the ENC and actually
compiling the catalog that my puppet classes disappear into the ether that
I have not yet identified. I would appreciate any suggestions people have
on further troubleshooting, or why the upgrade might have gotten things
into an odd state that rolling back the package somehow doesn't fix.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/8ab0dbef-8ffc-4b57-b26e-e30991c41085%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Re: missing msgpack gem following puppet 3.4.0 upgrade
Thanks for the confirmation Adrien, and for the tip on the puppet master
compile command. I don't do much administration so I'm not terribly
familiar with some of those common commands. I've got some more questions
you or others might be able to shed light on, but they're well beyond the
scope of my original question so I'll post a new thread.
Thanks!
On Monday, December 23, 2013 2:12:57 PM UTC-5, Adrien Thebo wrote:
>
> Puppet 3.4.0 adds experimental support for the msgpack protocol (
> http://docs.puppetlabs.com/puppet/3/reference/experiments_msgpack.html),
> but doesn't depend on it being present. To enable it you need to install
> the msgpack gem and set the preferred serialization format, so unless
> you're explicitly enabling it then it shouldn't be a concern. Since it's
> only experimental we aren't planning on moving to it in the near future,
> but it's available so that people can try it for themselves.
>
> With respect to your puppet runs being empty, you'll probably want to
> ensure that the puppet master is generating the catalogs that you expected.
> You can do `puppet master --compile nodename` to directly generate a
> catalog; if that returns an empty catalog then it is probably an issue with
> how the master is configured.
>
>
> On Mon, Dec 23, 2013 at 9:54 AM, Luke Alford
> > wrote:
>
>> Upon downgrading to 3.3.2-1 the problem still remains, so it does appear
>> that the missing msgpack gem is a red herring that is not the cause of my
>> problem. I'd still be interested in thoughts on that debug statement
>> though, particularly if we choose to move to the improved serialization in
>> the future.
>>
>>
>> On Monday, December 23, 2013 12:44:59 PM UTC-5, Luke Alford wrote:
>>>
>>> Hello all,
>>>
>>> Our puppet master and agents recently updated to 3.4.0 and all of our
>>> puppet runs are now compiling the catalog and promptly exiting happily
>>> without actually doing anything. There isn't anything unusual about what I
>>> see in the log output aside from a debug message about missing the msgpack
>>> gem. We are not attempting to use the new serialization at this time and no
>>> configurations have changed other than the package upgrade, but nonetheless
>>> puppet seems displeased about not having the library. Has anyone else
>>> encountered this? Is there something besides the package upgrade that is
>>> required for this to work?
>>>
>>> OS: CentOS 6.4
>>> Puppet: 3.4.0-1
>>>
>>> puppet.noarch 3.4.0-1.el6
>>> @PuppetLabs
>>> puppet-server.noarch 3.4.0-1.el6
>>> @PuppetLabs
>>>
>>> Some log output from an agent noop run, I am not 100% certain this is
>>> causing the problem since the missing gem is only logged at the debug
>>> level, but I'm only going off of nothing else seeming out of the ordinary.
>>> I appreciate any thoughts on the matter, in the meantime I'll trying going
>>> back to 3.3.2-1 and see if that restores proper function. Our normal runs
>>> take many minutes as puppet does quite a bit for us, but it now bows out
>>> awfully quick.
>>>
>>> ...
>>> Debug: Using settings: adding file resource 'resourcefile':
>>> 'File[/var/lib/puppet/state/resources.txt]{:loglevel=>:debug,
>>> :links=>:follow, :owner=>"root", :backup=>false, :mode=>"640",
>>> :ensure=>:file, :path=>"/var/lib/puppet/state/resources.txt"}'
>>> Debug: Using settings: adding file resource 'clientyamldir':
>>> 'File[/var/lib/puppet/client_yaml]{:loglevel=>:debug, :links=>:follow,
>>> :backup=>false, :mode=>"750", :ensure=>:directory, :path=>"/var/lib/puppet/
>>> client_yaml"}'
>>> Debug: Using settings: adding file resource 'classfile':
>>> 'File[/var/lib/puppet/classes.txt]{:loglevel=>:debug, :links=>:follow,
>>> :owner=>"root", :backup=>false, :mode=>"640", :ensure=>:file,
>>> :path=>"/var/lib/puppet/classes.txt"}'
>>> Debug: Using settings: adding file resource 'lastrunreport':
>>> 'File[/var/lib/puppet/state/last_run_report.yaml]{:loglevel=>:debug,
>>> :links=>:follow, :backup=>false, :mode=>"640", :ensure=>:file,
>>> :path=>"/var/lib/puppet/state/
[Puppet Users] Re: missing msgpack gem following puppet 3.4.0 upgrade
Upon downgrading to 3.3.2-1 the problem still remains, so it does appear
that the missing msgpack gem is a red herring that is not the cause of my
problem. I'd still be interested in thoughts on that debug statement
though, particularly if we choose to move to the improved serialization in
the future.
On Monday, December 23, 2013 12:44:59 PM UTC-5, Luke Alford wrote:
>
> Hello all,
>
> Our puppet master and agents recently updated to 3.4.0 and all of our
> puppet runs are now compiling the catalog and promptly exiting happily
> without actually doing anything. There isn't anything unusual about what I
> see in the log output aside from a debug message about missing the msgpack
> gem. We are not attempting to use the new serialization at this time and no
> configurations have changed other than the package upgrade, but nonetheless
> puppet seems displeased about not having the library. Has anyone else
> encountered this? Is there something besides the package upgrade that is
> required for this to work?
>
> OS: CentOS 6.4
> Puppet: 3.4.0-1
>
> puppet.noarch 3.4.0-1.el6
> @PuppetLabs
> puppet-server.noarch 3.4.0-1.el6
> @PuppetLabs
>
> Some log output from an agent noop run, I am not 100% certain this is
> causing the problem since the missing gem is only logged at the debug
> level, but I'm only going off of nothing else seeming out of the ordinary.
> I appreciate any thoughts on the matter, in the meantime I'll trying going
> back to 3.3.2-1 and see if that restores proper function. Our normal runs
> take many minutes as puppet does quite a bit for us, but it now bows out
> awfully quick.
>
> ...
> Debug: Using settings: adding file resource 'resourcefile':
> 'File[/var/lib/puppet/state/resources.txt]{:loglevel=>:debug,
> :links=>:follow, :owner=>"root", :backup=>false, :mode=>"640",
> :ensure=>:file, :path=>"/var/lib/puppet/state/resources.txt"}'
> Debug: Using settings: adding file resource 'clientyamldir':
> 'File[/var/lib/puppet/client_yaml]{:loglevel=>:debug, :links=>:follow,
> :backup=>false, :mode=>"750", :ensure=>:directory,
> :path=>"/var/lib/puppet/client_yaml"}'
> Debug: Using settings: adding file resource 'classfile':
> 'File[/var/lib/puppet/classes.txt]{:loglevel=>:debug, :links=>:follow,
> :owner=>"root", :backup=>false, :mode=>"640", :ensure=>:file,
> :path=>"/var/lib/puppet/classes.txt"}'
> Debug: Using settings: adding file resource 'lastrunreport':
> 'File[/var/lib/puppet/state/last_run_report.yaml]{:loglevel=>:debug,
> :links=>:follow, :backup=>false, :mode=>"640", :ensure=>:file,
> :path=>"/var/lib/puppet/state/last_run_report.yaml"}'
> Debug: Using settings: adding file resource 'clientbucketdir':
> 'File[/var/lib/puppet/clientbucket]{:loglevel=>:debug, :links=>:follow,
> :backup=>false, :mode=>"750", :ensure=>:directory,
> :path=>"/var/lib/puppet/clientbucket"}'
> Debug: Using settings: adding file resource 'graphdir':
> 'File[/var/lib/puppet/state/graphs]{:loglevel=>:debug, :links=>:follow,
> :backup=>false, :ensure=>:directory, :path=>"/var/lib/puppet/state/graphs"}'
> Debug: Using settings: adding file resource 'statefile':
> 'File[/var/lib/puppet/state/state.yaml]{:loglevel=>:debug, :links=>:follow,
> :backup=>false, :mode=>"660", :ensure=>:file,
> :path=>"/var/lib/puppet/state/state.yaml"}'
> Debug: Using settings: adding file resource 'client_datadir':
> 'File[/var/lib/puppet/client_data]{:loglevel=>:debug, :links=>:follow,
> :backup=>false, :mode=>"750", :ensure=>:directory,
> :path=>"/var/lib/puppet/client_data"}'
> Debug: Using settings: adding file resource 'lastrunfile':
> 'File[/var/lib/puppet/state/last_run_summary.yaml]{:loglevel=>:debug,
> :links=>:follow, :backup=>false, :mode=>"644", :ensure=>:file,
> :path=>"/var/lib/puppet/state/last_run_summary.yaml"}'
> Debug: Finishing transaction 70160328710680
> Debug: Loaded state in 0.06 seconds
> *Debug: Failed to load library 'msgpack' for feature 'msgpack'*
> Debug: node supports formats: pson b64_zlib_yaml yaml raw
> Debug: Using cached certificate for ca
> Debug: Using cached certificate for stack01.qa.tstllc.net
> D
[Puppet Users] missing msgpack gem following puppet 3.4.0 upgrade
Hello all,
Our puppet master and agents recently updated to 3.4.0 and all of our
puppet runs are now compiling the catalog and promptly exiting happily
without actually doing anything. There isn't anything unusual about what I
see in the log output aside from a debug message about missing the msgpack
gem. We are not attempting to use the new serialization at this time and no
configurations have changed other than the package upgrade, but nonetheless
puppet seems displeased about not having the library. Has anyone else
encountered this? Is there something besides the package upgrade that is
required for this to work?
OS: CentOS 6.4
Puppet: 3.4.0-1
puppet.noarch 3.4.0-1.el6
@PuppetLabs
puppet-server.noarch 3.4.0-1.el6
@PuppetLabs
Some log output from an agent noop run, I am not 100% certain this is
causing the problem since the missing gem is only logged at the debug
level, but I'm only going off of nothing else seeming out of the ordinary.
I appreciate any thoughts on the matter, in the meantime I'll trying going
back to 3.3.2-1 and see if that restores proper function. Our normal runs
take many minutes as puppet does quite a bit for us, but it now bows out
awfully quick.
...
Debug: Using settings: adding file resource 'resourcefile':
'File[/var/lib/puppet/state/resources.txt]{:loglevel=>:debug,
:links=>:follow, :owner=>"root", :backup=>false, :mode=>"640",
:ensure=>:file, :path=>"/var/lib/puppet/state/resources.txt"}'
Debug: Using settings: adding file resource 'clientyamldir':
'File[/var/lib/puppet/client_yaml]{:loglevel=>:debug, :links=>:follow,
:backup=>false, :mode=>"750", :ensure=>:directory,
:path=>"/var/lib/puppet/client_yaml"}'
Debug: Using settings: adding file resource 'classfile':
'File[/var/lib/puppet/classes.txt]{:loglevel=>:debug, :links=>:follow,
:owner=>"root", :backup=>false, :mode=>"640", :ensure=>:file,
:path=>"/var/lib/puppet/classes.txt"}'
Debug: Using settings: adding file resource 'lastrunreport':
'File[/var/lib/puppet/state/last_run_report.yaml]{:loglevel=>:debug,
:links=>:follow, :backup=>false, :mode=>"640", :ensure=>:file,
:path=>"/var/lib/puppet/state/last_run_report.yaml"}'
Debug: Using settings: adding file resource 'clientbucketdir':
'File[/var/lib/puppet/clientbucket]{:loglevel=>:debug, :links=>:follow,
:backup=>false, :mode=>"750", :ensure=>:directory,
:path=>"/var/lib/puppet/clientbucket"}'
Debug: Using settings: adding file resource 'graphdir':
'File[/var/lib/puppet/state/graphs]{:loglevel=>:debug, :links=>:follow,
:backup=>false, :ensure=>:directory, :path=>"/var/lib/puppet/state/graphs"}'
Debug: Using settings: adding file resource 'statefile':
'File[/var/lib/puppet/state/state.yaml]{:loglevel=>:debug, :links=>:follow,
:backup=>false, :mode=>"660", :ensure=>:file,
:path=>"/var/lib/puppet/state/state.yaml"}'
Debug: Using settings: adding file resource 'client_datadir':
'File[/var/lib/puppet/client_data]{:loglevel=>:debug, :links=>:follow,
:backup=>false, :mode=>"750", :ensure=>:directory,
:path=>"/var/lib/puppet/client_data"}'
Debug: Using settings: adding file resource 'lastrunfile':
'File[/var/lib/puppet/state/last_run_summary.yaml]{:loglevel=>:debug,
:links=>:follow, :backup=>false, :mode=>"644", :ensure=>:file,
:path=>"/var/lib/puppet/state/last_run_summary.yaml"}'
Debug: Finishing transaction 70160328710680
Debug: Loaded state in 0.06 seconds
*Debug: Failed to load library 'msgpack' for feature 'msgpack'*
Debug: node supports formats: pson b64_zlib_yaml yaml raw
Debug: Using cached certificate for ca
Debug: Using cached certificate for stack01.qa.tstllc.net
Debug: Using cached certificate_revocation_list for ca
Info: Retrieving plugin
*Debug: Failed to load library 'msgpack' for feature 'msgpack'*
Debug: file_metadata supports formats: pson b64_zlib_yaml yaml raw
Debug: Finishing transaction 70160331731520
Info: Loading facts in /var/lib/puppet/lib/facter/fqdn_int.rb
Info: Loading facts in /var/lib/puppet/lib/facter/domain_int.rb
Info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
Info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
Info: Loading facts in /var/lib/puppet/lib/facter/datacenter.rb
Info: Loading facts in /var/lib/puppet/lib/facter/environment.rb
Info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
Info: Loading facts in /var/lib/puppet/lib/facter/pe_version.rb
Info: Loading facts in /var/lib/puppet/lib/facter/role.rb
*Debug: Failed to load library 'msgpack' for feature 'msgpack'*
Debug: catalog supports formats: pson dot b64_zlib_yaml yaml raw
Info: Caching catalog for stack01.qa.tstllc.net
Debug: Creating default schedules
Debug: Loaded state in 0.03 seconds
Info: Applying configuration version '1387818458'
Debug: Finishing transaction 70160316686700
Debug: Storing state
Debug: Stored state in 0.16 seconds
Notice: Finished catalog run in 0.26 seconds
--
You received this message because
[Puppet Users] best practices for reusing ruby functions across custom providers
Hello puppet users, I have recently started dabbling in custom types and providers in order to solve some problems at work. I now find myself needing to have 2 custom providers that leverage a common function to fetch data from a web service and I very much do not want to have 2 copies of this function in my code base. I'd like to hear how others have addressed this kind of problem, and I'm certainly amenable to alternatives if there's a better way than the types/providers. Here is what I'm trying to do and the constraints I cannot currently get around. 1) I am dynamically configuring nginx virtual hosts and adding lines to /etc/hosts for internal routing (cloud hosted boxes all have external ip's and I need the various names we support to resolve to internal ip's of the load balancer) 2) The data I need for this configuration (fqdn's, certs, and keys) cannot be determined on the puppet master because the data source is a web service (delivers a json array) I install and configure and must be running on the node before I can query it. I already have the custom nginx provider written and working to configure the virtual hosts, but I also need the fqdn's available from the same web service and I would like to reuse the function that fetches and parses the json from the web service. This second custom provider will be responsible for adding the appropriate lines to /etc/hosts to handle my internal routing problem. Questions: 1) Is there a good way to have an arbitrary ruby module and function that I can require in both of my providers? I'm reasonably good with ruby, but we're not a ruby shop and I'd rather not have to get into packaging and managing gems for the purpose of sharing a 20 line function in 2 places. 2) Is there a way for custom providers to return data to a manifest? I'd rather use the file_line resource to handle my /etc/hosts needs, but I don't know of a means of returning data from a resource declaration (which would be my existing custom type using the nginx provider) that I could use in another resource. 3) Is there a way I can pass arguments to an executable fact? I don't know of one, but I suppose I could set environments variables (i.e. things like the web service host, port, user, password, etc...) provided that executable facts run each time they are referenced in a manifest? Thanks for any input. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users. For more options, visit https://groups.google.com/groups/opt_out.
