On Sep 1, 2009, at 10:30 AM, S H wrote:
> On Tue, Sep 1, 2009 at 12:05 PM, Luke Kanies <[email protected]> wrote:
>
> On Sep 1, 2009, at 5:21 AM, S H wrote:
>
> > On Mon, Aug 31, 2009 at 9:31 PM, Luke Kanies <[email protected]>
> wrote:
> >
> > On Aug 26, 2009, at 7:44 AM, S H wrote:
> >
> > > On Wed, Aug 26, 2009 at 1:20 AM, Luke Kanies <[email protected]>
> > wrote:
> > >
> > > On Aug 25, 2009, at 4:00 PM, S H wrote:
> > >
> > > >
> > > >
> > > > On Tue, Aug 25, 2009 at 5:33 PM, Luke Kanies <[email protected]>
> > > wrote:
> > > >
> > > > On Aug 25, 2009, at 1:57 PM, S H wrote:
> > > >
> > > > > If this is the wrong list, please let me know. I'm emailing
> > > puppet-
> > > > > dev since this is firmly in the realm of Puppet code rather
> than
> > > > > typical manifest implementation.
> > > >
> > > > I think this is the right list. At the least, I read this one
> far
> > > > more than -users, and I'm likely to be the only person who knows
> > > some
> > > > of this stuff, unfortunately.
> > > >
> > > > >
> > > > > I'm not sure if this is a bug or if I'm doing it wrong, so any
> > > help
> > > > > is appreciated. This issue exposed itself while I was
> working on
> > > my
> > > > > Bacula type/provider and I have confirmed that it is also a
> > > problem
> > > > > with Naginator.
> > > > >
> > > > > Using Naginator as the example, if I have a manifest like
> this:
> > > > >
> > > > > nagios_host { "test1":
> > > > > alias => "alias1",
> > > > > address => "address1",
> > > > > }
> > > > >
> > > > > nagios_contactgroup { "test2":
> > > > > target => "/etc/nagios/nagios_host.cfg",
> > > > > alias => "alias2",
> > > > > }
> > > > >
> > > > > Only one of the changes will go through at a time. If I add
> > > another
> > > > > nagios_host to the manifest, either both nagios_host
> > > configurations
> > > > > or the nagios_contactgroup configuration will go through.
> > > > >
> > > > > What I suspect is probably very different than the reality,
> but
> > > here
> > > > > it is anyway: provider instances for a given type share the
> > > @records
> > > > > value so modifications to one are reflected in all of
> > > them.Different
> > > > > types don't share @records values, so flushing leaves the last
> > man
> > > > > standing.
> > > > >
> > > > > Obviously, I still haven't figured out the exact details. Can
> > > anyone
> > > > > shed any light on this?
> > > >
> > > > Um, that's pretty strange, and I don't really know what could be
> > > > causing it. I've certainly never heard of anything like it.
> > > >
> > > > Each provider class should be completely independent, and
> > there's no
> > > > way that flushing one should cause another not to get flushed.
> > > >
> > > > Are you saying that the changes don't get logged, or that they
> > don't
> > > > happen? If they don't even get logged, do you get anything
> > > > informative if you run with --verbose --evaltrace?
> > > >
> > > > They all get flushed and the changes get logged - the problem
> lies
> > > > in the way they're flushed.
> > > >
> > > > nagios_host and nagios_contactgroup have each prefetched the
> file
> > > > (if it exists) and modified their own @records variables to
> > reflect
> > > > the changes as they deem necessary. Puppet then calls flush on
> > each
> > > > of these - let's say nagios_host goes first.
> > > >
> > > > So nagios_host's to_file() method produces the string that
> > > > represents the file according to its internal representation,
> and
> > > > that string is written to the file. Then the same thing happens
> > for
> > > > nagios_contactgroup, but since nagios_host changes didn't affect
> > its
> > > > own representation of the string, the file is written out as it
> > was
> > > > originally + nagios_contactgroup's changes.
> > > >
> > > > I'll get you the output tomorrow when I'm back at my
> workstation,
> > > > but it looks perfectly normal. I pieced this together by yaml-
> > > > dumping variables in various places until I had an idea of what
> > was
> > > > happening.
> > >
> > > I'm... still confused. I'll wait for your email with the outputs.
> > >
> > > puppet# cat test.pp
> > > nagios_host { "test1":
> > > alias => "alias1",
> > > address => "address1",
> > > }
> > >
> > > nagios_contactgroup { "test2":
> > > target => "/etc/nagios/nagios_host.cfg",
> > > alias => "alias2",
> > > }
> > >
> > > puppet# puppet --trace --evaltrace --debug --verbose test.pp
> > > puppet# rm /etc/nagios/nagios_host.cfg
> > > puppet# touch /etc/nagios/nagios_host.cfg
> > > puppet# puppet --trace --evaltrace --debug --verbose test.pp
> > > info: Loading fact fc_location
> > > /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:
> > > command not found:
> > > debug: Creating default schedules
> > > debug: Prefetching naginator resources for nagios_host
> > > debug: Prefetching naginator resources for nagios_contactgroup
> > > debug: //Nagios_host[test1]: Changing ensure
> > > debug: //Nagios_host[test1]: 1 change(s)
> > > notice: //Nagios_host[test1]/ensure: created
> > > debug: Flushing nagios_host provider target /etc/nagios/
> > > nagios_host.cfg
> > > debug: Failed to load library 'shadow' for feature 'libshadow'
> > > debug: Finishing transaction 341800180 with 0 changes
> > > debug: //Nagios_contactgroup[test2]: Changing ensure
> > > debug: //Nagios_contactgroup[test2]: 1 change(s)
> > > notice: //Nagios_contactgroup[test2]/ensure: created
> > > debug: Flushing nagios_contactgroup provider target /etc/nagios/
> > > nagios_host.cfg
> > > info: Filebucket[/var/puppet/clientbucket]: Adding /etc/nagios/
> > > nagios_host.cfg(901d65a4e8dacbf4e87ba544d5a86d9a)
> > > debug: Finishing transaction 345760510 with 2 changes
> > > puppet# cat /etc/nagios/nagios_host.cfg
> > > # HEADER: This file was autogenerated at Wed Aug 26 10:24:24 -0400
> > > 2009
> > > # HEADER: by puppet. While it can still be managed manually, it
> > > # HEADER: is definitely not recommended.
> > > define contactgroup {
> > > contactgroup_name test2
> > > alias alias2
> > > }
> > > puppet#
> > >
> > >
> > >
> > > If I add a yaml-dump to parsedfile.rb's self.flush_target(target)
> > > method so it looks like this:
> > >
> > > def self.flush_target(target)
> > > backup_target(target)
> > >
> > > records = target_records(target).reject { |r|
> > > r[:ensure] == :absent
> > > }
> > > require 'yaml'
> > > puts YAML::dump(records)
> > >
> > > target_object(target).write(to_file(records))
> > > end
> > >
> > >
> > > We get:
> > >
> > > puppet# puppet --trace --evaltrace --debug --verbose test.pp
> > > info: Loading fact fc_location
> > > /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:
> > > command not found:
> > > debug: Creating default schedules
> > > debug: Prefetching naginator resources for nagios_host
> > > debug: Prefetching naginator resources for nagios_contactgroup
> > > debug: //Nagios_host[test1]: Changing ensure
> > > debug: //Nagios_host[test1]: 1 change(s)
> > > notice: //Nagios_host[test1]/ensure: created
> > > debug: Flushing nagios_host provider target /etc/nagios/
> > > nagios_host.cfg
> > > debug: Failed to load library 'shadow' for feature 'libshadow'
> > > debug: Finishing transaction 341645600 with 0 changes
> > > ---
> > > - !ruby/object:host
> > > ensure: :present
> > > on_disk: true
> > > parameters:
> > > address: address1
> > > host_name: test1
> > > alias: alias1
> > > target: /etc/nagios/nagios_host.cfg
> > > debug: //Nagios_host[test2]: Changing ensure
> > > debug: //Nagios_host[test2]: 1 change(s)
> > > notice: //Nagios_host[test2]/ensure: created
> > > debug: Flushing nagios_host provider target /etc/nagios/
> > > nagios_host.cfg
> > > ---
> > > - !ruby/object:host
> > > ensure: :present
> > > on_disk: true
> > > parameters:
> > > address: address1
> > > host_name: test1
> > > alias: alias1
> > > target: /etc/nagios/nagios_host.cfg
> > > - !ruby/object:host
> > > ensure: :present
> > > on_disk: true
> > > parameters:
> > > address: address2
> > > host_name: test2
> > > alias: alias2
> > > target: /etc/nagios/nagios_host.cfg
> > > debug: //Nagios_contactgroup[test3]: Changing ensure
> > > debug: //Nagios_contactgroup[test3]: 1 change(s)
> > > notice: //Nagios_contactgroup[test3]/ensure: created
> > > debug: Flushing nagios_contactgroup provider target /etc/nagios/
> > > nagios_host.cfg
> > > info: Filebucket[/var/puppet/clientbucket]: Adding /etc/nagios/
> > > nagios_host.cfg(f81a6f0dd450b6617255a18c3787abfc)
> > > ---
> > > - !ruby/object:contactgroup
> > > ensure: :present
> > > on_disk: true
> > > parameters:
> > > contactgroup_name: test3
> > > alias: alias3
> > > target: /etc/nagios/nagios_host.cfg
> > > debug: Finishing transaction 345707210 with 3 changes
> > > puppet#
> > >
> > >
> > > So you can see that, while both nagios_host resources were flushed
> > > to the file in the second step, only the contactgroup was
> flushed in
> > > the third step. This overwrote the file that contained the two
> > > nagios hosts (which was backed up to the filebucket right before).
> > >
> > > Hopefully that makes things clearer.
> >
> >
> > I'm even more confused. It looks like you've got multiple
> > transactions in a single run; how is that happening? Did you just
> > skip the parts where you ran puppetd multiple times and include
> > concatenated output from multiple runs, or are there are actually
> > multiple transactions in the runs?
> >
> > There are two different runs in my previous email, but there are
> > multiple transactions per run.
> >
> > The first (I've removed the shell commands so you can just see the
> > run output):
> >
> > info: Loading fact fc_location
> > /usr/local/lib/ruby/site_ruby/
> > 1.8/rubygems/custom_require.rb:31: command not found:
> > debug: Creating default schedules
> > debug: Prefetching naginator resources for nagios_host
> > debug: Prefetching naginator resources for nagios_contactgroup
> > debug: //Nagios_host[test1]: Changing ensure
> > debug: //Nagios_host[test1]: 1 change(s)
> > notice: //Nagios_host[test1]/ensure: created
> > debug: Flushing nagios_host provider target /etc/nagios/
> > nagios_host.cfg
> > debug: Failed to load library 'shadow' for feature 'libshadow'
> > debug: Finishing transaction 341800180 with 0 changes
> > debug: //Nagios_contactgroup[test2]: Changing ensure
> > debug: //Nagios_contactgroup[test2]: 1 change(s)
> > notice: //Nagios_contactgroup[test2]/ensure: created
> > debug: Flushing nagios_contactgroup provider target /etc/nagios/
> > nagios_host.cfg
> > info: Filebucket[/var/puppet/clientbucket]: Adding /etc/nagios/
> > nagios_host.cfg(901d65a4e8dacbf4e87ba544d5a86d9a)
> > debug: Finishing transaction 345760510 with 2 changes
> >
> >
> > Ignore the second run (the one with YAML::dump in it) since I
> > apparently copied and pasted over my explanation of modifications to
> > the test.pp file. I think it's just muddying the waters anyway.
> >
> > As far as how the multiple transactions are happening, I don't know
> > what to tell you.
>
> Can you run that with --evaltrace? That will at least tell me when
> one resource ends and another begins.
>
> puppet# rm /etc/nagios/nagios_host.cfg
> puppet# touch /etc/nagios/nagios_host.cfg
> puppet# puppet --evaltrace --debug --verbose test.pp
> info: Loading fact fc_location
> /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:
> command not found:
> debug: Creating default schedules
> debug: Prefetching naginator resources for nagios_host
> debug: Prefetching naginator resources for nagios_contactgroup
> debug: //Nagios_host[test1]: Changing ensure
> debug: //Nagios_host[test1]: 1 change(s)
> notice: //Nagios_host[test1]/ensure: created
> debug: Flushing nagios_host provider target /etc/nagios/
> nagios_host.cfg
> debug: Failed to load library 'shadow' for feature 'libshadow'
> debug: Finishing transaction 341782830 with 0 changes
> debug: //Nagios_contactgroup[test2]: Changing ensure
> debug: //Nagios_contactgroup[test2]: 1 change(s)
> notice: //Nagios_contactgroup[test2]/ensure: created
> debug: Flushing nagios_contactgroup provider target /etc/nagios/
> nagios_host.cfg
> info: Filebucket[/var/puppet/clientbucket]: Adding /etc/nagios/
> nagios_host.cfg(b5e406fa9677c37052588054dec709d2)
> debug: Finishing transaction 345757570 with 2 changes
> puppet# # cat /etc/nagios/nagios_host.cfg
> # HEADER: This file was autogenerated at Tue Sep 01 13:29:57 -0400
> 2009
> # HEADER: by puppet. While it can still be managed manually, it
> # HEADER: is definitely not recommended.
> define contactgroup {
> contactgroup_name test2
> alias alias2
> }
> puppet#
This is even stranger. I can't conceive of a way in which you would
get two transactions in a single, um, transaction, and I'm guessing
that's the source of the problem.
>
> And maybe ping me on IRC so we can debug this in realtime; I'll answer
> if I'm available and we can work it out without the asynchronousness
> of email.
I should have a bit more time today and the rest of the week; try
pinging me again (as soon as freenode lets me log in again).
--
Should I say "I believe in physics", or "I know that physics is true"?
-- Ludwig Wittgenstein, On Certainty, 602.
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/puppet-dev?hl=en
-~----------~----~----~----~------~----~------~--~---
