On Thu, Sep 30, 2010 at 1:21 AM, Brice Figureau <[email protected]> wrote: > On Wed, 2010-09-29 at 17:32 -0700, Jason Wright wrote: >> On Wed, Sep 29, 2010 at 1:54 PM, Brice Figureau >> <[email protected]> wrote: >> > It would be great if you could add some debug statements to the >> > lib/puppet/indirector/yaml.rb file around line 22 to show what the YAML >> > look like, and/or what cache it was trying to load. >> >> I added >> >> Puppet.debug("FOO: failed to read YAML from #{file}") if yaml.nil? >> or yaml.to_s == "" >> >> at line 19 of puppet/indirector/yaml.rb and it's logging when I run >> puppet-load so it looks like something is failing in readlock(). > > Yes that was my gut feeling too. > I think part of the issue is that puppet-load asks always for the same > node. In real world setups it is improbable that the master has to > answer the same question at exactly the same time. > So I think there is a race in the indirector yaml caching subsystem. It > looks like readlock and writelock are not doing their job. > > A temporary workaround: > diff --git a/lib/puppet/indirector/yaml.rb b/lib/puppet/indirector/yaml.rb > index 23997e9..e2948e0 100644 > --- a/lib/puppet/indirector/yaml.rb > +++ b/lib/puppet/indirector/yaml.rb > @@ -19,7 +19,8 @@ class Puppet::Indirector::Yaml < > Puppet::Indirector::Terminus > begin > return from_yaml(yaml) > rescue => detail > - raise Puppet::Error, "Could not parse YAML data for > #{indirection.name} #{request.key}: #{detail}" > + Puppet.notice("cache of #{indirection.name} for #{request.key} is > corrupted/non-existant, content: #{yaml}" > + return nil > end > end > > Can you summarize on what os/filesystems type/ruby versions you are running > your master? > > Hmm, could it be that the node yaml (ie $yamldir) is on NFS or any > filesystem that have issues with file locks?
Just to avoid the timezone round trip because I woke up early :) Jason will either be benchmarking on Ubuntu Hardy or Lucid, and I think he's just on the standard Ruby versions there at the moment. Probably 1.8.6.111-2ubuntu1.3 or 1.8.7.249-2 They're definitely not on NFS. > -- > Brice Figureau > Follow the latest Puppet Community evolutions on www.planetpuppet.org! > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/puppet-dev?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en.
