Any chance you can share what configuration options were non-optimal?
We're having some timeout issues as well on CentOS.
On Thursday, 5 September 2013 14:32:56 UTC-4, Pete Hartman wrote:
>
> Being able to set the timeout long enough gave us enough data to find the
> problem.
>
> Our SPARC build of OpenSSL used some configuration options that were,
> shall we say, non-optimal :-).
>
> On a corrected OpenSSL build the SPARC times are now in the same ballpark
> as the x86 times.
>
> Thanks again for your help Cory.
>
> Pete
>
> On Wednesday, September 4, 2013 6:56:34 PM UTC-5, Cory Stoker wrote:
>>
>> We have lots of puppet clients on crappy bandwidth that would time out
>> like this as well. The option we changed to fix this is:
>>
>> #Specify the timeout to wait for catalog in seconds
>> configtimeout = 600
>>
>> The default time is like 60 or 120 secs. Another thing you should do
>> is check out the logs of the web server if you are using passenger.
>> You should see a ton of "GET" requests when you need to sync plugins.
>> To force your puppet agent to redownload stuff remove the $vardir/lib
>> directory on the agent.
>>
>>
>> On Wed, Sep 4, 2013 at 1:48 PM, Pete Hartman <pete.h...@gmail.com>
>> wrote:
>> > I'm having a similar problem.
>> >
>> > I know for a fact that I am not contending with other agents, because
>> this
>> > is in a lab environment and none of my agents is scheduled for periodic
>> runs
>> > (yet).
>> >
>> > I have successfully run puppet agent -t first time, signed the cert,
>> and run
>> > it a second time to pull over stdlib and other modules on agents
>> running
>> > RHEL 6 and Solaris 10u10 x86.
>> >
>> > But I'm getting this timeout on a Solaris 10u10 box running on a T4-1
>> SPARC
>> > system.
>> >
>> > This was my third run:
>> >
>> > # date;puppet agent -t;date
>> > Wed Sep 4 14:12:05 CDT 2013
>> > Info: Retrieving plugin
>> > Notice:
>> /File[/var/lib/puppet/lib/puppet/parser/functions/count.rb]/ensure:
>> > defined content as '{md5}9eb74eccd93e2b3c87fd5ea14e329eba'
>> > Notice:
>> >
>> /File[/var/lib/puppet/lib/puppet/parser/functions/validate_bool.rb]/ensure:
>> > defined content as '{md5}4ddffdf5954b15863d18f392950b88f4'
>> > Notice:
>> >
>> /File[/var/lib/puppet/lib/puppet/parser/functions/get_module_path.rb]/ensure:
>>
>>
>> > defined content as '{md5}d4bf50da25c0b98d26b75354fa1bcc45'
>> > Notice:
>> >
>> /File[/var/lib/puppet/lib/puppet/parser/functions/is_ip_address.rb]/ensure:
>> > defined content as '{md5}a714a736c1560e8739aaacd9030cca00'
>> > Error:
>> >
>> /File[/var/lib/puppet/lib/puppet/parser/functions/is_numeric.rb]/ensure:
>> > change from absent to file failed: execution expired
>> >
>> > Error: Could not retrieve plugin: execution expired
>> > Info: Caching catalog for AGENT
>> > Info: Applying configuration version '1378322110'
>> > Notice: Finished catalog run in 0.11 seconds
>> > Wed Sep 4 14:15:58 CDT 2013
>> >
>> >
>> > Each time I've run it, I get about 10 or so files and then I get
>> "execution
>> > expired".
>> >
>> > What I'd really like to see is whether I can increase the expiry
>> timeout.
>> >
>> >
>> > Some other details: The master is RHEL 6 on a Sun/Oracle X4800, lots
>> and
>> > lots of fast cores and memory. I'm using Puppet Open Source. I'm using
>> > passenger. I have no real modules other than some basic forge modules
>> I've
>> > installed to start out with.
>> >
>> > [root@MASTER audit]# cd /etc/puppet/modules
>> > [root@MASTER modules]# ls
>> > apache concat epel firewall inifile passenger puppet puppetdb
>> ruby
>> > stdlib
>> >
>> > I briefly disabled SELinux on the master, but saw no change in
>> behavior.
>> >
>> > I'm certain that the firewall is right because other agents have had no
>> > problems. iptables IS enabled, however.
>> >
>> > The master and the agent are on the same subnet, so I don't suspect a
>> > network performance issue directly.
>> >
>> > On Solaris, because the vendor supplied OpenSSL is antique and doesn't
>> > include SHA256, we have built our own OpenSSL and our own Ruby using
>> that
>> > OpenSSL Library. Even though SPARC is a 64 bit architecture, Ruby
>> seems to
>> > default to a 32 bit build, so we built OpenSSL as 32 bit as well to
>> match.
>> > I've got an open question to the guy responsible for that to see how
>> hard it
>> > would be to try to build Ruby as 64 bit, that's likely a next test.
>> >
>> > I have not yet run snoop on the communication to see what's going on
>> the
>> > network side, but as I say I don't really expect the network to be the
>> > problem, between being on the same subnet and success on other systems
>> with
>> > higher clock speeds.
>> >
>> > Any pointers to other possible causes or somewhere I can (even
>> temporarily)
>> > increase the timeout would be appreciated.
>> >
>> >
>> >
>> >
>> > On Thursday, August 8, 2013 8:56:33 AM UTC-5, jcbollinger wrote:
>> >>
>> >>
>> >>
>> >> On Wednesday, August 7, 2013 11:46:06 AM UTC-5, Cesar Covarrubias
>> wrote:
>> >>>
>> >>> I am already using Passenger. My master is still being minimally
>> >>> utilized, as I'm just now beginning the deployment process. In terms
>> of
>> >>> specs, it is running 4 cores and 8GB of mem and 4GB of swap. During a
>> run,
>> >>> the total system usage is no more than 2GB and no swap. No network
>> >>> congestion and I/O is low on the SAN which these VMs use.
>> >>>
>> >>> The odd thing is once the hosts get all the libs sync'd, performance
>> is
>> >>> fine on further changes. It's quite perplexing.
>> >>>
>> >>
>> >> To be certain that contention by multiple Puppet clients does not
>> >> contribute to the issue, ensure that the problem still occurs when
>> only one
>> >> client attempts to sync at a time. If it does, then the issue
>> probably has
>> >> something to do with the pattern of communication between client and
>> master,
>> >> for that's the main thing that differs between an initial run and
>> subsequent
>> >> ones.
>> >>
>> >> During the initial plugin sync, the master delivers a moderately large
>> >> number of small files to the client, whereas on subsequent runs it
>> usually
>> >> delivers only a catalog, and perhaps, later, 'source'd Files declared
>> in
>> >> your manifests. There may be a separate connection established
>> between
>> >> client and master for each synced file, and anything that might slow
>> that
>> >> down could contribute to the problem.
>> >>
>> >> For instance, if a firewall on client, master, or any device between
>> makes
>> >> it slow or unreliable to establish connections; if multiple clients
>> are
>> >> configured with the same IP number; if a router anywhere along the
>> network
>> >> path is marginal; if a leg of the path is wireless and subject to
>> >> substantial radio interference; if any part of your network is
>> suffering
>> >> from a denial-of-service attack; etc. then probabilistically speaking,
>> the
>> >> effect would be much more noticeable when a successful transaction
>> requires
>> >> many connections and data transfers than when it requires few.
>> >>
>> >>
>> >> John
>> >>
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups
>> > "Puppet Users" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an
>> > email to puppet-users...@googlegroups.com.
>> > To post to this group, send email to puppet...@googlegroups.com.
>> > Visit this group at http://groups.google.com/group/puppet-users.
>> > For more options, visit https://groups.google.com/groups/opt_out.
>>
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/42ffc4a2-65e2-4222-b2ea-3da50391a0d7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
