SOLVED: I didn't have the VMs in the host's /etc/hosts. Once I put all of them into the host (i.e, the Puppet Master), then everything sped up to full speed on the clients. (I had been thinking it was a problem on the client side, so all of my troubleshooting had been isolated to the clients. I finally thought to try the server side this morning.)
Question: Should I submit this as a bug? I'm not sure if this is a Puppet bug or an OS bug? The Puppet Master was eventually serving up the files, but only after 10 seconds (while it was trying to resolve the name?) Since the Puppet Master was able to reply without being able to resolve the name, why was it bothering to resolve the name instead of just replying directly to the request? Thanks, Kirk On Wednesday, January 9, 2013 5:45:16 PM UTC-5, Kirk Steffensen wrote: > > Josh, > > use_srv_records is not set in puppet.conf. 'puppet config print > use_srv_records" shows it set to the default of false. > > I ran tcpdump from inside the Vagrant VM during pluginsync. On eth1, > where the VM is connecting to the puppet master running on the host, the > only calls are puppet calls on 8140 with replies coming back on 34757. > Running 'tcpdump port 53' shows no DNS traffic at all during the > pluginsync. (FWIW, I'm not running DNS for these vagrant boxes. > Everything is in /etc/hosts, so it doesn't surprise me that there are zero > DNS calls.) > > Thanks, > Kirk > > On Wednesday, January 9, 2013 5:13:04 PM UTC-5, Josh Cooper wrote: >> >> Hi Kirk, >> >> Do you happen to have SRV lookups enabled via the `use_srv_records` >> setting? You may want to run tcpdump and look for extraneous DNS >> lookups. >> >> Josh >> >> On Wed, Jan 9, 2013 at 2:00 PM, Kirk Steffensen >> <ki...@steffensenfamily.com> wrote: >> > Here is the strace output from one of the 10-second periods while >> waiting >> > for the File notice to appear. https://gist.github.com/4497263 >> > >> > The strace output came in two bursts during this 10-seconds. >> > >> > The thing that leaps out at me is that of the 4061 lines of output, >> 3754 of >> > them are rt_sigprocmask calls. I'm wondering if it's a Ruby bug? For >> these >> > VMs, I'm using 1.8.7, installed from source: >> > http://ftp.ruby-lang.org/pub/ruby/ruby-1.8.7-p358.tar.gz >> > >> > I created another gist with the rt_sigprocmask calls to make it easier >> to >> > tell if there is anything else going on. >> https://gist.github.com/4497335 >> > From my quick review, nothing leaps out. >> > >> > Hope this helps point the troubleshooting in the right direction. >> > >> > Thanks, >> > Kirk >> > >> > >> > On Wednesday, January 9, 2013 12:59:40 PM UTC-5, Ken Barber wrote: >> >> >> >> > Ken, thanks. Unfortunately, (from a troubleshooting standpoint), it >> >> > only >> >> > took one or two seconds to sync stdlib on the local box. >> >> > >> >> > rm -rf /var/lib/puppet/lib/* >> >> > puppet agent --test >> >> > >> >> > I saw the same stream of File notices, but they streamed by in real >> >> > time, >> >> > instead of taking 10 seconds per notice. >> >> >> >> Yeah, thats gotta be some comms problem between your PM and clients >> >> then ... just not sure what. >> >> >> >> > John may still have something. There may still be some name >> resolution >> >> > issue on the client, but it's just not obvious from the testing that >> I >> >> > did. >> >> >> >> Perhaps perhaps ... I'd strace the puppet process to see whats >> >> happening in the gaps. 10-15 seconds would lead me to suspect DNS as >> >> well. Perhaps we haven't exhausted this avenue ... try adding all your >> >> hosts into /etc/hosts on each box perhaps? Disabling DNS temporarily >> >> (from say nsswitch.conf) then will remove that avenue of potential >> >> :-). >> >> >> >> > I haven't dug into the file provider code. What mechanism is it >> using >> >> > to >> >> > move the files from the server to the client? Could it be a delay >> in an >> >> > scp >> >> > handshake or the like? >> >> >> >> HTTPS actually ... with client and server certificates. >> >> >> >> ken. >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups >> > "Puppet Users" group. >> > To view this discussion on the web visit >> > https://groups.google.com/d/msg/puppet-users/-/OKNPyOQqZcQJ. >> > >> > To post to this group, send email to puppet...@googlegroups.com. >> > To unsubscribe from this group, send email to >> > puppet-users...@googlegroups.com. >> > For more options, visit this group at >> > http://groups.google.com/group/puppet-users?hl=en. >> >> >> >> -- >> Josh Cooper >> Developer, Puppet Labs >> > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/SjgPXE3VnbYJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
