I figured it out, partway. There's something about the CRL that my puppetmaster is generating that kills the client.
When I mv $ssldir/ca/ca_crl.pem out of the way on the server (so it can't be re-downloaded) and rm $ssldir/crl.pem on the client, they all work. Put it back in place, or generate a new one with puppet cert -- clean <somehost>, and I get this error. This explains why openssl was working - It doesn't use a CRL. 100% repeatable. Still working out why the CRL is fatal. I had actually suspected this because of my earlier trouble with it (#4948) but did not see the VIP (nor the endpoints) certs in the CRL so I discounted it. On Feb 18, 9:16 am, Eric Sorenson <[email protected]> wrote: > Hi Brice, thanks for the reply. > > On Feb 18, 1:32 am, Brice Figureau <[email protected]> > wrote: > > > This certainly means your local node CA cert is not able to verify the > > server proposed certificate (probably because the server advertised cert > > hasn't been signed by this CA, or you use a CA chain but don't send the > > full chain to the client). > > Normally I agree but I don't understand why all the other hosts > connecting to the same VIP are fine. There are just a couple of them > that exhibit this problem and scores that are fine. > > > You didn't ask openssl s_client to actually check the server certificate > > against the CA cert of the client. > > > Can you try: > > openssl s_client -connect puppet:8140 -CAfile certs/ca.pem -cert > > certs/db9.domain.com.pem -key private_keys/db9.domain.com.pem -showcerts > > -state -verify 2 > > Ah, I had actually had that in a different command but did not paste > it correctly. It also succeeds: > > (echo "GET /" ; echo; ) | openssl s_client -connect puppet:8140 -cert > certs/db9.domain.com.pem -key private_keys/db9.domain.com.pem - > showcerts -state -verify 2 -CAfile certs/ca.pem 2>&1 | grep -i verify > verify depth is 2 > verify return:1 > verify return:1 > SSL_connect:SSLv3 write certificate verify A > Verify return code: 0 (ok) > > > For an unknown reason your local node CA cert is not correct. > > You can solve this by overwriting it with the main CA cert, or check > > that your server certificate is indeed correctly signed by the CA you > > think it was signed with. > > I renamed the local file to ca-not.pem then put the cert from the CA > shown with openssl -showcerts into puppet's certs dir and compared > them: > > [/var/lib/puppet/ssl/certs]# openssl x509 -in ca.pem -modulus -noout > Modulus=B7491A13EFEDBBD259F90D914365DA8E2BF11F8AB6762723E3C91C1868AE999EAB3 > E0003F10C99363C7D9A5260B474EA193F52F05C42456FD0089E2284CDE9FD26F3F4F82D7A6B > 7E6A9B7A27EA2B2C94B46272926932A4FB6D6EDF7D6EEC720781B76261382927799E63FFA8D > EF5EA6239D5590ADD34A0C06DB9818834DBB6BF > [/var/lib/puppet/ssl/certs]# openssl x509 -in ca-not.pem -modulus - > noout > Modulus=B7491A13EFEDBBD259F90D914365DA8E2BF11F8AB6762723E3C91C1868AE999EAB3 > E0003F10C99363C7D9A5260B474EA193F52F05C42456FD0089E2284CDE9FD26F3F4F82D7A6B > 7E6A9B7A27EA2B2C94B46272926932A4FB6D6EDF7D6EEC720781B76261382927799E63FFA8D > EF5EA6239D5590ADD34A0C06DB9818834DBB6BF > > I am really starting to think something is amiss with the ruby->openssl > libraries on the affected hosts. They show the same behaviour > > connecting to other puppet vips in the environment, including one that > has the SSL connection terminated at Apache instead of the > loadbalancer. > > Thanks very much Brice. I will keep at it. > > -=Eric -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
