Hi, it seems the error happens when you run commands that require communication between IPA framework and the Certificate Server (like ipa ca-show). The workflow is the following: 1. the client (= the command "ipa ca-show") is a python process that communicates with httpd on the secure port. It seems this part is OK (ipa ca-find returns successfully). 2. the IPA framework is a wsgi app running inside httpd. The handling of "ipa ca-show" requires the framework to communicate with Dogtag, which is running inside pki-tomcat.The communication happens over a secure port with authentication based on the RA certificate. This communication is not working, probably because httpd doesn't trust the CA that issued Dogtag's server cert.
I think you need to check where httpd is getting its list of trusted CAs when it's acting as a client of Dogtag server. The code is using api.env.tls_ca_cert which is /etc/ipa/ca.crt on rhel/fedora (you can check with "ipa env tls_ca_cert" to find the value on your server) but may be different on ubuntu. flo On Thu, Jun 24, 2021 at 8:49 PM Chris Moody via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote: > Finally had a chance to circle back and work on this further. > > Based on my prior output of: > ===== > # ipa-cacert-manage list > *IPA.REDACTED.COM <http://IPA.REDACTED.COM> IPA CA* > * IPA.REDACTED.COM <http://IPA.REDACTED.COM> IPA CA* > DSTRootCAX3 > letsencryptx3 > isrgrootx1 > lets-encrypt-r3-cross-signed > The ipa-cacert-manage command was successful > ===== > which does show the IPA CA certificate as being recognized and > installed...and the manpage for the command references: > ...CA certificate of the IPA CA (NSS database nickname: "caSigningCert > cert-pki-ca")... > > I was also able to see that dogtag implies that the IPA CA component(s) > are installed/recognized/not-expired: > ========== > REDACTED-1:~# getcert list > Number of certificates and requests being tracked: 10. > Request ID '20200416204629': > status: MONITORING > stuck: no > key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key' > certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem' > CA: dogtag-ipa-ca-renew-agent > issuer: CN=Certificate Authority,O=IPA.REDACTED.COM > subject: CN=IPA RA,O=IPA.REDACTED.COM > expires: 2022-04-06 13:46:30 PDT > key usage: digitalSignature,keyEncipherment,dataEncipherment > eku: id-kp-clientAuth > pre-save command: /usr/lib/ipa/certmonger/renew_ra_cert_pre > post-save command: /usr/lib/ipa/certmonger/renew_ra_cert > track: yes > auto-renew: yes > Request ID '20200416204717': > status: MONITORING > stuck: no > key pair storage: > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert > cert-pki-ca',token='NSS Certificate DB',pin set > certificate: > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert > cert-pki-ca',token='NSS Certificate DB' > CA: dogtag-ipa-ca-renew-agent > issuer: CN=Certificate Authority,O=IPA.REDACTED.COM > subject: CN=CA Audit,O=IPA.REDACTED.COM > expires: 2022-04-06 13:45:48 PDT > key usage: digitalSignature,nonRepudiation > pre-save command: /usr/lib/ipa/certmonger/stop_pkicad > post-save command: /usr/lib/ipa/certmonger/renew_ca_cert > "auditSigningCert cert-pki-ca" > track: yes > auto-renew: yes > Request ID '20200416204718': > status: MONITORING > stuck: no > key pair storage: > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert > cert-pki-ca',token='NSS Certificate DB',pin set > certificate: > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert > cert-pki-ca',token='NSS Certificate DB' > CA: dogtag-ipa-ca-renew-agent > issuer: CN=Certificate Authority,O=IPA.REDACTED.COM > subject: CN=OCSP Subsystem,O=IPA.REDACTED.COM > expires: 2022-04-06 13:45:47 PDT > eku: id-kp-OCSPSigning > pre-save command: /usr/lib/ipa/certmonger/stop_pkicad > post-save command: /usr/lib/ipa/certmonger/renew_ca_cert > "ocspSigningCert cert-pki-ca" > track: yes > auto-renew: yes > Request ID '20200416204719': > status: MONITORING > stuck: no > key pair storage: > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert > cert-pki-ca',token='NSS Certificate DB',pin set > certificate: > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert > cert-pki-ca',token='NSS Certificate DB' > CA: dogtag-ipa-ca-renew-agent > issuer: CN=Certificate Authority,O=IPA.REDACTED.COM > subject: CN=CA Subsystem,O=IPA.REDACTED.COM > expires: 2022-04-06 13:45:47 PDT > key usage: > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment > eku: id-kp-clientAuth > pre-save command: /usr/lib/ipa/certmonger/stop_pkicad > post-save command: /usr/lib/ipa/certmonger/renew_ca_cert > "subsystemCert cert-pki-ca" > track: yes > auto-renew: yes > Request ID '20200416204720': > status: MONITORING > stuck: no > key pair storage: > *type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert > cert-pki-ca'*,token='NSS Certificate DB',pin set > certificate: > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert > cert-pki-ca',token='NSS Certificate DB' > CA: dogtag-ipa-ca-renew-agent > issuer: CN=Certificate Authority,O=IPA.REDACTED.COM > subject: CN=Certificate Authority,O=IPA.REDACTED.COM > expires: 2040-06-26 14:20:56 PDT > key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign > pre-save command: /usr/lib/ipa/certmonger/stop_pkicad > post-save command: /usr/lib/ipa/certmonger/renew_ca_cert > "caSigningCert cert-pki-ca" > track: yes > auto-renew: yes > Request ID '20200416204721': > status: MONITORING > stuck: no > key pair storage: > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert > cert-pki-ca',token='NSS Certificate DB',pin set > certificate: > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert > cert-pki-ca',token='NSS Certificate DB' > CA: dogtag-ipa-ca-renew-agent > issuer: CN=Certificate Authority,O=IPA.REDACTED.COM > subject: CN=REDACTED-1.ipa.REDACTED.com,O=IPA.REDACTED.COM > expires: 2022-04-06 13:45:47 PDT > dns: REDACTED-1.ipa.REDACTED.com > key usage: digitalSignature,keyEncipherment,dataEncipherment > eku: id-kp-serverAuth > pre-save command: /usr/lib/ipa/certmonger/stop_pkicad > post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "Server-Cert > cert-pki-ca" > track: yes > auto-renew: yes > Request ID '20200416204854': > status: MONITORING > stuck: no > key pair storage: type=FILE,location='/var/lib/ipa/certs/kdc.key' > certificate: type=FILE,location='/var/lib/ipa/certs/kdc.crt' > CA: IPA > issuer: CN=Certificate Authority,O=IPA.REDACTED.COM > subject: CN=REDACTED-1.ipa.REDACTED.com,O=IPA.REDACTED.COM > expires: 2022-04-17 13:48:54 PDT > principal name: krbtgt/ipa.redacted....@ipa.redacted.com > key usage: > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment > eku: id-kp-serverAuth,id-pkinit-KPKdc > pre-save command: > post-save command: /usr/lib/ipa/certmonger/renew_kdc_cert > track: yes > auto-renew: yes > Request ID '20200416205655': > status: MONITORING > stuck: no > key pair storage: > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert > cert-pki-kra',token='NSS Certificate DB',pin set > certificate: > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert > cert-pki-kra',token='NSS Certificate DB' > CA: dogtag-ipa-ca-renew-agent > issuer: CN=Certificate Authority,O=IPA.REDACTED.COM > subject: CN=KRA Audit,O=IPA.REDACTED.COM > expires: 2022-04-06 13:53:14 PDT > key usage: digitalSignature,nonRepudiation > pre-save command: /usr/lib/ipa/certmonger/stop_pkicad > post-save command: /usr/lib/ipa/certmonger/renew_ca_cert > "auditSigningCert cert-pki-kra" > track: yes > auto-renew: yes > Request ID '20200416205656': > status: MONITORING > stuck: no > key pair storage: > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='transportCert > cert-pki-kra',token='NSS Certificate DB',pin set > certificate: > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='transportCert > cert-pki-kra',token='NSS Certificate DB' > CA: dogtag-ipa-ca-renew-agent > issuer: CN=Certificate Authority,O=IPA.REDACTED.COM > subject: CN=KRA Transport Certificate,O=IPA.REDACTED.COM > expires: 2022-04-06 13:53:13 PDT > key usage: > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment > eku: id-kp-clientAuth > pre-save command: /usr/lib/ipa/certmonger/stop_pkicad > post-save command: /usr/lib/ipa/certmonger/renew_ca_cert > "transportCert cert-pki-kra" > track: yes > auto-renew: yes > Request ID '20200416205657': > status: MONITORING > stuck: no > key pair storage: > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='storageCert > cert-pki-kra',token='NSS Certificate DB',pin set > certificate: > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='storageCert > cert-pki-kra',token='NSS Certificate DB' > CA: dogtag-ipa-ca-renew-agent > issuer: CN=Certificate Authority,O=IPA.REDACTED.COM > subject: CN=KRA Storage Certificate,O=IPA.REDACTED.COM > expires: 2022-04-06 13:53:13 PDT > key usage: > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment > eku: id-kp-clientAuth > pre-save command: /usr/lib/ipa/certmonger/stop_pkicad > post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "storageCert > cert-pki-kra" > track: yes > auto-renew: yes > ===== > > Where else should I be looking to try and understand/debug why the server > is rejecting itś own connection to itself? From my (albeit limited) > understanding thus far, all the requisite components are present and > accounted for, no? > > Do my apache logs of the following give any hints to anyone as to what > isn´t being trusted? > ===== > [Tue Jun 15 17:11:34.674975 2021] [ssl:error] [pid 31830:tid > 139703550412544] [client 2604:XXX::36:4001:58500] AH02039: Certificate > Verification: Error (19): self signed certificate in certificate chain > [Tue Jun 15 17:11:34.675088 2021] [ssl:error] [pid 31830:tid > 139703550412544] [client 2604:XXX::36:4001:58500] AH02261: Re-negotiation > handshake failed > [Tue Jun 15 17:11:34.675111 2021] [ssl:error] [pid 31830:tid > 139703550412544] SSL Library Error: error:1417C086:SSL > routines:tls_process_client_certificate:certificate verify failed > ===== > > -Chris > > > On 6/16/21 3:32 PM, Chris Moody via FreeIPA-users wrote: > > That was kinda my belief thus far as well that the hosts were not trusting > themselves - not 100% sure how things got here though. I have a hunch it > might be related to the initial deployment and the prior admin using an > outdated method to install/manage/renew the LE-certificates. > > ===== > # ipa-cacert-manage list > IPA.REDACTED.COM IPA CA > IPA.REDACTED.COM IPA CA > DSTRootCAX3 > letsencryptx3 > isrgrootx1 > lets-encrypt-r3-cross-signed > The ipa-cacert-manage command was successful > ===== > > How would I go about forcing re-installation of the host's own CA > certificate to ensure it's trust? > > Also, since these nodes are not running on an RPM-based distro, the > typical cert-store locations I have seen on other systems are not in the > same location(s) so I'm not sure totally sure every location to point > certutil to be able to examine each cert-store in depth as well (if that > might help diagnose further). I ask because I believe these were initially > built and then had the "https://github.com/freeipa/freeipa-letsencrypt/"; > <https://github.com/freeipa/freeipa-letsencrypt/> project used to > initially deploy the LE-certs - prior to the ipa-cacert-manage command > being the official path toward installing/managing these external > certificates. I know because this code had been git-pulled onto these > nodes, but it obviously doesn't work properly since this git project > manipulates the paths below directly instead of managing via the > ipa-cacert-manage command. > > ex> > /etc/httpd/alias/ (<=== not on these systems) > /etc/pki/pki-tomcat/alias/ > /etc/ipa/nssdb/ > /etc/dirsrv/slapd-IPA-REDACTED-COM/ > > > Checking that project's git page now though, I see their readme now > mentions /var/lib/ipa/certs/, where I just noticed cacert.pem. > > ===== > /var/lib/ipa/certs# openssl x509 -text -noout -in cacert.pem > Certificate: > Data: > Version: 3 (0x2) > Serial Number: 1 (0x1) > Signature Algorithm: sha256WithRSAEncryption > Issuer: O = IPA.REDACTED.COM, CN = Certificate Authority > Validity > Not Before: Apr 16 20:45:46 2020 GMT > Not After : Apr 16 20:45:46 2040 GMT > Subject: O = IPA.REDACTED.COM, CN = Certificate Authority > ... > ===== > so I believe I might have just located the IPA CA cert in case I need to > re-install it. > > > > To the following question, I have the following LE-related certs > installed. And yes, I did run into issues a couple months back when LE > moved to the new certs on their end so had to import the new authority > certs to get the LE host certs to update & import. The LE certificates are > functioning and verify for both slapd and apache/tomcat. > > ===== > DSTRootCAX3.pem LetsEncryptAuthorityX3.pem isrgrootx1.pem > lets-encrypt-r3-cross-signed.pem > ===== > > Thank you all so much for the assistance through all this. > > -Chris > > > On 6/16/21 1:26 PM, Rob Crittenden via FreeIPA-users wrote: > > The error suggests that your > IPA server doesn't trust its own CA > certificate. > > Does ipa-cacert-manage list include the IPA CA? > > BTW the new certificate steps are unrelated. This affects all CA requests. > > rob > > > _______________________________________________ > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org > To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure >
_______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
