Thanks, Florence and Alexander. @Alexander Bokovoy <aboko...@redhat.com> ,
While we were debugging in one of a blog we came across a scenario where NSS DB had got corrupted during a certificate renewal, so wanted to eliminate that issue. We see all the private key. [root@ipa1 nikita.d]# certutil -K -d /etc/pki/pki-tomcat/alias -f /tmp/pwdfile.txt certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa b8763cec560cf751cdafa5b0006af9405bdfabf0 NSS Certificate DB:subsystemCert cert-pki-ca < 1> rsa c7d8b4bf5f7d60de906444744a3b512c801676d2 (orphan) < 2> rsa 49ddc4aff2ae7a59ed5c3a939217d006d059fea2 NSS Certificate DB:Server-Cert cert-pki-ca < 3> rsa 12717b4e7fec1f408c947015b069e94838198947 NSS Certificate DB:auditSigningCert cert-pki-ca < 4> rsa 249cfa8ef238a902bd45ce397eda0a8ce8dda01d caSigningCert cert-pki-ca < 5> rsa 079bf91860780244f89ee9509853ed3e975ca11d NSS Certificate DB:ocspSigningCert cert-pki-ca We will try renewing the HTTP cert in our environment, will let you know once the cert is updated successfully. https://access.redhat.com/solutions/3357261 Regards Nikita S On Fri, Nov 8, 2019 at 3:42 PM Florence Blanc-Renaud <f...@redhat.com> wrote: > On 11/7/19 11:16 AM, Nikita Deeksha via FreeIPA-users wrote: > > Thanks for the update Alexander will check this and get back to you, > > wanted to check on another thing as well. > > > > Can you please help us to understand this error that we see for the cert > > in pki > > > > [root@ipa1 nikita.d]# for i in $(certutil -d /etc/pki/pki-tomcat/alias > > -L | grep cert-pki-ca | awk'{print $1}');do certutil -K -d > > /etc/pki/pki-tomcat/alias -f /tmp/pwdfile.txt -n "$i cert-pki-ca";done > > > > certutil: Checking token "NSS Certificate DB" in slot "NSS User Private > > Key and Certificate Services" > > > > certutil: problem listing keys: SEC_ERROR_UNRECOGNIZED_OID: Unrecognized > > Object Identifier. > > > > certutil: Checking token "NSS Certificate DB" in slot "NSS User Private > > Key and Certificate Services" > > > > certutil: problem listing keys: SEC_ERROR_UNRECOGNIZED_OID: Unrecognized > > Object Identifier. > > > > certutil: Checking token "NSS Certificate DB" in slot "NSS User Private > > Key and Certificate Services" > > > > < 0> rsa 249cfa8ef238a902bd45ce397eda0a8ce8dda01d caSigningCert > > cert-pki-ca > > > > certutil: Checking token "NSS Certificate DB" in slot "NSS User Private > > Key and Certificate Services" > > > > certutil: problem listing keys: SEC_ERROR_UNRECOGNIZED_OID: Unrecognized > > Object Identifier. > > > > certutil: Checking token "NSS Certificate DB" in slot "NSS User Private > > Key and Certificate Services" > > > > certutil: problem listing keys: SEC_ERROR_UNRECOGNIZED_OID: Unrecognized > > Object Identifier. > > > Hi, > > If you use "certutil -K -d /etc/pki/pki-tomcat/alias -f > /tmp/pwdfile.txt" (without the -n alias option), you will see all the > keys present in the database and notice that some of them have a > prefixed nickname, for instance: > 'NSS Certificate DB:Server-Cert cert-pki-ca' > instead of 'Server-Cert cert-pki-ca'. > You need to provide this prefixed name with -K -n nickname. > > HTH, > flo > > > > > > These are the cert which is present /etc/pki/pki-tomcat/alias > > > > [root@ipa1 nikita.d]# certutil -L -d /etc/pki/pki-tomcat/alias > > > > Certificate Nickname Trust > > Attributes > > > > SSL,S/MIME,JAR/XPI > > > > COMODO CA BUNDLE CT,C,C > > ocspSigningCert cert-pki-ca u,u,u > > auditSigningCert cert-pki-ca u,u,Pu > > caSigningCert cert-pki-ca CTu,Cu,Cu > > subsystemCert cert-pki-ca u,u,u > > Server-Cert cert-pki-ca u,u,u > > > > > > > > Regards > > > > Nikita S > > > > > > On Wed, Nov 6, 2019 at 9:52 PM Alexander Bokovoy <aboko...@redhat.com > > <mailto:aboko...@redhat.com>> wrote: > > > > On ke, 06 marras 2019, Nikita Deeksha via FreeIPA-users wrote: > > >2. > > > > > >Status SUBMITTING means the renewal is not yet completed. It will > not > > >complete until you get Dogtag working. > > > > > >But now the status says CA_UNEACHABLE > > > > > >Request ID '20180412150739': > > >status: CA_UNREACHABLE > > >ca-error: Server at https://ipa1.xxx.xxxx.com/ipa/xml failed > > request, will > > >retry: -504 (libcurl failed to execute the HTTP POST transaction, > > >explaining: Peer's Certificate has expired.). > > > > This is exactly an issue with expired HTTP certificate. > > I guess you'd need to roll back time to when the certificate was > valid > > (before 2019-10-25) and restart certmonger. > > > > See discussion in this thread: > > > https://www.redhat.com/archives/freeipa-users/2016-July/msg00270.html > > > > In newer RHEL version (RHEL 7.7) there is a special tool, > ipa-cert-fix, > > that can help with fixing these issues. However, since you are on the > > version before it, you need to do manual renewal. > > > > >stuck: no > > >key pair storage: > type=NSSDB,location='/etc/httpd/alias',nickname='CN= > > >ipa1.xxx.xxxx.com <http://ipa1.xxx.xxxx.com>,O=CORP.ENDURANCE.COM > > <http://CORP.ENDURANCE.COM>',token='NSS Certificate > > >DB',pinfile='/etc/httpd/alias/pwdfile.txt' > > >certificate: type=NSSDB,location='/etc/httpd/alias',nickname='CN= > > >ipa1.xxx.xxxx.com <http://ipa1.xxx.xxxx.com>,O=CORP.ENDURANCE.COM > > <http://CORP.ENDURANCE.COM>',token='NSS Certificate DB' > > >CA: IPA > > >issuer: CN=Certificate Authority,O=xxx.xxxx.COM < > http://xxx.xxxx.COM> > > >subject: CN=ipa1.xxxx.xxxxx.com > > <http://ipa1.xxxx.xxxxx.com>,O=xxx.xxxx.COM <http://xxx.xxxx.COM> > > >expires: 2019-10-25 20:16:38 UTC > > >principal name: krbtgt/xxx.xxxxx....@xxx.xxxx.com > > <mailto:xxx.xxxxx....@xxx.xxxx.com> > > >key usage: > > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment > > >eku: id-kp-serverAuth,id-pkinit-KPKdc > > >pre-save command: > > >post-save command: /usr/libexec/ipa/certmonger/restart_httpd > > >track: yes > > > > > > > > > > > > > > >*Issue2:* > > > > > >We are getting this alert while we log in to UI in httpd error logs > > > > > > > > >#ipa: INFO: 401 Unauthorized: kinit: Preauthentication failed > > while getting > > >initial credentials > > > > > >and PKINIT was disabled > > > > > >[root@ipa2 httpd]# ipa-pkinit-manage status > > >PKINIT is disabled > > > > > >While I tried to enable this > > > > > >[root@ipa2 httpd]# ipa-pkinit-manage enable > > >Configuring Kerberos KDC (krb5kdc) > > > [1/1]: installing X509 Certificate for PKINIT > > > > > >the process was getting stuck, so I had to terminate it manually. > > After > > >trying to enable, I'm getting "Login failed due to an unknown > reason." > > >error in web UI when I try to login > > > > > >*Error in httpd:* > > > > > >[Wed Nov 06 10:13:37.465318 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] mod_wsgi (pid=24416): > > Exception occurred processing WSGI > > >script '/usr/share/ipa/wsgi.py'. > > >[Wed Nov 06 10:13:37.465433 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] Traceback (most recent > > call last): > > >[Wed Nov 06 10:13:37.468706 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] File > > "/usr/share/ipa/wsgi.py", line 59, in application > > >[Wed Nov 06 10:13:37.470083 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] return > > api.Backend.wsgi_dispatch(environ, > > >start_response) > > >[Wed Nov 06 10:13:37.470146 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] File > > >"/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line > > 267, in > > >__call__ > > >[Wed Nov 06 10:13:37.473376 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] return > > self.route(environ, start_response) > > >[Wed Nov 06 10:13:37.473437 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] File > > >"/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line > > 279, in > > >route > > >[Wed Nov 06 10:13:37.473462 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] return app(environ, > > start_response) > > >[Wed Nov 06 10:13:37.473475 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] File > > >"/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line > > 937, in > > >__call__ > > >[Wed Nov 06 10:13:37.476093 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] > > self.kinit(user_principal, password, ipa_ccache_name) > > >[Wed Nov 06 10:13:37.478843 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] File > > >"/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line > > 973, in > > >kinit > > >[Wed Nov 06 10:13:37.478864 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] > > pkinit_anchors=[paths.KDC_CERT, > > >paths.KDC_CA_BUNDLE_PEM], > > >[Wed Nov 06 10:13:37.478878 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] File > > >"/usr/lib/python2.7/site-packages/ipalib/install/kinit.py", line > > 127, in > > >kinit_armor > > >[Wed Nov 06 10:13:37.484351 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] run(args, env=env, > > raiseonerr=True, capture_error=True) > > >[Wed Nov 06 10:13:37.484381 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] File > > >"/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 562, > > in run > > >[Wed Nov 06 10:13:37.487470 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] raise > > CalledProcessError(p.returncode, arg_string, > > >str(output)) > > >[Wed Nov 06 10:13:37.488932 2019] [:error] [pid 24416] [remote > > >172.27.10.113:0 <http://172.27.10.113:0>] CalledProcessError: > > Command '/usr/bin/kinit -n -c > > >/var/run/ipa/ccaches/armor_24416 -X > > >X509_anchors=FILE:/var/kerberos/krb5kdc/kdc.crt -X > > >X509_anchors=FILE:/var/lib/ipa-client/pki/kdc-ca-bundle.pem' > returned > > >non-zero exit status 1 > > > > > >And when I try to list the certificates using *getcert list,* > > there is a > > >new cert which was added > > > > > >Request ID '20191106100258': > > >status: CA_UNREACHABLE > > >ca-error: Server at https://ipa2.xxx.xxxxx.com/ipa/xml failed > > request, will > > >retry: 907 (RPC failed at server. cannot connect to ' > > >https://ipa1.xxx.xxxxx.com:443/ca/rest/account/login': [SSL: > > >CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)). > > >stuck: no > > >key pair storage: > type=FILE,location='/var/kerberos/krb5kdc/kdc.key' > > >certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt' > > >CA: IPA > > >issuer: > > >subject: > > >expires: unknown > > >pre-save command: > > >post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert > > >track: yes > > >auto-renew: yes > > > > > > > > >Regards > > >Nikita S > > > > > >When > > >On Wed, Nov 6, 2019 at 6:39 PM Alexander Bokovoy > > <aboko...@redhat.com <mailto:aboko...@redhat.com>> > > >wrote: > > > > > >> On ti, 05 marras 2019, Nikita Deeksha via FreeIPA-users wrote: > > >> >Hi Team, > > >> >We have 2 IPA servers in Mater-Master setup are we facing the > > below issue > > >> >on these servers. > > >> > > > >> >Isuue1: > > >> >Our httpd certificate has expired because of which our IPA1 UI > > wasn't > > >> >working, we are getting “*loging failed due to an unknown > > reason*” error > > >> >while we log in to the UI > > >> > > > >> > > > >> >1. First, the IPA console was not working as httpd service was > > stopped, > > >> >httpd was not starting as HTTP certificate is expired. Added > > >> >*NSSEnforceValidCerts > > >> >off* line in nss.conf to start the service. > > >> > > > >> >2. After the change IPA console was loading we are not able to > > login to > > >> the > > >> >console as pki-tomcatd service was not running, > > >> >[root@ipa1 ca]# ipactl status > > >> >Directory Service: RUNNING > > >> >krb5kdc Service: RUNNING > > >> >kadmin Service: RUNNING > > >> >httpd Service: RUNNING > > >> >ipa-custodia Service: RUNNING > > >> >ntpd Service: RUNNING > > >> >pki-tomcatd Service: STOPPED > > >> >ipa-otpd Service: RUNNING > > >> > > > >> ># systemctl status pki-tomcatd@pki-tomcat.service -l > > >> >● pki-tomcatd@pki-tomcat.service - PKI Tomcat Server pki-tomcat > > >> > Loaded: loaded (/lib/systemd/system/pki-tomcatd@.service; > > enabled; > > >> >vendor preset: disabled) > > >> > Active: active (running) since Tue 2019-11-05 10:16:50 GMT; > > 31min ago > > >> > Process: 97068 ExecStartPre=/usr/bin/pkidaemon start %i > > (code=exited, > > >> >status=0/SUCCESS) > > >> > Main PID: 97233 (java) > > >> > CGroup: > > >> > > > >/system.slice/system-pki\x2dtomcatd.slice/pki-tomcatd@pki-tomcat.service > > >> > └─97233 /usr/lib/jvm/jre-1.8.0-openjdk/bin/java > > >> >-DRESTEASY_LIB=/usr/share/java/resteasy-base > > >> >-Djava.library.path=/usr/lib64/nuxwdog-jni -classpath > > >> > > >> > > > >/usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar > > >> >-Dcatalina.base=/var/lib/pki/pki-tomcat > > -Dcatalina.home=/usr/share/tomcat > > >> >-Djava.endorsed.dirs= > -Djava.io.tmpdir=/var/lib/pki/pki-tomcat/temp > > >> > > >> > > > >-Djava.util.logging.config.file=/var/lib/pki/pki-tomcat/conf/logging.properties > > >> > >-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager > > >> >-Djava.security.manager > > >> > > > >-Djava.security.policy==/var/lib/pki/pki-tomcat/conf/catalina.policy > > >> >org.apache.catalina.startup.Bootstrap start > > >> > > > >> >Nov 05 10:47:57 ipa1.xxx.xxxxx.com <http://ipa1.xxx.xxxxx.com> > > server[97233]: WARNING: Exception > > >> >processing realm com.netscape.cms.tomcat.ProxyRealm@1896e072 > > background > > >> >process > > >> >Nov 05 10:47:57 ipa1.xxx.xxxxx.com <http://ipa1.xxx.xxxxx.com> > > server[97233]: > > >> >javax.ws.rs <http://javax.ws.rs>.ServiceUnavailableException: > > Subsystem unavailable > > >> >Nov 05 10:47:57 ipa1.xxx.xxxxx.com <http://ipa1.xxx.xxxxx.com> > > server[97233]: at > > >> > > > >com.netscape.cms.tomcat.ProxyRealm.backgroundProcess(ProxyRealm.java:137) > > >> >Nov 05 10:47:57 ipa1.xxx.xxxxx.com <http://ipa1.xxx.xxxxx.com> > > server[97233]: at > > >> > > >> > > > >org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1356) > > >> >Nov 05 10:47:57 ipa1.xxx.xxxxx.com <http://ipa1.xxx.xxxxx.com> > > server[97233]: at > > >> > > >> > > > >org.apache.catalina.core.StandardContext.backgroundProcess(StandardContext.java:5958) > > >> >Nov 05 10:47:57 ipa1.xxx.xxxxx.com <http://ipa1.xxx.xxxxx.com> > > server[97233]: at > > >> > > >> > > > >org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1542) > > >> >Nov 05 10:47:57 ipa1.xxx.xxxxx.com <http://ipa1.xxx.xxxxx.com> > > server[97233]: at > > >> > > >> > > > >org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1552) > > >> >Nov 05 10:47:57 ipa1.xxx.xxxxx.com <http://ipa1.xxx.xxxxx.com> > > server[97233]: at > > >> > > >> > > > >org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1552) > > >> >Nov 05 10:47:57 ipa1.xxx.xxxxx.com <http://ipa1.xxx.xxxxx.com> > > server[97233]: at > > >> > > >> > > > >org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1520) > > >> >Nov 05 10:47:57 ipa1.xxx.xxxxx.com <http://ipa1.xxx.xxxxx.com> > > server[97233]: at > > >> >java.lang.Thread.run(Thread.java:748) > > >> > > > >> > > > >> >This service wasn’t starting with this error > > >> > > > >> ># less /var/log/pki/pki-tomcat/ca/debug > > >> >31/Oct/2019:13:24:23][localhost-startStop-1]: > > >> >SSLClientCertificateSelectionCB: desired cert found in list: > > subsystemCert > > >> >cert-pki-ca > > >> >[31/Oct/2019:13:24:23][localhost-startStop-1]: > > >> >SSLClientCertificateSelectionCB: returning: subsystemCert > > cert-pki-ca > > >> >[31/Oct/2019:13:24:23][localhost-startStop-1]: SSL handshake > > happened > > >> >Could not connect to LDAP server host ipa1.xxx.xxxx.com > > <http://ipa1.xxx.xxxx.com> port 636 Error > > >> >netscape.ldap.LDAPException: Authentication failed (49) > > >> > > >> Authentication failed means the RA agent certificate dogtag uses > to > > >> authenticate to LDAP server is not the same as the one mentioned > > in the > > >> LDAP entry for RA agent. > > >> > > >> I think there was some procedure to fix it but I don't have > > links handy. > > >> Also, you did not specify what versions of FreeIPA you run. > > >> > > >> > > >> >at > > >> > > >> > > > >com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConnection(LdapBoundConnFactory.java:205) > > >> > at > > >> > > >> > > > >com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:166) > > >> > at > > >> > > >> > > > >com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:130) > > >> > at > > com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:654) > > >> > at > > >> > > > >com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.java:1176) > > >> > at > > >> > > > >com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine.java:1082) > > >> > at > > com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:572) > > >> > at com.netscape.certsrv.apps.CMS.init(CMS.java:189) > > >> > at com.netscape.certsrv.apps.CMS.start(CMS.java:1631) > > >> > at > > >> > > >> > > > >com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:117) > > >> > at > > javax.servlet.GenericServlet.init(GenericServlet.java:158) > > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > > Method) > > >> > at > > >> > > >> > > > >sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > >> > at > > >> > > >> > > > >sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > >> > at java.lang.reflect.Method.invoke(Method.java:498) > > >> > at > > >> > > > >org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288) > > >> > at > > >> > > > >org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285) > > >> > at java.security.AccessController.doPrivileged(Native > > Method) > > >> > at > > javax.security.auth.Subject.doAsPrivileged(Subject.java:549) > > >> > at > > >> > > > >org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320) > > >> > at > > >> > > >> > > > >org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175) > > >> > at > > >> > > >> > > > >org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:124) > > >> > at > > >> > > >> > > > >org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1257) > > >> > at > > >> > > >> > > > >org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1182) > > >> > at > > >> > > > >org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1072) > > >> > at > > >> > > >> > > > >org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5368) > > >> > at > > >> > > >> > > > >org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5660) > > >> > at > > >> > > > >org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) > > >> > at > > >> > > >> > > > >org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899) > > >> > at > > >> > > > >org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133) > > >> > at > > >> > > >> > > > >org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156) > > >> > at > > >> > > >> > > > >org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145) > > >> > at java.security.AccessController.doPrivileged(Native > > Method) > > >> > at > > >> > > > >org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873) > > >> > at > > >> > > > >org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652) > > >> > at > > >> > > >> > > > >org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:679) > > >> > at > > >> > > >> > > > >org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1966) > > >> > at > > >> > > > >java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > >> > at > java.util.concurrent.FutureTask.run(FutureTask.java:266) > > >> > at > > >> > > >> > > > >java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > >> > at > > >> > > >> > > > >java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > >> > at java.lang.Thread.run(Thread.java:748) > > >> >Internal Database Error encountered: Could not connect to LDAP > > server host > > >> >ipa1.xxx.xxx.com <http://ipa1.xxx.xxx.com> port 636 Error > > netscape.ldap.LDAPException: > > >> Authentication > > >> >failed (49) > > >> > at > > com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:676) > > >> > at > > >> > > > >com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.java:1176) > > >> > at > > >> > > > >com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine.java:1082) > > >> > at > > com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:572) > > >> > at com.netscape.certsrv.apps.CMS.init(CMS.java:189) > > >> > at com.netscape.certsrv.apps.CMS.start(CMS.java:1631) > > >> > at > > >> > > >> > > > >com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:117) > > >> > at > > javax.servlet.GenericServlet.init(GenericServlet.java:158) > > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > > Method) > > >> > at > > >> > > >> > > > >sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > >> > at > > >> > > >> > > > >sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > >> > at java.lang.reflect.Method.invoke(Method.java:498) > > >> > at > > >> > > > >org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288) > > >> > at > > >> > > > >org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285) > > >> > at java.security.AccessController.doPrivileged(Native > > Method) > > >> > at > > javax.security.auth.Subject.doAsPrivileged(Subject.java:549) > > >> > at > > >> > > > >org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320) > > >> > at > > >> > > >> > > > >org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175) > > >> > at > > >> > > >> > > > >org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:124) > > >> > at > > >> > > >> > > > >org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1257) > > >> > at > > >> > > >> > > > >org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1182) > > >> > at > > >> > > > >org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1072) > > >> > at > > >> > > >> > > > >org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5368) > > >> > at > > >> > > >> > > > >org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5660) > > >> > at > > >> > > > >org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) > > >> > at > > >> > > >> > > > >org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899) > > >> > at > > >> > > > >org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133) > > >> > at > > >> > > >> > > > >org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156) > > >> > at > > >> > > >> > > > >org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145) > > >> > at java.security.AccessController.doPrivileged(Native > > Method) > > >> > at > > >> > > > >org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873) > > >> > at > > >> > > > >org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652) > > >> > at > > >> > > >> > > > >org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:679) > > >> > at > > >> > > >> > > > >org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1966) > > >> > at > > >> > > > >java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > >> > at > java.util.concurrent.FutureTask.run(FutureTask.java:266) > > >> > at > > >> > > >> > > > >java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > >> > at > > >> > > >> > > > >java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > >> > at java.lang.Thread.run(Thread.java:748) > > >> > > > >> ># getcert list > > >> >Request ID '20180412150739': > > >> >status: SUBMITTING > > >> >stuck: no > > >> >key pair storage: > > type=NSSDB,location='/etc/httpd/alias',nickname='CN= > > >> >ipa1.xxxx.xxxxx.com <http://ipa1.xxxx.xxxxx.com>,O=xxx.xxxx.COM > > <http://xxx.xxxx.COM>',token='NSS Certificate > > >> >DB',pinfile='/etc/httpd/alias/pwdfile.txt' > > >> >certificate: > type=NSSDB,location='/etc/httpd/alias',nickname='CN= > > >> >ipa1.xxxx.xxxxx.com > > <http://ipa1.xxxx.xxxxx.com>,O=xxx.xxxxx.COM > > <http://xxx.xxxxx.COM>',token='NSS Certificate DB' > > >> >CA: IPA > > >> >issuer: CN=Certificate Authority,O=xxx.xxxxx.COM > > <http://xxx.xxxxx.COM> > > >> >subject: CN=ipa1.xxxx.xxxx.com > > <http://ipa1.xxxx.xxxx.com>,O=xxx.xxxxx.COM <http://xxx.xxxxx.COM> > > >> >expires: 2019-10-25 20:16:38 UTC > > >> >principal name: krbtgt/xxxx.xxxx....@xxxx.xxxx.com > > <mailto:xxxx.xxxx....@xxxx.xxxx.com> > > >> >key usage: > > >> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment > > >> >eku: id-kp-serverAuth,id-pkinit-KPKdc > > >> >pre-save command: > > >> >post-save command: /usr/libexec/ipa/certmonger/restart_httpd > > >> >track: yes > > >> >auto-renew: yes > > >> > > >> Status SUBMITTING means the renewal is not yet completed. It > > will not > > >> complete until you get Dogtag working. > > >> > > >> > > > >> >Issue2: > > >> > > > >> >On the IPA2 server, we are unable to login with the admin user > > credentials > > >> >without OTP, but when an AD user is trying to login with 2FA > (i.e, > > >> >password and OTP) we are getting this error *"The password you > > entered is > > >> >incorrect."* > > >> > > >> AD users cannot use multifactor authentication defined in IPA. > > >> > > >> > > >> ># [root@ipa2 log]# ipactl status > > >> >Directory Service: RUNNING > > >> >krb5kdc Service: RUNNING > > >> >kadmin Service: RUNNING > > >> >httpd Service: RUNNING > > >> >ipa-custodia Service: RUNNING > > >> >ntpd Service: RUNNING > > >> >ipa-otpd Service: STOPPED > > >> >ipa: INFO: The ipactl command was successful > > >> > > > >> ># systemctl status ipa-otpd.socket -l > > >> >● ipa-otpd.socket - ipa-otpd socket > > >> > Loaded: loaded (/usr/lib/systemd/system/ipa-otpd.socket; > > disabled; > > >> >vendor preset: disabled) > > >> > Active: failed (Result: resources) since Tue 2019-11-05 > > 08:19:04 GMT; > > >> 1h > > >> >31min ago > > >> > Listen: /var/run/krb5kdc/DEFAULT.socket (Stream) > > >> > Accepted: 2; Connected: 0 > > >> > > > >> >Nov 05 07:42:53 ipa2.xxxx.xxxx.com <http://ipa2.xxxx.xxxx.com> > > systemd[1]: Listening on ipa-otpd > > >> socket. > > >> >Nov 05 08:19:04 ipa2.xxxx.xxxx.com <http://ipa2.xxxx.xxxx.com> > > systemd[1]: ipa-otpd.socket failed to > > >> >queue service startup job (Maybe the service file is missing or > > not a > > >> >template unit?): Resource temporarily unavailable > > >> >Nov 05 08:19:04 ipa2.xxxx.xxxx.com <http://ipa2.xxxx.xxxx.com> > > systemd[1]: Unit ipa-otpd.socket > > >> entered > > >> >failed state. > > >> > > > >> ># cat /usr/lib/systemd/system/ipa-otpd.socket > > >> >[Unit] > > >> >Description=ipa-otpd socket > > >> > > > >> >[Socket] > > >> >ListenStream=/var/run/krb5kdc/DEFAULT.socket > > >> >RemoveOnStop=true > > >> >SocketMode=0600 > > >> >Accept=true > > >> > > > >> >[Install] > > >> >WantedBy=krb5kdc.service > > >> > > > >> > > > >> > > > >> >We see that data replication is broken between the 2 IPA > > servers, as the > > >> >changes made on IPA2 is not reflecting on IPA1 > > >> This is most likely because your LDAP server certificate expired > as > > >> well. > > >> > > >> > > >> >We the below errors as well. > > >> > > > >> >IPA1 > > >> >Nov 05 10:09:23 ipa1.xxx.xxxx.com <http://ipa1.xxx.xxxx.com> > > krb5kdc[28021](info): TGS_REQ (8 etypes > > >> >{18 17 20 19 16 23 25 26}) x.x.x.x: ISSUE: authtime 1572948563, > > etypes > > >> >{rep=18 tkt=18 ses=18}, ldap/ipa1.xxxxx.xxxx....@xxxx.xxxxx.com > > <mailto:ipa1.xxxxx.xxxx....@xxxx.xxxxx.com> for ldap/ > > >> >ipa2.xxxx.xxxx....@xxxx.xxxx.com > > <mailto:ipa2.xxxx.xxxx....@xxxx.xxxx.com> > > >> >Nov 05 10:14:24 ipa1.corp.endurance.com > > <http://ipa1.corp.endurance.com> krb5kdc[28021](info): TGS_REQ (8 > > >> >etypes {18 17 20 19 16 23 25 26}) x.x.x.x: ISSUE: authtime > > 1572948863, > > >> >etypes {rep=18 tkt=18 ses=18}, > > ldap/ipa1.xxxx.xxx....@xxxx.xxxx.com > > <mailto:ipa1.xxxx.xxx....@xxxx.xxxx.com> for > > >> >ldap/ipa2.xxxx.xxxx....@xxxx.xxxx.com > > <mailto:ipa2.xxxx.xxxx....@xxxx.xxxx.com> > > >> > > >> These aren't errors. They are normal operations: ldap/ipa1 > > service (LDAP > > >> server on IPA1) asked for a Kerberos service ticket to LDAP > > service on > > >> IPA2 and was granted it. This is just as it should be for > > replication. > > >> > > >> > > > >> >IPA2 > > >> ># tailf krb5kdc.log > > >> >Nov 05 09:59:25 ipa2.xxxx.xxxx.com <http://ipa2.xxxx.xxxx.com> > > krb5kdc[2451](info): AS_REQ (8 etypes > > >> >{18 17 20 19 16 23 25 26}) y.y.y.y: NEEDED_PREAUTH: ldap/ > > >> >ipa2.xxxx.xxxx....@xxx.xxxx.com > > <mailto:ipa2.xxxx.xxxx....@xxx.xxxx.com> for > > krbtgt/xxxx.xxxx....@xxxx.xxxx.com <mailto: > xxxx.xxxx....@xxxx.xxxx.com>, > > >> >Additional pre-authentication required > > >> >Nov 05 09:59:25 ipa2.xxxx.xxxx.com <http://ipa2.xxxx.xxxx.com> > > krb5kdc[2451](info): closing down fd > > >> 11 > > >> >Nov 05 09:59:25 ipa2.xxxx.xxxx.com <http://ipa2.xxxx.xxxx.com> > > krb5kdc[2451](info): AS_REQ (8 etypes > > >> >{18 17 20 19 16 23 25 26}) y.y.y.y: ISSUE: authtime 1572947965, > > etypes > > >> >{rep=18 tkt=18 ses=18}, ldap/ipa2.xxxx.xxxx....@xxx.xxxx.com > > <mailto:ipa2.xxxx.xxxx....@xxx.xxxx.com> for krbtgt/ > > >> >xxx.xxxx....@xxx.xxxx.com <mailto:xxx.xxxx....@xxx.xxxx.com> > > >> >Nov 05 09:59:25 ipa2.xxxx.xxxx.com <http://ipa2.xxxx.xxxx.com> > > krb5kdc[2451](info): closing down fd > > >> 11 > > >> >Nov 05 09:59:25 ipa2.xxxx.xxxx.com <http://ipa2.xxxx.xxxx.com> > > krb5kdc[2451](info): TGS_REQ (8 etypes > > >> >{18 17 20 19 16 23 25 26}) y.y.y.y: ISSUE: authtime 1572947965, > > etypes > > >> >{rep=18 tkt=18 ses=18}, ldap/ipa2.xxxx.xxxx....@xxxx.xxxx.com > > <mailto:ipa2.xxxx.xxxx....@xxxx.xxxx.com> for ldap/ > > >> >ipa2.xxxx.xxxx....@xxx.xxxx.com > > <mailto:ipa2.xxxx.xxxx....@xxx.xxxx.com> > > >> >Nov 05 09:59:25 ipa2.xxxx.xxxx.com <http://ipa2.xxxx.xxxx.com> > > krb5kdc[2451](info): closing down fd > > >> 11 > > >> > > >> Same here. LDAP server on IPA2 operated against itself here. > > >> > > >> -- > > >> / Alexander Bokovoy > > >> Sr. Principal Software Engineer > > >> Security / Identity Management Engineering > > >> Red Hat Limited, Finland > > >> > > >> > > > > > > -- > > / Alexander Bokovoy > > Sr. Principal Software Engineer > > Security / Identity Management Engineering > > Red Hat Limited, Finland > > > > > > _______________________________________________ > > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org > > To unsubscribe send an email to > freeipa-users-le...@lists.fedorahosted.org > > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > > List Archives: > https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org > > > >
_______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
