On 05/18/2017 03:49 PM, Michael Plemmons wrote:




*Mike Plemmons | Senior DevOps Engineer | CROSSCHX
*
614.427.2411
mike.plemm...@crosschx.com <mailto:mike.plemm...@crosschx.com>
www.crosschx.com <http://www.crosschx.com/>

On Thu, May 18, 2017 at 8:02 AM, Florence Blanc-Renaud <f...@redhat.com
<mailto:f...@redhat.com>> wrote:

    On 05/15/2017 08:33 PM, Michael Plemmons wrote:

        I have done more searching in my logs and I see the following
        errors.

        This is in the localhost log file /var/lib/pki/pki-tomcat/logs

        May 15, 2017 3:08:08 PM
        org.apache.catalina.core.ApplicationContext log
        SEVERE: StandardWrapper.Throwable
        java.lang.NullPointerException

        May 15, 2017 3:08:08 PM org.apache.catalina.core.StandardContext
        loadOnStartup
        SEVERE: Servlet [castart] in web application [/ca] threw load()
        exception
        java.lang.NullPointerException

        May 15, 2017 3:08:09 PM
        org.apache.catalina.core.StandardHostValve invoke
        SEVERE: Exception Processing /ca/admin/ca/getStatus
        javax.ws.rs <http://javax.ws.rs>
        <http://javax.ws.rs>.ServiceUnavailableException: Subsystem
        unavailable


        Looking at the debug log it says Authentication failed for port 636.

        [15/May/2017:17:39:25][localhost-startStop-1]: LdapAuthInfo: init()
        [15/May/2017:17:39:25][localhost-startStop-1]: LdapAuthInfo:
        init begins
        [15/May/2017:17:39:25][localhost-startStop-1]: LdapAuthInfo:
        init ends
        [15/May/2017:17:39:25][localhost-startStop-1]: init: before
        makeConnection errorIfDown is true
        [15/May/2017:17:39:25][localhost-startStop-1]: makeConnection:
        errorIfDown true
        [15/May/2017:17:39:25][localhost-startStop-1]:
        SSLClientCertificateSelectionCB: Setting desired cert nickname to:
        subsystemCert cert-pki-ca
        [15/May/2017:17:39:25][localhost-startStop-1]: LdapJssSSLSocket: set
        client auth cert nickname subsystemCert cert-pki-ca
        [15/May/2017:17:39:25][localhost-startStop-1]:
        SSLClientCertificatSelectionCB: Entering!
        [15/May/2017:17:39:25][localhost-startStop-1]:
        SSLClientCertificateSelectionCB: returning: null
        [15/May/2017:17:39:25][localhost-startStop-1]: SSL handshake
        happened
        Could not connect to LDAP server host ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>
        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>> port 636 Error
        netscape.ldap.LDAPException: Authentication failed (48)
                at
        
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConnection(LdapBoundConnFactory.java:205)


        I looked at the validity of the cert it mentions and it is fine.

        (root)>getcert status -v -d /etc/pki/pki-tomcat/alias -n
        'subsystemCert
        cert-pki-ca'
        State MONITORING, stuck: no.


        I then looked at the ldap errors around the time of this failure
        and I
        am seeing this log entry.


        [15/May/2017:17:38:42.063080758 +0000] set_krb5_creds - Could
        not get
        initial credentials for principal
        [ldap/ipa12.mgmt.crosschx....@mgmt.crosschx.com
        <mailto:ipa12.mgmt.crosschx....@mgmt.crosschx.com>
        <mailto:ipa12.mgmt.crosschx....@mgmt.crosschx.com
        <mailto:ipa12.mgmt.crosschx....@mgmt.crosschx.com>>] in keytab
        [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any
        KDC for
        requested realm)

        When I perform a klist against that keytab nothing appears out
        of the
        ordinary compared to working IPA servers.

        I am not sure what to look at next.


    Hi,

    you can try the following to manually replay the connection
    established by Dogtag to LDAP server:

    root$ export LDAPTLS_CACERTDIR=/etc/pki/pki-tomcat/alias
    root$ export LDAPTLS_CERT='subsystemCert cert-pki-ca'

    The above commands specify the NSSDB containing the user certificate
    and its name for SASL-EXTERNAL authentication.

    Then note the value obtained below as it will be used for the next
    step as the password to access the private key in the NSSDB:
    root$ grep internal /etc/pki/pki-tomcat/password.conf
    internal=<some value>

    root$ ldapsearch -H ldaps://`hostname`:636 -b "" -s base -Y EXTERNAL
    -Q -LLL dn namingcontexts
    Please enter pin, password, or pass phrase for security token
    'ldap(0)':                        <<<< here supply the value found above
    dn:
    namingcontexts: cn=changelog
    namingcontexts: dc=ipadomain,dc=com
    namingcontexts: o=ipaca



So I guess I found my problem.

(root)>ldapsearch -H ldaps://`hostname`:636 -b "" -s base -Y EXTERNAL -Q
-LLL dn namingcontexts
Please enter pin, password, or pass phrase for security token 'ldap(0)':
ldap_sasl_interactive_bind_s: Can't contact LDAP server (-1)
  additional info: TLS error -12195:Peer does not recognize and trust
the CA that issued your certificate.


I looked at our certs in /etc/dirsrv/slapd-IPADOMAIN-COM and found the
following.

IPA12 - problem server
(root)>certutil -L -d /etc/dirsrv/slapd-IPADOMAIN-COM

Certificate Nickname                                         Trust
Attributes

 SSL,S/MIME,JAR/XPI

Server-Cert                                                  u,u,u
IPADOMAIN-COM IPA CA                                     C,,



IPA11/IPA13 - 11 was the master and 13 is the new master
(root)>certutil -L -d /etc/dirsrv/slapd-IPADOMAIN-COM

Certificate Nickname                                         Trust
Attributes

 SSL,S/MIME,JAR/XPI

Server-Cert                                                  u,u,u
IPADOMAIN-COM IPA CA                                     CT,C,C



Good news! In this case the fix is trivial:
root$ certutil -M -d /etc/dirsrv/slapd-IPADOMAIN-COM -n 'IPADOMAIN-COM IPA CA' -t CT,C,C

Flo.

    In the LDAP server access log (in
    /etc/dirsrv/slapd-IPADOMAIN.COM/access), you should see the
    corresponding connection:

    [18/May/2017:13:35:14.822090417 +0200] conn=297 fd=150 slot=150 SSL
    connection from xxx to yyy
    [18/May/2017:13:35:15.789414017 +0200] conn=297 TLS1.2 128-bit
    AES-GCM; client CN=CA Subsystem,O=IPADOMAIN.COM
    <http://IPADOMAIN.COM>; issuer CN=Certificate
    Authority,O=IPADOMAIN.COM <http://IPADOMAIN.COM>
    [18/May/2017:13:35:15.793108509 +0200] conn=297 TLS1.2 client bound
    as uid=pkidbuser,ou=people,o=ipaca
    [18/May/2017:13:35:15.798101505 +0200] conn=297 op=0 BIND dn=""
    method=sasl version=3 mech=EXTERNAL
    [18/May/2017:13:35:15.800322076 +0200] conn=297 op=0 RESULT err=0
    tag=97 nentries=0 etime=0 dn="uid=pkidbuser,ou=people,o=ipaca"

    HTH,
    Flo.




        *Mike Plemmons | Senior DevOps Engineer | CROSSCHX
        *
        614.427.2411
        mike.plemm...@crosschx.com <mailto:mike.plemm...@crosschx.com>
        <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>>
        www.crosschx.com <http://www.crosschx.com>
        <http://www.crosschx.com/>

        On Wed, May 10, 2017 at 3:35 PM, Michael Plemmons
        <michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>
        <mailto:michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>>>
        wrote:

            The PKI service came up successfully but only when it uses
        BasicAuth
            rather than SSL auth.  I am not sure about what I need to do in
            order to get the auth working over SSL again.

            None of the certs are expired when I run getcert list and
            ipa-getcert list.

            Since the failure is with attempts to login to LDAP over 636.  I
            have been attempting to auth to LDAP via port 636 and the
        ldapsearch
            is not completing.  When looking at packet captures, I see
        some the
            TCP handshake and what appears to be the start of a SSL
        process and
            then everything hangs.

            What is the proper method to test performing a ldapsearch
        over 636?
            Also, the CS.cfg shows it wants to auth as cn=Directory
        Manager.  I
            can successfully auth with cn=Directory Manager over 389 but
        I think
            I am not performing ldapsearch over 636 correctly.



            *Mike Plemmons | Senior DevOps Engineer | CROSSCHX
            *
            614.427.2411
            mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>
        <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>>
            www.crosschx.com <http://www.crosschx.com>
        <http://www.crosschx.com/>

            On Fri, May 5, 2017 at 3:33 PM, Michael Plemmons
            <michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>
            <mailto:michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>>> wrote:

                I think I found the email thread.  Asking for help with
        crashed
                freeIPA istance.  That email pointed to this
                link,
        https://www.redhat.com/archives/freeipa-users/2017-January/msg00215.html
        
<https://www.redhat.com/archives/freeipa-users/2017-January/msg00215.html>

        
<https://www.redhat.com/archives/freeipa-users/2017-January/msg00215.html
        
<https://www.redhat.com/archives/freeipa-users/2017-January/msg00215.html>>.
                That link talked about changing the CS.cfg file to use
        port 389
                for PKI to auth to LDAP.  I made the necessary changes
        and PKI
                came up successfully.



                *Mike Plemmons | Senior DevOps Engineer | CROSSCHX
                *
                614.427.2411
                mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>
        <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>>
                www.crosschx.com <http://www.crosschx.com>
        <http://www.crosschx.com/>

                On Fri, May 5, 2017 at 3:19 PM, Michael Plemmons
                <michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>
                <mailto:michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>>> wrote:





                    *Mike Plemmons | Senior DevOps Engineer | CROSSCHX
                    *
                    614.427.2411
                    mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>
        <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>>
                    www.crosschx.com <http://www.crosschx.com>
        <http://www.crosschx.com/>

                    On Fri, May 5, 2017 at 3:15 PM, Rob Crittenden
                    <rcrit...@redhat.com <mailto:rcrit...@redhat.com>
        <mailto:rcrit...@redhat.com <mailto:rcrit...@redhat.com>>> wrote:

                        Michael Plemmons wrote:
                        > I just realized that I sent the reply directly
        to Rob
                        and not to the
                        > list.  My response is inline

                        Ok, this is actually good news.

                        I made a similar proposal in another case and I was
                        completely wrong.
                        Flo had the user do something and it totally
        fixed their
                        auth error, I
                        just can't remember what it was or find the e-mail
                        thread. I'm pretty
                        sure it was this calendar year though.

                        rob


                    Do you or Flo know what I could search for in the past
                    emails to find the answer to the problem?



                        >
                        >
                        >
                        > *Mike Plemmons | Senior DevOps Engineer | CROSSCHX
                        > *
                        > 614.427.2411
                        > mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>
                        <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>>
                        <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>
                        <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>>>
                        > www.crosschx.com <http://www.crosschx.com>
        <http://www.crosschx.com>
                        <http://www.crosschx.com/>
                        >
                        > On Thu, May 4, 2017 at 9:39 AM, Michael Plemmons
                        > <michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>
                        <mailto:michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>>
                        <mailto:michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>
                        <mailto:michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>>>>
                        > wrote:
                        >
                        >
                        >
                        >
                        >
                        >     *Mike Plemmons | Senior DevOps Engineer |
        CROSSCHX
                        >     *
                        >     614.427.2411
                        >     mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>
                        <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>>
                        <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>
                        <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>>>
                        >     www.crosschx.com <http://www.crosschx.com>
        <http://www.crosschx.com>
                        <http://www.crosschx.com/>
                        >
                        >     On Thu, May 4, 2017 at 9:24 AM, Rob Crittenden
                        <rcrit...@redhat.com
        <mailto:rcrit...@redhat.com> <mailto:rcrit...@redhat.com
        <mailto:rcrit...@redhat.com>>
                        >     <mailto:rcrit...@redhat.com
        <mailto:rcrit...@redhat.com>
                        <mailto:rcrit...@redhat.com
        <mailto:rcrit...@redhat.com>>>> wrote:
                        >
                        >         Michael Plemmons wrote:
                        >         > I realized that I was not very clear
        in my
                        statement about
                        >         testing with
                        >         > ldapsearch.  I had initially run it
        without
                        logging in with a
                        >         DN.  I was
                        >         > just running the local ldapsearch -x
                        command.  I then tested on
                        >         > ipa12.mgmt and ipa11.mgmt logging in
        with a
                        full DN for the
                        >         admin and
                        >         > "cn=Directory Manager" from ipa12.mgmt
                        (broken server) and
                        >         ipa11.mgmt
                        >         > and both ldapsearch command succeeded.
                        >         >
                        >         > I ran the following from ipa12.mgmt and
                        ipa11.mgmt as a non
                        >         root user.
                        >         > I also ran the command showing a
        line count
                        for the output and
                        >         the line
                        >         > counts for each were the same when
        run from
                        ipa12.mgmt and
                        >         ipa11.mgmt.
                        >         >
                        >         > ldapsearch -LLL -h
        ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>>
                        >         <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>>>
                        >         > <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>>
                        >         <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>>>> -D "DN" -w PASSWORD -b
                        >         >
                        "cn=users,cn=accounts,dc=mgmt,dc=crosschx,dc=com" dn
                        >         >
                        >         > ldapsearch -LLL -h
        ipa12.mgmt.crosschx.com <http://ipa12.mgmt.crosschx.com>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>>
                        >         <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>>>
                        >         > <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>>
                        >         <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>>>> -D "cn=directory
                        manager" -w
                        >         PASSWORD dn
                        >
                        >         The CA has its own suffix and replication
                        agreements. Given the auth
                        >         error and recent (5 months) renewal of CA
                        credentials I'd check
                        >         that the
                        >         CA agent authentication entries are
        correct.
                        >
                        >         Against each master with a CA run:
                        >
                        >         $ ldapsearch -LLL -x -D 'cn=directory
        manager'
                        -W -b
                        >         uid=ipara,ou=people,o=ipaca description
                        >
                        >         The format is 2;serial#,subject,issuer
                        >
                        >         Then on each run:
                        >
                        >         # certutil -L -d /etc/httpd/alias -n
        ipaCert
                        |grep Serial
                        >
                        >         The serial # should match that in the
                        description everywhere.
                        >
                        >         rob
                        >
                        >
                        >
                        >     On the CA (IPA13.MGMT) I ran the ldapsearch
                        command and see that the
                        >     serial number is 7.  I then ran the certutil
                        command on all three
                        >     servers and the serial number is 7 as well.
                        >
                        >
                        >     I also ran the ldapsearch command against the
                        other two servers and
                        >     they also showed a serial number of 7.
                        >
                        >
                        >
                        >
                        >         >
                        >         >
                        >         >
                        >         >
                        >         >
                        >         > *Mike Plemmons | Senior DevOps
        Engineer |
                        CROSSCHX
                        >         > *
                        >         > 614.427.2411
                        >         > mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>
                        <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>>
                        <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>
                        <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>>>
                        >         <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>
                        <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>>
                        >         <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>
                        <mailto:mike.plemm...@crosschx.com
        <mailto:mike.plemm...@crosschx.com>>>>
                        >         > www.crosschx.com
        <http://www.crosschx.com> <http://www.crosschx.com>
                        <http://www.crosschx.com>
                        >         <http://www.crosschx.com/>
                        >         >
                        >         > On Wed, May 3, 2017 at 5:28 PM,
        Michael Plemmons
                        >         > <michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>
                        <mailto:michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>>
                        >         <mailto:michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>
                        <mailto:michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>>>
                        >         <mailto:michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>
                        <mailto:michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>>
                        >         <mailto:michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>
                        <mailto:michael.plemm...@crosschx.com
        <mailto:michael.plemm...@crosschx.com>>>>>
                        >         > wrote:
                        >         >
                        >         >     I have a three node IPA cluster.
                        >         >
                        >         >     ipa11.mgmt - was a master over 6
        months ago
                        >         >     ipa13.mgmt - current master
                        >         >     ipa12.mgmt
                        >         >
                        >         >     ipa13 has agreements with ipa11 and
                        ipa12.  ipa11 and
                        >         ipa12 do not
                        >         >     have agreements between each other.
                        >         >
                        >         >     It appears that either
        ipa12.mgmt lost
                        some level of its
                        >         replication
                        >         >     agreement with ipa13.  I saw
        some level
                        because users /
                        >         hosts were
                        >         >     replicated between all systems
        but we
                        started seeing DNS
                        >         was not
                        >         >     resolving properly from ipa12.
        I do not
                        know when this
                        >         started.
                        >         >
                        >         >     When looking at replication
        agreements
                        on ipa12 I did not
                        >         see any
                        >         >     agreement with ipa13.
                        >         >
                        >         >     When I run ipa-replica-manage
        list all
                        three hosts show
                        >         has master.
                        >         >
                        >         >     When I run ipa-replica-manage
        ipa11.mgmt
                        I see ipa13.mgmt
                        >         is a replica.
                        >         >
                        >         >     When I run ipa-replica-manage
        ipa12.mgmt
                        nothing returned.
                        >         >
                        >         >     I ran ipa-replica-manage connect
                        --cacert=/etc/ipa/ca.crt
                        >         >     ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>>>
                        >         <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>>>>
                        >         >     ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>
                        <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>>
                        <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>
                        <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>>>
                        >         <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>
                        <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>>
                        >         <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>
                        <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>>>> on ipa12.mgmt
                        >         >
                        >         >     I then ran the following
                        >         >
                        >         >     ipa-replica-manage force-sync --from
                        >         ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>
                        <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>>
                        <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>
                        <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>>>
                        >         >     <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>
                        <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>>
                        >         <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>
                        <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>>>>
                        >         >
                        >         >     ipa-replica-manage re-initialize
        --from
                        >         ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>
                        <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>>
                        <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>
                        <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>>>
                        >         >     <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>
                        <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>>
                        >         <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>
                        <http://ipa13.mgmt.crosschx.com
        <http://ipa13.mgmt.crosschx.com>>>>
                        >         >
                        >         >     I was still seeing bad DNS
        returns when
                        dig'ing against
                        >         ipa12.mgmt.
                        >         >     I was able to create user and DNS
                        records and see the
                        >         information
                        >         >     replicated properly across all
        three nodes.
                        >         >
                        >         >     I then ran ipactl stop on
        ipa12.mgmt and
                        then ipactl start on
                        >         >     ipa12.mgmt because I wanted to
        make sure
                        everything was
                        >         running
                        >         >     fresh after the changes above.
        While
                        IPA was staring up (DNS
                        >         >     started) we were able to see
        valid DNS
                        queries returned but
                        >         >     pki-tomcat would not start.
                        >         >
                        >         >     I am not sure what I need to do
        in order
                        to get this
                        >         working.  I
                        >         >     have included the output of
        certutil and
                        getcert below
                        >         from all
                        >         >     three servers as well as the debug
                        output for pki.
                        >         >
                        >         >
                        >         >     While the IPA system is coming
        up I am
                        able to
                        >         successfully run
                        >         >     ldapsearch -x as the root user
        and see
                        results.  I am also
                        >         able to
                        >         >     login with the "cn=Directory
        Manager"
                        account and see results.
                        >         >
                        >         >
                        >         >     The debug log shows the
        following error.
                        >         >
                        >         >
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
                        >         >
         ============================================
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        =====  DEBUG
                        >         >     SUBSYSTEM INITIALIZED   =======
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
                        >         >
         ============================================
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         restart at
                        >         >     autoShutdown? false
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         >     autoShutdown crumb file path?
                        >         >
                         /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         about to
                        >         >     look for cert for auto-shutdown
                        support:auditSigningCert
                        >         cert-pki-ca
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         found
                        >         >     cert:auditSigningCert cert-pki-ca
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         done init
                        >         >     id=debug
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         >     initialized debug
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         >     initSubsystem id=log
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         ready to
                        >         >     init id=log
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        Creating
                        >         >
                        >

        RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/signedAudit/ca_audit)
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        Creating
                        >         >

         RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/system)
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        Creating
                        >         >

         RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/transactions)
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         restart at
                        >         >     autoShutdown? false
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         >     autoShutdown crumb file path?
                        >         >
                         /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         about to
                        >         >     look for cert for auto-shutdown
                        support:auditSigningCert
                        >         cert-pki-ca
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         found
                        >         >     cert:auditSigningCert cert-pki-ca
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         done init
                        >         >     id=log
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         >     initialized log
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         >     initSubsystem id=jss
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         ready to
                        >         >     init id=jss
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         restart at
                        >         >     autoShutdown? false
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         >     autoShutdown crumb file path?
                        >         >
                         /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         about to
                        >         >     look for cert for auto-shutdown
                        support:auditSigningCert
                        >         cert-pki-ca
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         found
                        >         >     cert:auditSigningCert cert-pki-ca
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         done init
                        >         >     id=jss
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         >     initialized jss
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         >     initSubsystem id=dbs
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        CMSEngine:
                        >         ready to
                        >         >     init id=dbs
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
                        >         DBSubsystem: init()
                        >         >      mEnableSerialMgmt=true
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        Creating
                        >         >     LdapBoundConnFactor(DBSubsystem)
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
                        >         LdapBoundConnFactory:
                        >         >     init
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
                        >         >     LdapBoundConnFactory:doCloning true
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
                        >         LdapAuthInfo: init()
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
                        >         LdapAuthInfo: init begins
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
                        >         LdapAuthInfo: init ends
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
        init: before
                        >         >     makeConnection errorIfDown is true
                        >         >
                         [03/May/2017:21:22:01][localhost-startStop-1]:
                        makeConnection:
                        >         >     errorIfDown true
                        >         >
                         [03/May/2017:21:22:02][localhost-startStop-1]:
                        >         >     SSLClientCertificateSelectionCB:
        Setting
                        desired cert
                        >         nickname to:
                        >         >     subsystemCert cert-pki-ca
                        >         >
                         [03/May/2017:21:22:02][localhost-startStop-1]:
                        >         LdapJssSSLSocket: set
                        >         >     client auth cert nickname
        subsystemCert
                        cert-pki-ca
                        >         >
                         [03/May/2017:21:22:02][localhost-startStop-1]:
                        >         >     SSLClientCertificatSelectionCB:
        Entering!
                        >         >
                         [03/May/2017:21:22:02][localhost-startStop-1]:
                        >         >     SSLClientCertificateSelectionCB:
                        returning: null
                        >         >
                         [03/May/2017:21:22:02][localhost-startStop-1]: SSL
                        >         handshake happened
                        >         >     Could not connect to LDAP server
        host
                        >         ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>>>
                        >         >     <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>>
                        >         <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>
                        <http://ipa12.mgmt.crosschx.com
        <http://ipa12.mgmt.crosschx.com>>>> port 636 Error
                        >         >     netscape.ldap.LDAPException:
                        Authentication failed (48)
                        >         >       at
                        >         >
                        >

        
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConnection(LdapBoundConnFactory.java:205)
                        >         >       at
                        >         >
                        >

        
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:166)
                        >         >       at
                        >         >
                        >

        
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:130)
                        >         >       at
                        >

         com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:654)
                        >         >       at
                        >         >
                        >

        com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.java:1169)
                        >         >       at
                        >         >
                        >

        com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine.java:1075)
                        >         >       at
                        >

         com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)
                        >         >       at
                        com.netscape.certsrv.apps.CMS.init(CMS.java:187)
                        >         >       at
                        com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
                        >         >       at
                        >         >
                        >

        
com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114)
                        >         >       at
                        >

         javax.servlet.GenericServlet.init(GenericServlet.java:158)
                        >         >       at
                        sun.reflect.NativeMethodAccessorImpl.invoke0(Native
                        >         Method)
                        >         >       at
                        >         >
                        >

        
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                        >         >       at
                        >         >
                        >

        
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                        >         >       at
                        java.lang.reflect.Method.invoke(Method.java:498)
                        >         >       at
                        >         >
                        >

        org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
                        >         >       at
                        >         >
                        >

        org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
                        >         >       at
                        java.security.AccessController.doPrivileged(Native
                        >         Method)
                        >         >       at
        javax.security.auth.Subject.do
        <http://javax.security.auth.Subject.do>
                        <http://javax.security.auth.Subject.do
        <http://javax.security.auth.Subject.do>>
                        >         <http://javax.security.auth.Subject.do
        <http://javax.security.auth.Subject.do>
                        <http://javax.security.auth.Subject.do
        <http://javax.security.auth.Subject.do>>>AsPrivileged(Subject.java:549)
                        >         >       at
                        >         >
                        >

        org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
                        >         >       at
                        >         >
                        >

        
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
                        >         >       at
                        >         >
                        >

        
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:124)
                        >         >       at
                        >         >
                        >

        
org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1270)
                        >         >       at
                        >         >
                        >

        
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1195)
                        >         >       at
                        >         >
                        >

        org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1085)
                        >         >       at
                        >         >
                        >

        
org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5318)
                        >         >       at
                        >         >
                        >

        
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5610)
                        >         >       at
                        >         >
                        >

        org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
                        >         >       at
                        >         >
                        >

        
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)
                        >         >       at
                        >         >
                        >

        
org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133)
                        >         >       at
                        >         >
                        >

        
org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156)
                        >         >       at
                        >         >
                        >

        
org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145)
                        >         >       at
                        java.security.AccessController.doPrivileged(Native





--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Reply via email to