-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 07/28/2014 11:07 AM, Ade Lee wrote: > On Mon, 2014-07-28 at 08:26 -0700, Erinn Looney-Triggs wrote: >> On 07/28/2014 08:04 AM, Ade Lee wrote: >>> On Mon, 2014-07-28 at 07:41 -0700, Erinn Looney-Triggs wrote: >>>> On 07/28/2014 07:17 AM, Rob Crittenden wrote: >>>>> Rob Crittenden wrote: >>>>>> Erinn Looney-Triggs wrote: >>>>>>> On 07/27/2014 12:02 AM, Erinn Looney-Triggs wrote: >>>>>>>> On 07/26/2014 07:12 PM, Erinn Looney-Triggs wrote: >>>>>>>>> On 07/26/2014 05:25 PM, Erinn Looney-Triggs wrote: >>>>>>>>>> Well it hasn't been all the pretty trying to >>>>>>>>>> move from RHEL 6.5 to RHEL 7. >>>>>>> >>>>>>>>>> I have two servers providing my ipa instances >>>>>>>>>> ipa and ipa2. Given that I don't have a great >>>>>>>>>> deal of spare capacity the plan was to remove >>>>>>>>>> ipa2 from the replication agreement, modify DNS >>>>>>>>>> so that only IPA was available in SRV logs (IPA >>>>>>>>>> does not manage DNS at this point, was waiting >>>>>>>>>> for DNSSEC). As well, I would change my sudo-ldap >>>>>>>>>> config files to point to ipa and remove ipa2. >>>>>>> >>>>>>>>>> Well that all worked well, installed RHEL 7 on >>>>>>>>>> the system and began working through the steps in >>>>>>>>>> the upgrade guide. >>>>>>> >>>>>>>>>> First major problem was running into this bug: >>>>>>>>>> https://fedorahosted.org/freeipa/ticket/4375 >>>>>>>>>> ValueError: nsDS5ReplicaId has 2 values, one >>>>>>>>>> expected. >>>>>>> >>>>>>>>>> Went and patched the replication.py file to get >>>>>>>>>> around that issue, and we moved on. >>>>>>> >>>>>>>>>> Next up is my current issue: Exception from Java >>>>>>>>>> Configuration Servlet: Clone does not have all >>>>>>>>>> the required certificates. >>>>>>> >>>>>>>>>> I suspect this is because I am running the CA as >>>>>>>>>> a subordinate to an AD CS instance, but I am >>>>>>>>>> unsure at this point. >>>>>>> >>>>>>>>>> It has been a haul to get here, despite the short >>>>>>>>>> explanation. It seems that my primary ipa >>>>>>>>>> instance is working on only a hit or miss basis >>>>>>>>>> for kerberos tickets which has made all this a >>>>>>>>>> bit of a pain. You can kinit as admin once it >>>>>>>>>> will fail unable to find KDC, try again another >>>>>>>>>> three times, it will work. I have even modified >>>>>>>>>> the krb5.conf file to point directly at the >>>>>>>>>> server, thus bypassing DNS SRV lookups, however, >>>>>>>>>> that hasn't worked. >>>>>>> >>>>>>>>>> Point is, any help would be appreciated on the >>>>>>>>>> aforementioned error. >>>>>>> >>>>>>>>>> -Erinn >>>>>>> >>>>>>> >>>>>>>>> To reply to myself here, I believe the problem may >>>>>>>>> be that I had to renew the CA certificates and as >>>>>>>>> such the certificates in /root/cacert.p12 are no >>>>>>>>> longer valid. It is this file that gets bundled up >>>>>>>>> with whatever else using ipa-replica-prepare, so I >>>>>>>>> will have to create a new one that has the valid >>>>>>>>> certificates in it. >>>>>>> >>>>>>>>> One way or another though, if it isn't already >>>>>>>>> documented, during a CA renewal this file should >>>>>>>>> probably be updated with the correct certificates. >>>>>>> >>>>>>>>> -Erinn >>>>>>> >>>>>>>>> -Erinn >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Well thanks to this: >>>>>>>> http://www.freeipa.org/page/Howto/Change_Directory_Manager_Password >>>>>>> >>>>>>>> >>>>>>>> >>>> >>>>>>>> >> >>>>>>>> I have gotten a little further down the road an created a new >>>>>>>> cacert.p12 which looks to be complete. >>>>>>> >>>>>>>> However, installation still fails in the same place: >>>>>>> >>>>>>>> 2014-07-27T06:33:04Z DEBUG Starting external process >>>>>>>> 2014-07-27T06:33:04Z DEBUG args=/usr/sbin/pkispawn >>>>>>>> -s CA -f /tmp/tmp5QGhUx 2014-07-27T06:33:25Z DEBUG >>>>>>>> Process finished, return code=1 2014-07-27T06:33:25Z >>>>>>>> DEBUG stdout=Loading deployment configuration from >>>>>>>> /tmp/tmp5QGhUx. Installing CA into >>>>>>>> /var/lib/pki/pki-tomcat. Storing deployment >>>>>>>> configuration into >>>>>>>> /etc/sysconfig/pki/tomcat/pki-tomcat/ca/deployment.cfg. >>>>>>>> Installation failed. >>>>>>> >>>>>>> >>>>>>>> 2014-07-27T06:33:25Z DEBUG stderr=pkispawn : >>>>>>>> WARNING ....... unable to validate security domain >>>>>>>> user/password through REST interface. Interface not >>>>>>>> available pkispawn : ERROR ....... Exception from >>>>>>>> Java Configuration Servlet: Clone does not have all >>>>>>>> the required certificates >>>>>>> >>>>>>>> 2014-07-27T06:33:25Z CRITICAL failed to configure ca >>>>>>>> instance Command '/usr/sbin/pkispawn -s CA -f >>>>>>>> /tmp/tmp5QGhUx' returned non-zero exit status 1 >>>>>>>> 2014-07-27T06:33:25Z DEBUG File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>> >>>>>>>> >> >>>>>>>> line 638, in run_script >>>>>>>> return_value = main_function() >>>>>>> >>>>>>>> File "/usr/sbin/ipa-replica-install", line 667, in >>>>>>>> main CA = cainstance.install_replica_ca(config) >>>>>>> >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>> >>>>>>>> >> >>>>>>>> line 1678, in install_replica_ca >>>>>>>> subject_base=config.subject_base) >>>>>>> >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>> >>>>>>>> >> >>>>>>>> line 478, in configure_instance >>>>>>>> self.start_creation(runtime=210) >>>>>>> >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", >>>>>>>> >>>>>>>> >>>> >>>>>>>> >> >>>>>>>> line 364, in start_creation method() >>>>>>> >>>>>>>> File >>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>> >>>>>>>> >> >>>>>>>> line 604, in __spawn_instance >>>>>>>> raise RuntimeError('Configuration of CA failed') >>>>>>> >>>>>>>> 2014-07-27T06:33:25Z DEBUG The ipa-replica-install >>>>>>>> command failed, exception: RuntimeError: >>>>>>>> Configuration of CA failed >>>>>>> >>>>>>> >>>>>>>> So some of the required certificates must be missing >>>>>>>> still. >>>>>>> >>>>>>>> Unhelpfully, the ipa-server-install --uninstall >>>>>>>> process is not cleaning up everything after this >>>>>>>> failure, it leaves the CA intact and the next run >>>>>>>> through the installer believes the CA is working so >>>>>>>> it does not configure it. As such, I guess a >>>>>>>> re-install is necessary or some other steps to truly >>>>>>>> clean everything that I haven't found yet. >>>>>>> >>>>>>>> -Erinn >>>>>>> >>>>>>> Continuing on, in order to remove the CA I am manually >>>>>>> running: pkidestroy -s CA -i pki-tomcat >>>>>>> >>>>>>> And indeed there is a bug: >>>>>>> https://fedorahosted.org/freeipa/ticket/2796 >>>>>>> >>>>>>> Interesting that the installer detects that the CA is >>>>>>> installed, but the uninstaller does not detect it. I >>>>>>> guess they are doing their detection in different >>>>>>> ways. >>>>>> >>>>>> The uninstaller doesn't rely on detection. There is a >>>>>> stored log of what needs to be done. Unfortunately in >>>>>> this case the fact that the CA was configured was added >>>>>> AFTER it was successfully installed and not when we >>>>>> started, so if installation fails it can leave things >>>>>> half-installed but not recorded. >>>>>> >>>>>>> At this point I wanted to explore how feasible it would >>>>>>> be to have a RHEL 7 replica without the CA replica >>>>>>> portion, this ought to alleviate the KDC issues I seem >>>>>>> to be having on the primary, which I have still to >>>>>>> figure out. >>>>>>> >>>>>>> So any reason not to do that? Would I simply be able to >>>>>>> do a ipa-ca-install on the rhel 7 system at a future >>>>>>> juncture and then perform the rest of the migration? >>>>>> >>>>>> This would be a reasonable short-term stop-gap measure >>>>>> though if you can live without a second CA. You would >>>>>> likely have the same problem with ipa-ca-install, at >>>>>> least until we figure out what this missing cert error >>>>>> means. >>>>>> >>>>>> I've seen that error about missing certs before but I >>>>>> can't recall what it means. I have the vague notion it is >>>>>> a little misleading though, and that something else has >>>>>> already failed. I think we'll need one of the dogtag devs >>>>>> to chime in. I'll poke them out-of-band. >>>>> >>>>> Ok, start with the debug log on the clone ( >>>>> /var/log/pki/pki-tomcat/ca/debug ). It should tell you >>>>> which cert is missing or unreadable. >>>>> >>>>> How did you re-create the PKCS#12 file on the RHEL-6 >>>>> server? You used PKCS12Export, right? >>>>> >>>>> rob >>>>> >>>> >>>> Correct, I just did the steps as if I was changing the dir >>>> manager password, to re-export the certificates. >>>> >>>> To my untrained eye it looks like the server-cert that is >>>> failing, but here are what I believe the pertinent bits from >>>> the debug log: >>>> >>>> [27/Jul/2014:20:46:24][http-bio-8443-exec-3]: >>>> updateNumberRange: Failed to contact master using admin >>>> portorg.xml.sax.SAXParseException; lineNumber: 1; >>>> columnNumber: 50; White spaces are required between publicId >>>> and systemId. [27/Jul/2014:20:46:24][http-bio-8443-exec-3]: >>>> updateNumberRange: Attempting to contact master using EE port >>>> [27/Jul/2014:20:46:25][http-bio-8443-exec-3]: content from >>>> ee interface =<?xml version="1.0" encoding="UTF-8" >>>> standalone="no"?><XMLResponse><Status>0</Status><beginNumber>66</beginNumber><endNumber>70</endNumber></XMLResponse> >>>> >>>> >> >>>> [27/Jul/2014:20:46:25][http-bio-8443-exec-3]: updateNumberRange(): >>>> status=0 [27/Jul/2014:20:46:25][http-bio-8443-exec-3]: >>>> updateConfigEntries start >>>> [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: >>>> updateConfigEntries: status=0 >>>> [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: >>>> deleteExistingCerts: >>>> Exception=org.mozilla.jss.crypto.ObjectNotFoundException >>>> [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: >>>> deleteExistingCerts: >>>> Exception=org.mozilla.jss.crypto.NoSuchItemOnTokenException >>>> [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: >>>> deleteExistingCerts: >>>> Exception=org.mozilla.jss.crypto.ObjectNotFoundException >>>> [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: >>>> deleteExistingCerts: >>>> Exception=org.mozilla.jss.crypto.NoSuchItemOnTokenException >>>> [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: >>>> deleteExistingCerts: >>>> Exception=org.mozilla.jss.crypto.ObjectNotFoundException >>>> [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: >>>> deleteExistingCerts: >>>> Exception=org.mozilla.jss.crypto.NoSuchItemOnTokenException >>>> [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: >>>> deleteExistingCerts: >>>> Exception=org.mozilla.jss.crypto.ObjectNotFoundException >>>> [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: >>>> deleteExistingCerts: >>>> Exception=org.mozilla.jss.crypto.NoSuchItemOnTokenException >>>> [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: Key Algorithm >>>> 'RSA' [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: Key >>>> Algorithm 'RSA' [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: >>>> Key Algorithm 'RSA' >>>> [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: Key Algorithm >>>> 'RSA' [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: Key >>>> Algorithm 'RSA' [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: >>>> Key Algorithm 'RSA' >>>> [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: Key Algorithm >>>> 'RSA' [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: Key >>>> Algorithm 'RSA' [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: >>>> Ignoring key CN=ipa.example.com,O=EXAMPLE.COM >>>> [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: Key Algorithm >>>> 'RSA' [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: Not >>>> importing Server-Cert cert-pki-ca >>> >>> The above line is absolutely correct. The server cert is not >>> imported into the clone. The clone will generate its own >>> server cert. >>> >>>> [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: isCertdbCloned: >>>> caSigningCert cert-pki-ca >>>> [27/Jul/2014:20:46:26][http-bio-8443-exec-3]: clone does not >>>> have all the certificates. >>>> >>> This actually indicates to me that the signing cert is not >>> present in the P12 replica file, or that some error has been >>> encountered in reading the cert. >>> >>> Also, check in the journal to see if any exceptions have been >>> thrown. journalctl -u pki-tomcatd@pki-tomcat.service >>> >>> Ade >>> >> >> No exceptions thrown in the journal. >> >> When investigating the cacert.p12 file that is bundled up for >> the replica's I see two caSigningCert's. One is the older one, >> before I renewed and one is the new, valid, post renewal one. Are >> these the certs that are used or are they requested from the >> primary much like the servercert? > > I think the problem might be that there are two caSigningCerts, > with presumably the same nickname. We need to try and generate a > p12 file without the old pre-renewal signing cert. > > Does the master certdb contain both certs with the same nickname? > If so, you could try to remove the old signing cert from the > master certdb and then regenerate the pkcs12 using PKCS12Export. > > Here is one way you could try to do this: 1. Export the caSigning > cert from the certdb using pk12util. Check that only the latest > cert/key has been exported. Make sure to note down the exact cert > nickname. If so, then continue .. 2. Shut down the CA. 3. > IMPORTANT: Back up the certdb. 4. Delete the caSigning cert from > the certdb using certutil. You may have to do this twice to remove > both certs. 5. Re-import the saved caSigningCert using pk12util. 6. > Restart the CA. > >> >> However, when examining the /etc/pki/pki-tomcat/alias db there is >> no casigningcert, hence I suppose the failure. So somewhere in >> there it is getting lost, I'll keep looking but throw me any >> ideas. >> > By this, you are implying looking at the clone certdb, right? The > cert should certainly still exist on the master. The cert not > being in the clone certdb is because it fails to be imported from > the PKCS12 file. > > >> -Erinn >> >> >> > >
Using the following: pk12util -o /root/casigningcert.p12 -d /var/lib/pki-ca/alias/ -n 'caSigningCert cert-pki-ca' I end up with both the old and the new certs, as well as two copies of the root CA. Any other ways to change around the DB? I don't know NSS tools all that well. - -Erinn -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBCAAGBQJT1pcjAAoJEFg7BmJL2iPOrE0IAIzfjjmmK+lmMfA7q869sAEq hF30gceUy+XRKFZZxZsgg8HVnmr3UmBT1f3JY19ADkW4HQ5PkWr7/zDGIkoSTfvT WTn9wnYtqj/zg5ZQH6wOtlSoyu1hBJXKgwJQn0RNiGdeHaVYh6mtLT+/Z4Dsgdh1 +KP9fQIlGTwLmYtrEHSbRfjQ3bH3slIGzcuNjiDmH37kpkEwVkhfKh0+QOvdko4p HSX1ulNRbKEDgc08BXHUpwtuJ+R/bs96DM0jF9/PjgdDgI2J+Q3j6BJoLMqNCqNB P6OwJlY6Guq3PotVBeSSrZG4CzHYHi/xweW9mN/vAyIBSJk6QwkswtKF8EKLW+I= =eltq -----END PGP SIGNATURE----- -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go To http://freeipa.org for more info on the project