On 28.08.2012 16:40, Petr Viktorin wrote: > On 08/17/2012 06:04 PM, Ade Lee wrote: >> On Fri, 2012-08-17 at 09:34 -0400, Ade Lee wrote: >>> On Thu, 2012-08-16 at 18:45 +0200, Martin Kosek wrote: >>>> On 08/16/2012 01:28 PM, Ade Lee wrote: >>>>> Patch attached this time. I should know better than to do this in the >>>>> middle of the night .. >>>>> >>>>> On Thu, 2012-08-16 at 09:12 +0200, Martin Kosek wrote: >>>>>> On 08/16/2012 07:53 AM, Ade Lee wrote: >>>>>>> On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: >>>>>>>> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: >>>>>>>>> On 08/15/2012 03:54 PM, Ade Lee wrote: >>>>>>>>>> On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: >>>>>>>>>>> On 08/08/2012 10:05 PM, Ade Lee wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Dogtag 10 is being released on f18, and has a number of >>>>>>>>>>>> changes that >>>>>>>>>>>> will affect IPA. In particular, the following changes will >>>>>>>>>>>> affect >>>>>>>>>>>> current IPA code. >>>>>>>>>>>> >>>>>>>>>>>> * The directory layout of the dogtag instance has changed. >>>>>>>>>>>> Instead of >>>>>>>>>>>> using separate tomcat instances to host different >>>>>>>>>>>> subsystems, the >>>>>>>>>>>> standard dogtag installation will allow one to install a CA. >>>>>>>>>>>> KRA, OCSP >>>>>>>>>>>> and TKS within the same instance. There have been >>>>>>>>>>>> corresponding changes >>>>>>>>>>>> in the directory layout, as well as the default instance name >>>>>>>>>>>> (pki-tomcat instead of pki-ca), and startup daemon >>>>>>>>>>>> (pki-tomcatd, instead >>>>>>>>>>>> of pki-cad, pki-krad etc.) >>>>>>>>>>>> >>>>>>>>>>>> * The default instance will use only four ports (HTTPS, >>>>>>>>>>>> HTTP, AJP and >>>>>>>>>>>> tomcat shutdown port) rather than the 6 previously used. >>>>>>>>>>>> The default >>>>>>>>>>>> ports will be changed to the standard tomcat ports. As >>>>>>>>>>>> these ports are >>>>>>>>>>>> local to the ipa server machine, this should not cause too much >>>>>>>>>>>> disruption. >>>>>>>>>>>> >>>>>>>>>>>> * There is a new single step installer written in python. >>>>>>>>>>>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. >>>>>>>>>>>> >>>>>>>>>>>> * Dogtag 10 runs on tomcat7 - with a new corresponding >>>>>>>>>>>> version of >>>>>>>>>>>> tomcatjss. >>>>>>>>>>>> >>>>>>>>>>>> The attached patch integrates all the above changes in IPA >>>>>>>>>>>> installation >>>>>>>>>>>> and maintenance code. Once the patch is applied, users will >>>>>>>>>>>> be able to: >>>>>>>>>>>> >>>>>>>>>>>> 1. run ipa-server-install to completion on f18 with dogtag 10. >>>>>>>>>>>> 2. install a new replica on f18 on dogtag 10. >>>>>>>>>>>> 3. upgrade an f17 machine with an existing IPA instance to >>>>>>>>>>>> f18/ dogtag >>>>>>>>>>>> 10 - and have that old-style dogtag instance continue to run >>>>>>>>>>>> correctly. >>>>>>>>>>>> This will require the installation of the latest version of >>>>>>>>>>>> tomcatjss as >>>>>>>>>>>> well as the installation of tomcat6. The old-style instance >>>>>>>>>>>> will >>>>>>>>>>>> continue to use tomcat6. >>>>>>>>>>>> 4. in addition, the new cert renewal code has been patched >>>>>>>>>>>> and should >>>>>>>>>>>> continue to work. >>>>>>>>>>>> >>>>>>>>>>>> What is not yet completed / supported: >>>>>>>>>>>> >>>>>>>>>>>> 1. Installation with an external CA is not yet completed in >>>>>>>>>>>> the new >>>>>>>>>>>> installer. We plan to complete this soon. >>>>>>>>>>>> >>>>>>>>>>>> 2. There is some IPA upgrade code that has not yet been touched >>>>>>>>>>>> (install/tools/ipa-upgradeconfig). >>>>>>>>>>>> >>>>>>>>>>>> 3. A script needs to be written to allow admins to convert >>>>>>>>>>>> their >>>>>>>>>>>> old-style dogtag instances to new style instances, as well >>>>>>>>>>>> as code to >>>>>>>>>>>> periodically prompt admins to do this. >>>>>>>>>>>> >>>>>>>>>>>> 4. Installation of old-style instances using >>>>>>>>>>>> pkicreate/pkisilent on >>>>>>>>>>>> dogtag 10 will no longer be supported, and will be disabled >>>>>>>>>>>> soon. >>>>>>>>>>>> >>>>>>>>>>>> 5. The pki-selinux policy has been updated to reflect these >>>>>>>>>>>> changes, >>>>>>>>>>>> but is still in flux. In fact, it is our intention to place >>>>>>>>>>>> the dogtag >>>>>>>>>>>> selinux policy in the base selinux policy for f18. In the >>>>>>>>>>>> meantime, it >>>>>>>>>>>> may be necessary to run installs in permissive mode. >>>>>>>>>>>> >>>>>>>>>>>> The dogtag 10 code will be released shortly into f18. Prior >>>>>>>>>>>> to that >>>>>>>>>>>> though, we have placed the new dogtag 10 and tomcatjss code >>>>>>>>>>>> in a >>>>>>>>>>>> developer repo that is located at >>>>>>>>>>>> http://nkinder.fedorapeople.org/dogtag-devel/ >>>>>>>>>>>> >>>>>>>>>>>> Testing can be done on both f18 and f17 - although the >>>>>>>>>>>> target platform - >>>>>>>>>>>> and the only platform for which official builds will be >>>>>>>>>>>> created is f18. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Ade >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Ade, >>>>>>>>>>> >>>>>>>>>>> Thanks for the patch, I started with review and integration >>>>>>>>>>> tests (currently >>>>>>>>>>> running on Fedora 17 with Nathan's repo). >>>>>>>>>>> >>>>>>>>>>> Installation on single master was smooth, it worked just >>>>>>>>>>> fine, even with >>>>>>>>>>> enforced SELinux, without any error - kudos to you and the >>>>>>>>>>> whole dogtag team. >>>>>>>>>>> The resulting logs and the structure of your log directory >>>>>>>>>>> seems improved. I >>>>>>>>>>> believe that the brand new Python installers will make it >>>>>>>>>>> easier to debug >>>>>>>>>>> issues with dogtag installation when they come. When I tried >>>>>>>>>>> our unit tests or >>>>>>>>>>> some simple cert operation, it worked fine as well. >>>>>>>>>>> >>>>>>>>>>> Now the bad news, or rather few issues and suggestions I found: >>>>>>>>>>> >>>>>>>>>>> 1) As we already discussed on IRC, tomcat 7 was not pulled >>>>>>>>>>> automatically on >>>>>>>>>>> Fedora 17 when I updated pki-ca, you somewhere miss a Requires. >>>>>>>>>>> >>>>>>>>>> We have a dogtag patch that is currently in review that will >>>>>>>>>> address >>>>>>>>>> this. Once this is in, tomcatjss >=7.0.0 will be required for >>>>>>>>>> f17+, >>>>>>>>>> rather than f18+ >>>>>>>>>> >>>>>>>>>>> 2) I had installed IPA with dogtag10 on master. However, CA >>>>>>>>>>> installation on a >>>>>>>>>>> replica (ipa-ca-install) with dogtag9 failed - is this >>>>>>>>>>> expectable? >>>>>>>>>>> >>>>>>>>>> Yes. The current IPA patch is designed to work with dogtag 10 >>>>>>>>>> only, >>>>>>>>>> which will be officially available on f18+. So if you update >>>>>>>>>> to dogtag >>>>>>>>>> 10, you must have this patch and visa versa. We probably need >>>>>>>>>> to add >>>>>>>>>> the relevant requires to IPA to enforce this. >>>>>>>>>> >>>>>>>>>> If you have an existing dogtag 9 instance, and you upgrade to >>>>>>>>>> the new >>>>>>>>>> dogtag 10 and patched IPA, then that instance will continue to >>>>>>>>>> work. >>>>>>>>>> But any new instances would be created using dogtag 10. >>>>>>>>>> >>>>>>>>>>> 3) I had installed IPA with dogtag10 on master. Replica had >>>>>>>>>>> dogtag10 as well >>>>>>>>>>> and I got the following error: >>>>>>>>>>> >>>>>>>>>>> # ipa-ca-install >>>>>>>>>>> /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>>>>>>>>> ... >>>>>>>>>>> Configuring certificate server: Estimated time 3 minutes 30 >>>>>>>>>>> seconds >>>>>>>>>>> [1/14]: creating certificate server user >>>>>>>>>>> [2/14]: configuring certificate server instance >>>>>>>>>>> >>>>>>>>>>> Your system may be partly configured. >>>>>>>>>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>>>>>>>>> >>>>>>>>>>> Unexpected error - see /var/log/ipareplica-ca-install.log for >>>>>>>>>>> details: >>>>>>>>>>> IOError: [Errno 2] No such file or directory: >>>>>>>>>>> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12' >>>>>>>>>>> >>>>>>>>>>> Root cause: >>>>>>>>>>> ... >>>>>>>>>>> File >>>>>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", >>>>>>>>>>> line >>>>>>>>>>> 625, in __spawn_instance >>>>>>>>>>> "/root/cacert.p12") >>>>>>>>>>> ... >>>>>>>>>>> >>>>>>>>>> I need to look into this. I had fixed ipa-replica-install, >>>>>>>>>> rather than >>>>>>>>>> ipa-ca-install to create replicas. I didn't know ipa-ca-install >>>>>>>>>> existed! Should not be too bad to fix though - most likely >>>>>>>>>> just need to >>>>>>>>>> move files to the right place. >>>>>>>>> >>>>>>>>> Ok, thanks! Btw. CA on replica can be either installed during >>>>>>>>> ipa-replica-install time (when --setup-ca option is passed, you >>>>>>>>> probably used >>>>>>>>> that one) and the aforementioned ipa-ca-install run after >>>>>>>>> ipa-replica-install. >>>>>>>>> >>>>>>>> I will be testing this out again. As ipa-ca-install uses the >>>>>>>> same code >>>>>>>> as ipa-replica-install, I would have expected it to work. Did >>>>>>>> you try >>>>>>>> it with selinux in permissive mode? >>>>>>>> >>>>>>> OK -- I tested this out and realized that between the time I >>>>>>> initially >>>>>>> tested this and your tests, we had changed a URL >>>>>>> from /ca/pki/installer/installToken to >>>>>>> /ca/rest/installer/installToken, >>>>>>> but I forgot to update ipa-pki-proxy.conf. >>>>>>> >>>>>>> The new patch has been rebased and changes the relevant patch. With >>>>>>> this, the CA replica installs correctly. >>>>>> >>>>>> Umh, where can I find the new rebased patch? :-) >>>>>> >>>>>>> >>>>>>> There was one issue that I encountered. I have seen this once >>>>>>> before >>>>>>> and will need to investigate further. IPA sometimes restarts the >>>>>>> dogtag >>>>>>> instance by doing systemctl stop pki-tomcatd.target. and then >>>>>>> systemctl >>>>>>> start pki-tomcatd.target. I have noticed that occasionally the >>>>>>> start >>>>>>> invocation fails, while the stop and restart invocations succeed >>>>>>> just >>>>>>> fine. This makes absolutely no sense because restart is essentially >>>>>>> just stop and then start. >>>>>>> >>>>>>> I think this is actually a bug in systemd. If I reboot the >>>>>>> machine, it >>>>>>> all works perfectly. If we can't figure out whats going on with >>>>>>> systemd, we can always replace this start/stop with >>>>>>> starting/stopping >>>>>>> the service directly. (pki-tomcatd@pki-tomcat.service). That >>>>>>> seems to >>>>>>> work all the time with no problems. >>>>>>> >>>>>>> I suggest we leave this fix (if needed) for a separate patch. >>>>>> >>>>>> Ok, I will see if I hit this issue again. Btw. is it recommended >>>>>> in systemd to >>>>>> use target units this way? I thought that only service files are >>>>>> meant to be >>>>>> used for manipulation with a service. As you said yourself, using >>>>>> service unit >>>>>> file for restart worked every time. >>>>>> >>>>>> Martin >>>>> >>>> >>>> Now, I tested the rebased patch with VMs with permissive SELinux. >>>> IPA server >>>> install was OK, as always, but replica installation ended up with >>>> error. >>>> Although it got much further than before: >>>> >>>> # ipa-ca-install >>>> /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg >>>> ... >>>> [3/3]: restarting directory server >>>> done configuring pkids. >>>> Configuring certificate server: Estimated time 3 minutes 30 seconds >>>> [1/14]: creating certificate server user >>>> [2/14]: configuring certificate server instance >>>> [3/14]: disabling nonces >>>> [4/14]: importing CA chain to RA certificate database >>>> [5/14]: fixing RA database permissions >>>> [6/14]: setting up signing cert profile >>>> [7/14]: set up CRL publishing >>>> [8/14]: set certificate subject base >>>> [9/14]: enabling Subject Key Identifier >>>> [10/14]: configuring certificate server to start on boot >>>> [11/14]: configure certmonger for renewals >>>> [12/14]: configure clone certificate renewals >>>> [13/14]: configure Server-Cert certificate renewal >>>> [14/14]: Configure HTTP to proxy connections >>>> done configuring pki-tomcatd. >>>> Restarting the directory and certificate servers >>>> >>>> Your system may be partly configured. >>>> Run /usr/sbin/ipa-server-install --uninstall to clean up. >>>> >>>> It seems like that CA on a replica did not start and so a check for >>>> port 8080 >>>> failed. Attached logs (pki-ca-logs.tgz) may provide more information >>>> to this issue. >>>> >>> This is the systemd issue I mentioned before. I will submit a patch >>> which will change the restart mechanism to restart the service and not >>> the target. >>>> >> >> Patch attached. This patch should be applied on top of the large patch >> (being reviewed). >> >>>> Now I am also testing upgrade from IPA+dogtag9 to PatchedIPA+dogtag10 >>>> (permissive SELinux). Now, the service was upgraded smoothly, CA >>>> service >>>> could be restarted without failure. However, certificate operations now >>>> did not work: >>>> >>>> # ipactl restart >>>> Restarting Directory Service >>>> Restarting KDC Service >>>> Restarting KPASSWD Service >>>> Restarting MEMCACHE Service >>>> Restarting HTTP Service >>>> Restarting CA Service >>>> # ipactl status >>>> Directory Service: RUNNING >>>> KDC Service: RUNNING >>>> KPASSWD Service: RUNNING >>>> MEMCACHE Service: RUNNING >>>> HTTP Service: RUNNING >>>> CA Service: RUNNING >>>> # ipa cert-show 1 >>>> ipa: ERROR: Certificate operation cannot be completed: Unable to >>>> communicate with CMS (Internal Server Error) >>>> >>>> Attaching logs for this case as well (ipa-ca-logs-upgrade.tgz). >>>> >>> The reason for this is that the old dogtag instances are missing: >>> 1. a new parameter in CS.cfg >>> 2. a couple of symlinks >>> >>> I plan to fix this in the dogtag code. Specifically, >>> 1. I will modify the dogtag code to provide a default value in the case >>> where the parameter does not exist. >>> 2. I plan to add a function to the dogtag startup scripts to check and >>> remake any required symlinks. >>> >>> Both of these changes are in the dogtag code, and do not affect this >>> patch. As a workaround, to confirm that the upgrade actually works, do >>> the following manual steps on your upgraded instance: >>> >>> 1. Add the following parameter to CS.cfg >>> pidDir=/var/run >>> >>> 2. Add the following links; >>> cd /var/lib/pki-ca/common/lib >>> ln -s /usr/share/java/apache-commons-codec.jar >>> apache-commons-codec.jar >>> ln -s /usr/share/java/log4j.jar log4j.jar >>> >>> 3 Restart the dogtag instance >>> >>>> HTH, >>>> Martin >>> >>> > > The attached patch conditionalizes the changes, so that IPA supports > both Dogtag versions. > > I didn't put it in platform code for two reasons > - One platform (f17) can have either Dogtag version installed > - I didn't want to conflict with Timo Aaltonen's "platformizing" work. > According to his WIP patches, he plans to move the whole dogtag module > into platform code.
Nothing is set in stone yet, I was merely experimenting there :) That said, I'd like to see the dogtag patches in master soon in order to rebase what I have so it could be sent out, and ask for directions where to go from there.. -- t _______________________________________________ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
