Winfried de Heiden wrote: > Hi all, > > Fixed it, thanks for the tip Rob :-)! > Certmonger was to blame or my rather slow Udooboard Celeron processor. > Anyway, instead of hacking the upgrade script, I modified the > certmonger.serivce file by adding a 180 secs (!!) sleep and extra > Timeout: (The modified certmonger.service was removed after the upgrade) > > [Unit] > Description=Certificate monitoring and PKI enrollment > After=syslog.target network.target dbus.service > > [Service] > Type=dbus > PIDFile=/var/run/certmonger.pid > EnvironmentFile=-/etc/sysconfig/certmonger > ExecStart=/usr/sbin/certmonger -S -p /var/run/certmonger.pid -n $OPTS > ExecStartPost=/bin/sleep 180 > TimeoutSec=240 > BusName=org.fedorahosted.certmonger > > Runing "ipa-server-upgrade" finished OK now. Certmonger takes itś time > when it's (restarted, some dogtag-ipa-ca-r(enew ?) processes eating most > of the cpu: > > top - 16:00:24 up 18:51, 3 users, load average: 2.41, 1.87, 1.37 > Tasks: 261 total, 6 running, 221 sleeping, 0 stopped, 34 zombie > %Cpu0 : 90.2 us, 7.8 sy, 0.0 ni, 0.0 id, 0.0 wa, 1.6 hi, 0.3 > si, 0.0 st > %Cpu1 : 92.4 us, 6.6 sy, 0.0 ni, 0.0 id, 0.0 wa, 1.0 hi, 0.0 > si, 0.0 st > %Cpu2 : 95.1 us, 3.6 sy, 0.0 ni, 0.0 id, 0.0 wa, 1.3 hi, 0.0 > si, 0.0 st > %Cpu3 : 88.6 us, 9.2 sy, 0.0 ni, 0.0 id, 0.0 wa, 1.3 hi, 1.0 > si, 0.0 st > MiB Mem : 3847.2 total, 335.4 free, 2154.9 used, 1356.9 buff/cache > MiB Swap: 3968.0 total, 3968.0 free, 0.0 used. 1452.0 avail Mem > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ > COMMAND > 21750 root 20 0 401244 85296 22612 R 85.9 2.2 0:13.36 > dogtag-ipa-ca-r > 21764 root 20 0 386700 72880 22508 R 78.4 1.8 0:06.93 > dogtag-ipa-ca-r > 21771 root 20 0 161788 27332 10812 R 74.5 0.7 0:03.65 > dogtag-ipa-ca-r > 21758 root 20 0 394512 78340 22436 R 67.3 2.0 0:10.65 > dogtag-ipa-ca-r > 21746 root 20 0 0 0 0 Z 51.6 0.0 0:15.36 > dogtag-ipa-ca-r > 21778 root 20 0 106004 1220 0 R 24.8 0.0 0:00.76 > certmonger > > This seems like a new issue for me... Certainly, the Udoo x86 isn't the > fasted in the world, but was running IPA bravely the last year... Am I > hitting the bug Rob mentioned? Is there a bug report somewhere to > track... I'll like to see it fixed in CentOS 8.
It should be in 8.2 beta, https://bugzilla.redhat.com/show_bug.cgi?id=1763745 > "getcert list" showed "/var/lib/ipa/private/httpd.key" and > "/var/lib/ipa/certs/httpd.crt" wating for PIN. Running "ipa-getcert > resubmit -i 20200126151811 -p /var/lib/ipa/passwds/ipa.xxx-443-RSA" > fixed it. I can't explain that. rob > > Winfried > > -----Oorspronkelijk bericht----- > *Van*: Rob Crittenden <rcrit...@redhat.com > <mailto:rob%20crittenden%20%3crcrit...@redhat.com%3e>> > *Aan*: FreeIPA users list <freeipa-users@lists.fedorahosted.org > <mailto:freeipa%20users%20list%20%3cfreeipa-us...@lists.fedorahosted.org%3e>> > *Cc*: Winfried de Heiden <w...@dds.nl > <mailto:winfried%20de%20heiden%20%3c...@dds.nl%3e>> > *Onderwerp*: Re: [Freeipa-users] Re: ipa-server-upgrade failed > *Datum*: Sat, 25 Jan 2020 17:04:39 -0500 > > Winfried de Heiden via FreeIPA-users wrote: >> Hi all, >> >> /var/lib/ipa/private/httpd.key was in a status "waiting for PIN", but I >> did brong is back to life using "ipa-getcert resubmit -i 20200117075404 >> -p /var/lib/ipa/passwds/xxxx-443-RSA. All certss look fine now. >> "getcert list" works, although it's a bit slow the first time (running >> on a Udoo x86 board with a celeron....) >> >> Just to be shure about dbus, I restarted the entire machine; no success. :-( >> >> Timing issue and/or casued by my rather slow Udoo board.....? > > It is very possible. I fixed an issue in certmonger where every time it > > forked (and it forks a LOT) it closed ALL the fds it knew about. On > > containers this was 1M. It took a LONG time. The default is a more > > modest 1k but can still take a while given the amount of forks that > > certmonger does. This is fixed upstream, and I don't know of a > > workaround, but this can definitely lead to timeout issues if certmonger > > is being restarted immediately before this failure. > > > To diagnose it see what the load on the system is and what processes are > > running. If you see dozens of certmonger processes with high load then > > that's probably it. You'd have to hack the update script to do a sleep > > to give things a chance to settle down. > > > rob > > >> >> Winfried >> >> >> >> >> >> Rob Crittenden schreef op za 25-01-2020 om 14:53 [-0500]: >>> Winfried de Heiden via FreeIPA-users wrote: >>>> Hi all, >>>> >>>> Using CentOS Linux release 8.1.1911 and the Stream repositories, >>>> upgrading IPA fails: >>>> >>>> ( Upgrade ipa-server-common-4.8.0-13.module_el8.1.0+265+e1e65be4.noarch >>>> @AppStream >>>> Upgraded >>>> ipa-server-common-4.8.0-11.module_el8.1.0+253+3b90c921.noarch @@System ) >>>> >>>> Running ipa-server-upgrade manually will result in: >>>> >>>> [Upgrading CA schema] >>>> CA schema update complete (no changes) >>>> [Verifying that CA audit signing cert has 2 year validity] >>>> [Update certmonger certificate renewal configuration] >>>> Introspect error on :1.417:/org/fedorahosted/certmonger: >>>> dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NoReply: Did >>>> not receive a reply. Possible causes include: the remote application did >>>> not send a reply, the message bus security policy blocked the reply, the >>>> reply timeout expired, or the network connection was broken. >>> >>> I assume certmonger and dbus services are running? >>> >>> Does `getcert list` work? >>> >>> The dbus service sometimes isn't too fond of being restarted but you >>> could try that. >>> >>> rob >>> >> >> >> _______________________________________________ >> FreeIPA-users mailing list -- >> freeipa-users@lists.fedorahosted.org >> <mailto:freeipa-users@lists.fedorahosted.org> >> >> To unsubscribe send an email to >> freeipa-users-le...@lists.fedorahosted.org >> <mailto:freeipa-users-le...@lists.fedorahosted.org> >> >> Fedora Code of Conduct: >> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >> >> List Guidelines: >> https://fedoraproject.org/wiki/Mailing_list_guidelines >> >> List Archives: >> https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org >> >> > _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
