Hi. Into my environment I have two IPA server, replicating each other. They are both 7.6 OS systems, ipa-server RPM version is 4.6.4-10.0.1.el7_6.2.x86_64.
The first server installed was srv01 (many years ago), then I installed the replica into srv02 (like a year later the 1st node). When I had a single server I did also a trust with my corporate Active Directory. VMs are running in 2 different hypervisor clusters. Now the replication doesn't works. Into log files I have this error: *[16/Apr/2020:12:25:36.856632697 +0200] - ERR - csngen_adjust_time - Adjustment limit exceeded; value - 23221226, limit - 86400[16/Apr/2020:12:25:36.857909222 +0200] - ERR - NSMMReplicationPlugin - repl5_inc_run - agmt="cn=meTosrv01.ipa.mydomain.com <http://meTosrv01.ipa.mydomain.com>" (srv01:389): Fatal error - too much time skew between replicas![16/Apr/2020:12:25:36.862233147 +0200] - ERR - NSMMReplicationPlugin - repl5_inc_run - agmt="cn=meTosrv01.ipa.mydomain.com <http://meTosrv01.ipa.mydomain.com>" (srv01:389): Incremental update failed and requires administrator action* I tried to force the replica, but the limit exceeded problem doesn't allow the sync. I know that the problem is that CSN generator has become grossly skewed. Using the external script readNsState.py I found that there was as offset time for about a month, so ... I waited for a month and then the issue disappeared. But now the offset is about 9 months ... I can't wait so much time :) *[root@srv01 scripts]# ./readNsState.py /etc/dirsrv/slapd-IPA-MYDOMAIN-COM/dse.ldifnsState is BAAAAAAAAACCN/xfAAAAAHbiBAAAAAAABCgAAAAAAAANdQAAAAAAAA==Little EndianFor replica cn=replica,cn=dc\3Dipa\2Cdc\3Dmydomain\2Cdc\3Dcom,cn=mapping tree,cn=con fmtstr=[H6x3QH6x] size=40 len of nsstate is 40 CSN generator state: Replica ID : 4 Sampled Time : 1610364802 Gen as csn : 5ffc37822996500040000 Time as str : Mon Jan 11 12:33:22 2021 Local Offset : 320118 Remote Offset : 10244 Seq. num : 29965 System time : Tue Apr 21 15:03:45 2020 Diff in sec. : -22890577 Day:sec diff : -265:5423nsState is YAAAAAAAAAADLZheAAAAAAAAAAAAAAAAXSgAAAAAAAATAAAAAAAAAA==Little EndianFor replica cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config fmtstr=[H6x3QH6x] size=40 len of nsstate is 40 CSN generator state: Replica ID : 96 Sampled Time : 1587031299 Gen as csn : 5e982d03001900960000 Time as str : Thu Apr 16 12:01:39 2020 Local Offset : 0 Remote Offset : 10333 Seq. num : 19 System time : Tue Apr 21 15:03:45 2020 Diff in sec. : 442926 Day:sec diff : 5:10926[root@srv02 scripts]# ./readNsState.py /etc/dirsrv/slapd-IPA-MYDOMAIN-COM/dse.ldifnsState is AwAAAAAAAABU7p5eAAAAAAAAAAAAAAAAsVNiAQAAAAAAAAAAAAAAAA==Little EndianFor replica cn=replica,cn=dc\3Dipa\2Cdc\3Dmydomain\2Cdc\3Dcom,cn=mapping tree,cn=con fmtstr=[H6x3QH6x] size=40 len of nsstate is 40 CSN generator state: Replica ID : 3 Sampled Time : 1587474004 Gen as csn : 5e9eee54000000030000 Time as str : Tue Apr 21 15:00:04 2020 Local Offset : 0 Remote Offset : 23221169 Seq. num : 0 System time : Tue Apr 21 15:02:38 2020 Diff in sec. : 154 Day:sec diff : 0:154nsState is YQAAAAAAAAAuLZheAAAAAEUBAAAAAAAA7SYAAAAAAAASAAAAAAAAAA==Little EndianFor replica cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config fmtstr=[H6x3QH6x] size=40 len of nsstate is 40 CSN generator state: Replica ID : 97 Sampled Time : 1587031342 Gen as csn : 5e982d2e001800970000 Time as str : Thu Apr 16 12:02:22 2020 Local Offset : 325 Remote Offset : 9965 Seq. num : 18 System time : Tue Apr 21 15:02:38 2020 Diff in sec. : 442816 Day:sec diff : 5:10816* As you can see in the 1st node the Time as str is Jan 11 of 2021. With timedatectl command I see that both VMs use the same Time zone and the clock is correct. I found this old article to fix my issue: *https://www.redhat.com/archives/freeipa-users/2014-February/msg00007.html <https://www.redhat.com/archives/freeipa-users/2014-February/msg00007.html>* But ... I had the same issue in the past, always in the 1st server. So, in my mind I don't want to try to use that fix. I have a new hypervisor cluster, so I would prefer to reinstall the 1st server, using these steps: 1) check if all roles (also the CA) is installed in srv02 You can find here some data about the VMs: *[root@srv01 ~]# ipa server-show srv01.ipa.mydomain.com <http://srv01.ipa.mydomain.com> Server name: srv01.ipa.mydomain.com <http://srv01.ipa.mydomain.com> Managed suffixes: domain, ca Min domain level: 0 Max domain level: 1 Enabled server roles: CA server, IPA master, DNS server, NTP server, AD trust controller[root@srv02 ~]# ipa server-show srv02.ipa.mydomain.com <http://srv02.ipa.mydomain.com> Server name: srv02.ipa.mydomain.com <http://srv02.ipa.mydomain.com> Managed suffixes: domain, ca Min domain level: 0 Max domain level: 1 Enabled server roles: CA server, IPA master, DNS server, NTP server[root@srv01 ~]# ipa config-show Maximum username length: 32 Home directory base: /home Default shell: /bin/bash Default users group: ipausers Default e-mail domain: ipa.mydomain.com <http://ipa.mydomain.com> Search time limit: 2 Search size limit: 100 User search fields: uid,givenname,sn,telephonenumber,ou,title Group search fields: cn,description Enable migration mode: FALSE Certificate Subject base: O=IPA.MYDOMAIN.COM <http://IPA.MYDOMAIN.COM> Password Expiration Notification (days): 4 Password plugin features: AllowNThash SELinux user map order: guest_u:s0$xguest_u:s0$user_u:s0$staff_u:s0-s0:c0.c1023$unconfined_u:s0-s0:c0.c1023 Default SELinux user: unconfined_u:s0-s0:c0.c1023 Default PAC types: MS-PAC, nfs:NONE IPA masters: srv01.ipa.mydomain.com <http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com <http://srv02.ipa.mydomain.com> IPA CA servers: srv01.ipa.mydomain.com <http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com <http://srv02.ipa.mydomain.com> IPA NTP servers: srv01.ipa.mydomain.com <http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com <http://srv02.ipa.mydomain.com> IPA CA renewal master: srv01.ipa.mydomain.com <http://srv01.ipa.mydomain.com>[root@srv02 ~]# ipa config-show Maximum username length: 32 Home directory base: /home Default shell: /bin/bash Default users group: ipausers Default e-mail domain: ipa.mydomain.com <http://ipa.mydomain.com> Search time limit: 2 Search size limit: 100 User search fields: uid,givenname,sn,telephonenumber,ou,title Group search fields: cn,description Enable migration mode: FALSE Certificate Subject base: O=IPA.MYDOMAIN.COM <http://IPA.MYDOMAIN.COM> Password Expiration Notification (days): 4 Password plugin features: AllowNThash SELinux user map order: guest_u:s0$xguest_u:s0$user_u:s0$staff_u:s0-s0:c0.c1023$unconfined_u:s0-s0:c0.c1023 Default SELinux user: unconfined_u:s0-s0:c0.c1023 Default PAC types: MS-PAC, nfs:NONE IPA masters: srv01.ipa.mydomain.com <http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com <http://srv02.ipa.mydomain.com> IPA CA servers: srv01.ipa.mydomain.com <http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com <http://srv02.ipa.mydomain.com> IPA NTP servers: srv01.ipa.mydomain.com <http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com <http://srv02.ipa.mydomain.com> IPA CA renewal master: srv01.ipa.mydomain.com <http://srv01.ipa.mydomain.com>[root@srv01 ~]# ipactl statusDirectory Service: RUNNINGkrb5kdc Service: RUNNINGkadmin Service: RUNNINGnamed Service: RUNNINGhttpd Service: RUNNINGipa-custodia Service: RUNNINGntpd Service: RUNNINGpki-tomcatd Service: STOPPEDsmb Service: RUNNINGwinbind Service: RUNNINGipa-otpd Service: RUNNINGipa-dnskeysyncd Service: RUNNINGipa: INFO: The ipactl command was successful[root@srv02 ~]# ipactl statusDirectory Service: RUNNINGkrb5kdc Service: RUNNINGkadmin Service: RUNNINGnamed Service: RUNNINGhttpd Service: RUNNINGipa-custodia Service: RUNNINGntpd Service: RUNNINGpki-tomcatd Service: STOPPEDipa-otpd Service: RUNNINGipa-dnskeysyncd Service: RUNNINGipa: INFO: The ipactl command was successful[root@srv01 ~]# certutil -L -d /etc/pki/pki-tomcat/aliasCertificate Nickname Trust Attributes SSL,S/MIME,JAR/XPIServer-Cert cert-pki-ca u,u,usubsystemCert cert-pki-ca u,u,ucaSigningCert cert-pki-ca CTu,Cu,CuocspSigningCert cert-pki-ca u,u,uauditSigningCert cert-pki-ca u,u,Pu[root@srv02 ~]# certutil -L -d /etc/pki/pki-tomcat/aliasCertificate Nickname Trust Attributes SSL,S/MIME,JAR/XPIServer-Cert cert-pki-ca u,u,usubsystemCert cert-pki-ca u,u,ucaSigningCert cert-pki-ca CTu,u,uocspSigningCert cert-pki-ca u,u,uauditSigningCert cert-pki-ca u,u,Pu* It seems that AD trust controller role, IPA CA renewal master, smb and windbind are only in the 1st server. And also caSigningCert cert-pki-ca entry is different (CTu,Cu,Cu vs CTu,u,u). I can see only in the 1st server these DNS records: *_kerberos._tcp.Default-First-Site-Name._sites.dc._msdcs SRV 0 100 88 srv01_kerberos._tcp.dc._msdcs SRV 0 100 88 srv01_kerberos._udp.Default-First-Site-Name._sites.dc._msdcs SRV 0 100 88 srv01_kerberos._udp.dc._msdcs SRV 0 100 88 srv01_ldap._tcp.Default-First-Site-Name._sites.dc._msdcs 0 100 389 srv01_ldap._tcp.dc._msdcs 0 100 389 srv01* Srv01 is the first master, I know, but is the server VM that has clock problems, in both situations. So I want to keep srv02 and install a new one. What do I have to do to let the 2nd VM be a single server? Could I use these URLs? *https://www.freeipa.org/page/Backup_and_Restore#One_Server_Loss_-_First_Master <https://www.freeipa.org/page/Backup_and_Restore#One_Server_Loss_-_First_Master>https://www.freeipa.org/page/V4/Server_Roles#Upgrade <https://www.freeipa.org/page/V4/Server_Roles#Upgrade>* 2) uninstall ipa-server from the 1st server (srv01) and then powering off it, assuming that all data into the 2nd one are ok (srv02) 3) update freeipa and all other RPM packages into the VM srv02 4) install a new fresh VM, always with 7 release, and create a new replica Could I use the same old hostname (srv01) and IP address for this new VM? Or is better to use the same IP but a new name, like srv03? Do you think this is the right way to solve my issue? Or do you have any better idea? Please let me know, thanks. Bye, Morgan
_______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
