I'm now noticing in /var/log/dirsrv/slapd-*/errors a bunch of lines like this:
WARN - csngen_new_csn - Too much time skew (-15785961 secs). Current seqnum=1a22 And so on. All 3 servers are correctly time-synced to our internal NTP server, so could this be something internal? A counter of some kind? Bret Wortman Founder, Damascus Products, LLC 855-644-2783 (tel:855-644-2783) | b...@wrapbuddies.co (https://link.getmailspring.com/link/8dac0bd8-567a-4d4c-8651-f3966bdd4...@getmailspring.com/0?redirect=mailto%3Abret%40wrapbuddies.co&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn) http://wrapbuddies.co/ (https://link.getmailspring.com/link/8dac0bd8-567a-4d4c-8651-f3966bdd4...@getmailspring.com/1?redirect=http%3A%2F%2Fwrapbuddies.co%2F&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn) 70 Main St. Suite 23 Warrenton, VA 20186 On Mar 26 2019, at 11:43 am, Bret Wortman via FreeIPA-users <freeipa-users@lists.fedorahosted.org> wrote: > > On Mar 26 2019, at 11:10 am, Florence Blanc-Renaud <f...@redhat.com> wrote: > > On 3/26/19 2:23 PM, Bret Wortman via FreeIPA-users wrote: > > > I broke out of it, but the two are still out of sync. Is there a way to > > > get past that? > > > > > > > > > photo > > > *Bret Wortman* > > > Founder, Damascus Products, LLC > > > > > > 855-644-2783 <tel:855-644-2783> | b...@wrapbuddies.co > > > <https://link.getmailspring.com/link/76fbb986-2615-4565-a74d-e3c1d7a38...@getmailspring.com/0?redirect=mailto%3Abret%40wrapbuddies.co&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > > > > http://wrapbuddies.co/ > > > <https://link.getmailspring.com/link/76fbb986-2615-4565-a74d-e3c1d7a38...@getmailspring.com/1?redirect=http%3A%2F%2Fwrapbuddies.co%2F&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > > > > 70 Main St. Suite 23 Warrenton, VA 20186 > > > <https://link.getmailspring.com/link/76fbb986-2615-4565-a74d-e3c1d7a38...@getmailspring.com/2?redirect=http%3A%2F%2Ffacebook.com%2Fwrapbuddiesco&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > <https://link.getmailspring.com/link/76fbb986-2615-4565-a74d-e3c1d7a38...@getmailspring.com/3?redirect=http%3A%2F%2Fwww.linkedin.com%2Fin%2Fbretwortman&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > > > > <https://link.getmailspring.com/link/76fbb986-2615-4565-a74d-e3c1d7a38...@getmailspring.com/4?redirect=http%3A%2F%2Ftwitter.com%2Fwrapbuddiesco&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > > > > <https://link.getmailspring.com/link/76fbb986-2615-4565-a74d-e3c1d7a38...@getmailspring.com/5?redirect=http%3A%2F%2Finstagram.com%2Fwrapbuddies&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > > > > On Mar 26 2019, at 9:07 am, Rob Crittenden <rcrit...@redhat.com> wrote: > > > Bret Wortman via FreeIPA-users wrote: > > > Oops. I spoke too soon. The one I thought I fixed is now just > > > scrolling > > > "No status yet" over and over... > > > > > > > > > You can break out of that. There is a bug where we are checking the > > > wrong status. I can't find the BZ at the moment but IIRC it will be > > > fixed in the next release. > > > > > > > The BZ is https://bugzilla.redhat.com/show_bug.cgi?id=1666843 > > > rob > > > > > > > > > photo > > > *Bret Wortman* > > > Founder, Damascus Products, LLC > > > > > > 855-644-2783 <tel:855-644-2783> | b...@wrapbuddies.co > > > <https://link.getmailspring.com/link/1183d1dd-2462-44d7-a501-d9f2a79e8...@getmailspring.com/0?redirect=mailto%3Abret%40wrapbuddies.co&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > > > > http://wrapbuddies.co/ > > > <https://link.getmailspring.com/link/1183d1dd-2462-44d7-a501-d9f2a79e8...@getmailspring.com/1?redirect=http%3A%2F%2Fwrapbuddies.co%2F&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > > > > 70 Main St. Suite 23 Warrenton, VA 20186 > > > <https://link.getmailspring.com/link/1183d1dd-2462-44d7-a501-d9f2a79e8...@getmailspring.com/2?redirect=http%3A%2F%2Ffacebook.com%2Fwrapbuddiesco&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > <https://link.getmailspring.com/link/1183d1dd-2462-44d7-a501-d9f2a79e8...@getmailspring.com/3?redirect=http%3A%2F%2Fwww.linkedin.com%2Fin%2Fbretwortman&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > <https://link.getmailspring.com/link/1183d1dd-2462-44d7-a501-d9f2a79e8...@getmailspring.com/4?redirect=http%3A%2F%2Ftwitter.com%2Fwrapbuddiesco&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > <https://link.getmailspring.com/link/1183d1dd-2462-44d7-a501-d9f2a79e8...@getmailspring.com/5?redirect=http%3A%2F%2Finstagram.com%2Fwrapbuddies&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > > > > On Mar 26 2019, at 8:54 am, Bret Wortman > > > <bret.wort...@damascusgrp.com> > > > wrote: > > > > > > One had a clock skew error (fixed), but the other non-CA replica > > > shows: > > > > > > ipa3.spx.net: > > > <https://link.getmailspring.com/link/1183d1dd-2462-44d7-a501-d9f2a79e8...@getmailspring.com/6?redirect=ipa3.spx.net%3A&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn>replica > > > last init status: None > > > last init ended: 1970-01-01 00:00:00+00:00 > > > last update status: Error (3) Replication error acquiring > > > replica: Unable to acquire replica: permission denied. The bind dn > > > does not have permission to supply replication updates to the > > > replica. Will retry later. (permission denied) > > > > > > Do I need to re-init this replica from scratch (as in, remove it, > > > unbind it from the servers, re-add it as a client and then > > > re-promote it)? > > > > > The "init" status is updated when a full reinitialization is done, not > > during normal replication updates. The "last update status" is the > > relevant information in your case. > > > > > Ours is still showing that status from 2019-03-13. > > > > Can you check if each master has a valid keytab and is able to use this > > keytab to authenticate to the other masters? See > > https://www.freeipa.org/page/Troubleshooting/Directory_Server#Replication_issues > > > > (https://link.getmailspring.com/link/8dac0bd8-567a-4d4c-8651-f3966bdd4...@getmailspring.com/6?redirect=https%3A%2F%2Flink.getmailspring.com%2Flink%2F96DADE96-C434-437D-AF79-883C922FEB0A%40getmailspring.com%2F0%3Fredirect%3Dhttps%253A%252F%252Fwww.freeipa.org%252Fpage%252FTroubleshooting%252FDirectory_Server%2523Replication_issues%26recipient%3DZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn) > > > > > The two ldapsearches worked on both replicas having issues. > > > > What is your 389-ds version? > 1.3.8.4-22 > (https://link.getmailspring.com/link/8dac0bd8-567a-4d4c-8651-f3966bdd4...@getmailspring.com/7?redirect=https%3A%2F%2Flink.getmailspring.com%2Flink%2F96DADE96-C434-437D-AF79-883C922FEB0A%40getmailspring.com%2F1%3Fredirect%3D1.3.8.4-22%26recipient%3DZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn) > on CentOS 7. > > You may check that the group "cn=replication > > managers,cn=sysaccounts,cn=etc,$BASEDN" contains as member all your > > replication principals, for instance: > > > > dn: cn=replication managers,cn=sysaccounts,cn=etc,$BASEDN > > cn: replication managers > > member: > > krbprincipalname=ldap/master.domain....@domain.com,cn=services,cn=accounts,$BASEDN > > member: > > krbprincipalname=ldap/replica.domain....@domain.com,cn=services,cn=accounts,$BASEDN > > > > and that the group is configured as nsds5replicabinddngroup in > > cn=replica,cn=dc\3Ddomain\2Cdc\3Dcom,cn=mapping tree,cn=config > > > > If you have an older version, I believe nsds5replicabinddn is used > > instead of nsds5replicabinddngroup. > > > > To try to get replication flowing again, I stopped and started IPA on the > ipa5 server (using ipactl stop && ipactl start), and now: > # ipa-replica-manage list > ipa3.my.net: > (https://link.getmailspring.com/link/8dac0bd8-567a-4d4c-8651-f3966bdd4...@getmailspring.com/8?redirect=https%3A%2F%2Flink.getmailspring.com%2Flink%2F96DADE96-C434-437D-AF79-883C922FEB0A%40getmailspring.com%2F2%3Fredirect%3Dipa3.my.net%253A%26recipient%3DZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn) > master > ipa4.my.net: > (https://link.getmailspring.com/link/8dac0bd8-567a-4d4c-8651-f3966bdd4...@getmailspring.com/9?redirect=https%3A%2F%2Flink.getmailspring.com%2Flink%2F96DADE96-C434-437D-AF79-883C922FEB0A%40getmailspring.com%2F3%3Fredirect%3Dipa3.my.net%253A%26recipient%3DZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn) > master > ipa5.my.net: > (https://link.getmailspring.com/link/8dac0bd8-567a-4d4c-8651-f3966bdd4...@getmailspring.com/10?redirect=https%3A%2F%2Flink.getmailspring.com%2Flink%2F96DADE96-C434-437D-AF79-883C922FEB0A%40getmailspring.com%2F4%3Fredirect%3Dipa3.my.net%253A%26recipient%3DZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn) > master > # ipa-replica-manage list -v ipa5.spx.net > (https://link.getmailspring.com/link/8dac0bd8-567a-4d4c-8651-f3966bdd4...@getmailspring.com/11?redirect=https%3A%2F%2Flink.getmailspring.com%2Flink%2F96DADE96-C434-437D-AF79-883C922FEB0A%40getmailspring.com%2F5%3Fredirect%3Dipa5.spx.net%26recipient%3DZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn) > # > > > In fact, ipa-replica-manage list with a hostname on any of our servers > returns nothing now. > > HTH, > > flo > > > > > > > > photo > > > *Bret Wortman* > > > Founder, Damascus Products, LLC > > > > > > 855-644-2783 <tel:855-644-2783> | b...@wrapbuddies.co > > > <https://link.getmailspring.com/link/1183d1dd-2462-44d7-a501-d9f2a79e8...@getmailspring.com/7?redirect=mailto%3Abret%40wrapbuddies.co&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > > > > http://wrapbuddies.co/ > > > <https://link.getmailspring.com/link/1183d1dd-2462-44d7-a501-d9f2a79e8...@getmailspring.com/8?redirect=http%3A%2F%2Fwrapbuddies.co%2F&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > > > > 70 Main St. Suite 23 Warrenton, VA 20186 > > > <https://link.getmailspring.com/link/1183d1dd-2462-44d7-a501-d9f2a79e8...@getmailspring.com/9?redirect=http%3A%2F%2Ffacebook.com%2Fwrapbuddiesco&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > <https://link.getmailspring.com/link/1183d1dd-2462-44d7-a501-d9f2a79e8...@getmailspring.com/10?redirect=http%3A%2F%2Fwww.linkedin.com%2Fin%2Fbretwortman&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > <https://link.getmailspring.com/link/1183d1dd-2462-44d7-a501-d9f2a79e8...@getmailspring.com/11?redirect=http%3A%2F%2Ftwitter.com%2Fwrapbuddiesco&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > <https://link.getmailspring.com/link/1183d1dd-2462-44d7-a501-d9f2a79e8...@getmailspring.com/12?redirect=http%3A%2F%2Finstagram.com%2Fwrapbuddies&recipient=ZnJlZWlwYS11c2Vyc0BsaXN0cy5mZWRvcmFob3N0ZWQub3Jn> > > > > > > On Mar 26 2019, at 8:47 am, Rob Crittenden <rcrit...@redhat.com> > > > wrote: > > > > > > Bret Wortman via FreeIPA-users wrote: > > > Looks like I've somehow managed to get my 3 IPA servers out > > > of sync: > > > > > > [root@ipa3 ~]# ipa-replica-manage list > > > ipa3.my.net:master > > > ipa4.my.net:master > > > ipa5.my.net:master > > > [root@ipa3 ~]# ipa host-find solr14.my.net > > > --------------- > > > 0 hosts matched > > > --------------- > > > ---------------------------- > > > Number of entries returned 0 > > > ---------------------------- > > > > > > On ipa4: > > > [root@ipa3 ~]# ipa host-find solr14.my.net > > > --------------- > > > 1 hosts matched > > > --------------- > > > Host name: solr14.my.net > > > ---------------------------- > > > Number of entries returned 1 > > > ---------------------------- > > > > > > On ipa5: > > > [root@ipa3 ~]# ipa host-find solr14.my.net > > > --------------- > > > 1 hosts matched > > > --------------- > > > Host name: solr14.my.net > > > Principal name: host/solr14.my....@my.net > > > <mailto:host/solr14.my....@my.net> > > > : > > > : > > > ---------------------------- > > > Number of entries returned 1 > > > ---------------------------- > > > > > > So they've obviously stopped talking. What's the right way > > > to get them > > > back in sync and ensure that they don't drift again? Is there a > > > replication entry that's "stuck" and causing this? > > > > > > > > > On each master run: ipa-replica-manage list -v `hostname` > > > That will give you the replication status. > > > You can try to wake up an agreement with: ipa-replica-manage > > > force-sync > > > --from <host> > > > > > > rob > > > Sent from Mailspring > > > > > > _______________________________________________ > > > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org > > > To unsubscribe send an email to > > > freeipa-users-le...@lists.fedorahosted.org > > > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > > > List Guidelines: > > > https://fedoraproject.org/wiki/Mailing_list_guidelines > > > List Archives: > > > https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org > > > > > > Sent from Mailspring > > > _______________________________________________ > > > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org > > > To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org > > > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > > > List Archives: > > > https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org > > > > > > > _______________________________________________ > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org > To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org >
_______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
