[Freeipa-users] Replication broken after upgrade
Hello there,
Something went wrong after recent yum update (CentOS 7)
The current version is 4.6.8-5.el7.centos.9
I have two FreeIPA replicas and one Active Directory agreement (winsync)
Here what i'm getting from cn=replicacn=mapping tree,cn=config
nsds5replicaLastUpdateStart: 1970010100Z
nsds5replicaLastUpdateEnd: 1970010100Z
nsds5replicaLastInitStart: 1970010100Z
nsds5replicaLastInitEnd: 1970010100Z
This is for both agreements, however winsync is still alive somehow.
Replication to the second FreeIPA node no longer works, and
when trying to re-initialize, here's what i'm getting:
ipa-replica-manage re-initialize --from= --verbose
Traceback (most recent call last):
File "/sbin/ipa-replica-manage", line 1624, in
main(options, args)
File "/sbin/ipa-replica-manage", line 1567, in main
options.nolookup)
File "/sbin/ipa-replica-manage", line 1220, in re_initialize
repl.initialize_replication(agreement.dn, repl.conn)
File "/usr/lib/python2.7/site-packages/ipaserver/install/replication.py",
line 1358, in initialize_replication
conn.modify_s(dn, mod)
File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 792,
in modify_s
return self.conn.modify_s(dn, modlist)
File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 357,
in modify_s
return self.result(msgid,all=1,timeout=self.timeout)
File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 458,
in result
resp_type, resp_data, resp_msgid = self.result2(msgid,all,timeout)
File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 462,
in result2
resp_type, resp_data, resp_msgid, resp_ctrls =
self.result3(msgid,all,timeout)
File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 469,
in result3
resp_ctrl_classes=resp_ctrl_classes
File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 476,
in result4
ldap_result =
self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop)
File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 99, in
_ldap_call
result = func(*args,**kwargs)
TYPE_OR_VALUE_EXISTS: {'desc': 'Type or value exists'}
Unexpected error: {'desc': 'Type or value exists'}
I feel that the exception is related to time set to 1970010100Z or some
other cn=config parameter.
Another suspicious thing which may be related is:
Running on node0:
ipa-replica-manage list -v
Failed to get data from 'node1': Insufficient access: SASL(-1): generic
failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide
more information (Server krbtgt/ not found in
Kerberos database)
Any advice on how to fix without rebuilding everything ?
Thank you
___
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
[Freeipa-users] Replication broken
Hello,
I'm on freeipa 4.9.0 on CentOS Stream. (1 master and 1 replica)
I have noticed that my replication is broken. Unfortunatly, I don't know since
when...
First Question, can it b fixed?
Second question, is it possible to peform a restore (on one node, both nodes)
to fix the issue.
I recently upgraded from CentOS 8 to CentOS Stream (ipa with it). So can I
restore from a previous version?
Here are some snipets of what I see.
$ sudo ipa-healthcheck
Internal server error HTTPSConnectionPool(host='ipa-master-tmp.empire.lan',
port=443): Max retries exceeded with url:
/ca/rest/certs/search?size=3 (Caused by
NewConnectionError(': Failed to
establish a new connection: [Errno -2] Name or service not known',))
[
{
"source": "pki.server.healthcheck.clones.connectivity_and_data",
"check": "ClonesConnectivyAndDataCheck",
"result": "ERROR",
"uuid": "66815b82-56d9-43a4-9035-78333c5cb5cd",
"when": "20210308162643Z",
"duration": "0.364202",
"kw": {
"status": "ERROR: pki-tomcat : Internal error testing CA clone. Host:
ipa-master-tmp.empire.lan Port: 443"
}
},
{
"source": "ipahealthcheck.ds.replication",
"check": "ReplicationCheck",
"result": "WARNING",
"uuid": "55addd45-6440-4317-8d0b-8eb0d516bd4e",
"when": "20210308162645Z",
"duration": "0.353734",
"kw": {
"key": "DSREPLLE0002",
"items": [
"Replication",
"Conflict Entries"
],
"msg": "There were 6 conflict entries found under the replication suffix
\"dc=empire,dc=lan\"."
}
}
]
pki-tomcatd seems ok :
$ sudo journalctl -u pki-tomcatd@pki-tomcat
-- Logs begin at Mon 2021-03-08 17:24:39 CET, end at Mon 2021-03-08 17:35:01
CET. --
Mar 08 17:25:01 ipa-master.empire.lan systemd[1]: Starting PKI Tomcat Server
pki-tomcat...
Mar 08 17:25:04 ipa-master.empire.lan java[1613]: usr/lib/api/apiutil.c Could
not open /run/lock/opencryptoki/LCK..APIlock
Mar 08 17:25:05 ipa-master.empire.lan server[1716]: Java virtual machine used:
/usr/lib/jvm/java-1.8.0-openjdk/bin/java
Mar 08 17:25:05 ipa-master.empire.lan server[1716]: classpath used:
/usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-
juli.jar:/usr/share/java/ant.jar:/usr/share/java/ant-la>
Mar 08 17:25:05 ipa-master.empire.lan server[1716]: main class used:
org.apache.catalina.startup.Bootstrap
Mar 08 17:25:05 ipa-master.empire.lan server[1716]: flags used:
-Dcom.redhat.fips=false
Mar 08 17:25:05 ipa-master.empire.lan server[1716]: options used:
-Dcatalina.base=/var/lib/pki/pki-tomcat -Dcatalina.home=/usr/share/tomcat
-Djava.endorsed.dirs= -Djava.io.tmpdir=/var/lib/>
Mar 08 17:25:05 ipa-master.empire.lan server[1716]: arguments used: start
Mar 08 17:25:05 ipa-master.empire.lan ipa-pki-wait-running[1717]: pki.client:
/usr/libexec/ipa/ipa-pki-wait-running:63: The subsystem in
PKIConnection.__init__() has been deprecated (https>
Mar 08 17:25:05 ipa-master.empire.lan ipa-pki-wait-running[1717]:
ipa-pki-wait-running: Created connection
http://ipa-master.empire.lan:8080/ca
Mar 08 17:25:05 ipa-master.empire.lan ipa-pki-wait-running[1717]:
ipa-pki-wait-running: Connection failed: HTTPConnectionPool(host='ipa-
master.empire.lan', port=8080): Max retries exceeded>
Mar 08 17:25:06 ipa-master.empire.lan java[1716]: usr/lib/api/apiutil.c Could
not open /run/lock/opencryptoki/LCK..APIlock
Mar 08 17:25:06 ipa-master.empire.lan server[1716]: WARNING: Some of the
specified [protocols] are not supported by the SSL engine and have
been skipped: [[TLSv1, TLSv1.1]]
Mar 08 17:25:07 ipa-master.empire.lan ipa-pki-wait-running[1717]:
ipa-pki-wait-running: Connection failed: HTTPConnectionPool(host='ipa-
master.empire.lan', port=8080): Read timed out. (rea>
Mar 08 17:25:09 ipa-master.empire.lan ipa-pki-wait-running[1717]:
ipa-pki-wait-running: Connection failed: HTTPConnectionPool(host='ipa-
master.empire.lan', port=8080): Read timed out. (rea>
Mar 08 17:25:11 ipa-master.empire.lan ipa-pki-wait-running[1717]:
ipa-pki-wait-running: Connection failed: HTTPConnectionPool(host='ipa-
master.empire.lan', port=8080): Read timed out. (rea>
Mar 08 17:25:12 ipa-master.empire.lan ipa-pki-wait-running[1717]:
ipa-pki-wait-running: Success, subsystem ca is running!
Mar 08 17:25:12 ipa-master.empire.lan systemd[1]: Started PKI Tomcat Server
pki-tomcat.
Best
Antoine
___
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
[Freeipa-users] replication broken
So for some reason yesterday my replication broke. Checked out the logs and found this:Mar 20 14:16:02 freeipa01 systemd: ipa-dnskeysyncd.service: main process exited, code=exited, status=1/FAILUREMar 20 14:16:02 freeipa01 systemd: Unit ipa-dnskeysyncd.service entered failed state.Mar 20 14:16:02 freeipa01 systemd: ipa-dnskeysyncd.service failed.Mar 20 14:17:02 freeipa01 systemd: ipa-dnskeysyncd.service holdoff time over, scheduling restart.Mar 20 14:17:02 freeipa01 systemd: Started IPA key daemon.Mar 20 14:17:02 freeipa01 systemd: Starting IPA key daemon...Mar 20 14:17:05 freeipa01 ipa-dnskeysyncd: ipa : INFO LDAP bind...Mar 20 14:17:05 freeipa01 ipa-dnskeysyncd: ipa : INFO Commencing sync processMar 20 14:17:05 freeipa01 ipa-dnskeysyncd: ipa.ipaserver.dnssec.keysyncer.KeySyncer: INFO Initial LDAP dump is done, sychronizing with ODS and BINDMar 20 14:17:09 freeipa01 ipa-dnskeysyncd: Traceback (most recent call last):Mar 20 14:17:09 freeipa01 ipa-dnskeysyncd: File "/usr/libexec/ipa/ipa-dnskeysyncd", line 114, in Mar 20 14:17:09 freeipa01 ipa-dnskeysyncd: while ldap_connection.syncrepl_poll(all=1, msgid=ldap_search):Mar 20 14:17:09 freeipa01 ipa-dnskeysyncd: File "/usr/lib64/python2.7/site-packages/ldap/syncrepl.py", line 405, in syncrepl_pollMar 20 14:17:09 freeipa01 ipa-dnskeysyncd: self.syncrepl_refreshdone()Mar 20 14:17:09 freeipa01 ipa-dnskeysyncd: File "/usr/lib/python2.7/site-packages/ipaserver/dnssec/keysyncer.py", line 115, in syncrepl_refreshdoneMar 20 14:17:09 freeipa01 ipa-dnskeysyncd: self.hsm_replica_sync()Mar 20 14:17:09 freeipa01 ipa-dnskeysyncd: File "/usr/lib/python2.7/site-packages/ipaserver/dnssec/keysyncer.py", line 181, in hsm_replica_syncMar 20 14:17:09 freeipa01 ipa-dnskeysyncd: ipautil.run([paths.IPA_DNSKEYSYNCD_REPLICA])Mar 20 14:17:09 freeipa01 ipa-dnskeysyncd: File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 512, in runMar 20 14:17:09 freeipa01 ipa-dnskeysyncd: raise CalledProcessError(p.returncode, arg_string, str(output))Mar 20 14:17:09 freeipa01 ipa-dnskeysyncd: subprocess.CalledProcessError: Command '/usr/libexec/ipa/ipa-dnskeysync-replica' returned non-zero exit status 1Mar 20 14:17:09 freeipa01 systemd: ipa-dnskeysyncd.service: main process exited, code=exited, status=1/FAILUREMar 20 14:17:09 freeipa01 systemd: Unit ipa-dnskeysyncd.service entered failed state.Mar 20 14:17:09 freeipa01 systemd: ipa-dnskeysyncd.service failed.Mar 20 14:17:39 freeipa01 su: (to root) gatewayblend on pts/0Mar 20 14:17:39 freeipa01 dbus[742]: [system] Activating service name='org.freedesktop.problems' (using servicehelper)Mar 20 14:17:39 freeipa01 dbus-daemon: dbus[742]: [system] Activating service name='org.freedesktop.problems' (using servicehelper)Mar 20 14:17:39 freeipa01 dbus[742]: [system] Successfully activated service 'org.freedesktop.problems'Mar 20 14:17:39 freeipa01 dbus-daemon: dbus[742]: [system] Successfully activated service 'org.freedesktop.problems'Mar 20 14:18:09 freeipa01 systemd: ipa-dnskeysyncd.service holdoff time over, scheduling restart.Mar 20 14:18:09 freeipa01 systemd: Started IPA key daemon.Mar 20 14:18:09 freeipa01 systemd: Starting IPA key daemon...Mar 20 14:18:13 freeipa01 ipa-dnskeysyncd: ipa : INFO LDAP bind...Mar 20 14:18:13 freeipa01 ipa-dnskeysyncd: ipa : INFO Commencing sync processMar 20 14:18:13 freeipa01 ipa-dnskeysyncd: ipa.ipaserver.dnssec.keysyncer.KeySyncer: INFO Initial LDAP dump is done, sychronizing with ODS and BINDMar 20 14:18:17 freeipa01 ipa-dnskeysyncd: Traceback (most recent call last):Mar 20 14:18:17 freeipa01 ipa-dnskeysyncd: File "/usr/libexec/ipa/ipa-dnskeysyncd", line 114, in Mar 20 14:18:17 freeipa01 ipa-dnskeysyncd: while ldap_connection.syncrepl_poll(all=1, msgid=ldap_search):Mar 20 14:18:17 freeipa01 ipa-dnskeysyncd: File "/usr/lib64/python2.7/site-packages/ldap/syncrepl.py", line 405, in syncrepl_pollMar 20 14:18:17 freeipa01 ipa-dnskeysyncd: self.syncrepl_refreshdone()Mar 20 14:18:17 freeipa01 ipa-dnskeysyncd: File "/usr/lib/python2.7/site-packages/ipaserver/dnssec/keysyncer.py", line 115, in syncrepl_refreshdoneMar 20 14:18:17 freeipa01 ipa-dnskeysyncd: self.hsm_replica_sync()Mar 20 14:18:17 freeipa01 ipa-dnskeysyncd: File "/usr/lib/python2.7/site-packages/ipaserver/dnssec/keysyncer.py", line 181, in hsm_replica_syncMar 20 14:18:17 freeipa01 ipa-dnskeysyncd: ipautil.run([paths.IPA_DNSKEYSYNCD_REPLICA])Mar 20 14:18:17 freeipa01 ipa-dnskeysyncd: File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 512, in runMar 20 14:18:17 freeipa01 ipa-dnskeysyncd: raise CalledProcessError(p.returncode, arg_string, str(output))Mar 20 14:18:17 freeipa01 ipa-dnskeysyncd: subprocess.CalledProcessError: Command '/usr/libexec/ipa/ipa-dnskeysync-replica' returned non-zero exit status 1Mar 20 14:18:17 freeipa01 systemd: ipa-dnskeysyncd.service: main process exited, code=exited, status=1/FAILUREMar 20
