Re: [Freeipa-users] Windows sync agreement becomes uninitialized and crashes directory server
On 07/13/2015 07:07 PM, nat...@nathanpeters.com wrote:
2 FreeIPA 4.1.4 servers running on CentOS 7.
dc1 has a sync agreement to a windows server.
It has been running fine since June 5 when I re-initialized a sync
agreement that had somehow uninitialized itself. Original issue report
here :
https://www.redhat.com/archives/freeipa-users/2015-June/msg00147.html
Bug report here : https://fedorahosted.org/freeipa/ticket/5054
It appears the same thing may have happened again, one month later, but
this time randomly, as we have not made any changes to our sync agreement
since the initial change in June. it appears to have unitialized itself
without us changing it and managed to crash the directory server in doing
so.
Crash? http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-crashes
# debuginfo-install 389-ds-base ipa-server slapi-nis
Note that during the last week I could still login to the web ui, but
around the time the log entries change, I became unable to.
I tried to login to the web server today and it would not let me login, so
I went to the shell on the server and noticed that ipactl command would
freeze up again. I looked at the logs (which I will paste below) and
restarted the directory server service.
I then found that my sync agreement had become uninitialized.
--- output ---
[root@dc1 slapd-IPADOMAIN-NET]# ldapsearch -xLLL -D cn=directory manager
-W -b cn=config objectclass=nsDSWindowsReplicationAgreement
Enter LDAP Password:
dn: cn=meToofficedc2.office.addomain.net,cn=replica,cn=dc\3Dipadomain
\2Cdc\3Dnet,cn=mapping tree,cn=config
nsds7WindowsReplicaSubtree: OU=Staff,DC=office,DC=addomain,DC=net
nsds7DirectoryReplicaSubtree: cn=users,cn=accounts,dc=ipadomain,dc=net
cn: meToofficedc2.office.addomain.net
nsds7NewWinGroupSyncEnabled: false
objectClass: nsDSWindowsReplicationAgreement
objectClass: top
nsDS5ReplicaTransportInfo: TLS
description: me to officedc2.office.addomain.net
nsDS5ReplicaRoot: dc=ipadomain,dc=net
nsDS5ReplicaHost: officedc2.office.addomain.net
nsds5replicaTimeout: 120
nsDS5ReplicaBindDN: cn=freeipa syncuser,ou=Service
Account,dc=office,dc=addomain,dc=net
nsds7NewWinUserSyncEnabled: true
nsDS5ReplicaPort: 389
nsds7WindowsDomain: ipadomain.net
nsDS5ReplicatedAttributeList: (objectclass=*) $ EXCLUDE memberof
idnssoaserial
entryusn krblastsuccessfulauth krblastfailedauth krbloginfailedcount
nsDS5ReplicaBindMethod: simple
nsDS5ReplicaCredentials:
{AES-TUhNR0NTcUdTSWIzRFFFRkRUQm1NRVVHQ1NxR1NJYjNEUUVG
RERBNEJDUmtOelUzTTJJNVlpMDBaV1EyTTJRMQ0KWXkwNU0yTm1aV05sTVMxbU5qRXpaak5oTlFBQ
0FRSUNBU0F3Q2dZSUtvWklodmNOQWdjd0hRWUpZSVpJQVdVRA0KQkFFcUJCQ2k0N0NxRGZFd2JIdm
I0MFVFZVI3MA==}gWI9NIB8lbt9tmNszzbBFCAe4Vs/e0sMyn5+NZPJg9E=
nsds7DirsyncCookie::
TVNEUwMAAABoJPGME7jQAQAAYAEAAPc1qQAAA
AD3NakAAMUjuImqVZhBkOkdt24C0IsBAA4AY4GwFkVcvEmMMExrVon4d6
13PwAADGzFNzznrESIxHzA74fbsz4HUgAAOnFoO5OE2E27lR/g4EcjQTLbIwAAuEm
PWjYok0qGS0HM/+TDmK7FgAMA6PTFXvAdnkaJSIkZT1lS+/FcJAAA4qTQaC46/Ua4KXgP
/ixNcbK4dgAAWowbgYD1akibZ+sCul5C4VNsMQAAxSO4iapVmEGQ6R23bgLQi/c1qQAAA
AAAogC6jFcyFUmhBp4B7FkaBcRHwwEAyhKMxsP0uUKGEnG2lsyA8eTUwgYA4n8Xx1bAlU
mBUl3zhlZ9WBngDAAA71vM2ebFEkCJkBaLjB4CGU+4CQMAGfO+4ndZCkaVKnwZNlNsf90
NDAAAgD6n+M2bcUGkOwo5gPLx7IOjAwAA
nsds50ruv: {replicageneration} 553fe9bb0004
nsds50ruv: {replica 4 ldap://dc1.ipadomain.net:389} 553fe9c9
0004 557f49fb00020004
nsds50ruv: {replica 3 ldap://dc2.ipadomain.net:389} 553fe9c
40003 557f3e4a00020003
nsruvReplicaLastModified: {replica 4 ldap://dc1.ipadomain.ne
t:389} 557f494a
nsruvReplicaLastModified: {replica 3 ldap://dc2.ipadomain.n
et:389} 557f3d95
oneWaySync: fromWindows
nsds5ReplicaEnabled: on
nsds5replicareapactive: 0
nsds5replicaLastUpdateStart: 0
nsds5replicaLastUpdateEnd: 0
nsds5replicaChangesSentSinceStartup:
nsds5replicaLastUpdateStatus: -1 - LDAP error: Can't contact LDAP server
nsds5replicaUpdateInProgress: FALSE
nsds5replicaLastInitStart: 0
nsds5replicaLastInitEnd: 0
--- output ---
Here are the error logs for the last month for the directory server. They
are totally empty until July 2.
---output---
389-Directory/1.3.3.8 B2015.040.128
dc1.ipadomain.net:636 (/etc/dirsrv/slapd-IPADOMAIN-NET)
[02/Jul/2015:03:19:02 +] NSMMReplicationPlugin - windows sync - failed
to send dirsync search request: 2
[02/Jul/2015:06:10:29 +] - Entry
uid=jenkinsdev,cn=users,cn=accounts,dc=ipadomain,dc=net missing
attribute sn required by object class person
[03/Jul/2015:02:04:02 +] NSMMReplicationPlugin - windows sync - failed
to send dirsync search request: 2
[03/Jul/2015:05:39:01 +] NSMMReplicationPlugin - windows sync - failed
to send dirsync search request: 2
[03/Jul/2015:17:09:00 +] NSMMReplicationPlugin - windows sync - failed
to send dirsync search request: 2
[03/Jul/2015:22:41:32 +] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id
[Freeipa-users] Windows sync agreement becomes uninitialized and crashes directory server
2 FreeIPA 4.1.4 servers running on CentOS 7.
dc1 has a sync agreement to a windows server.
It has been running fine since June 5 when I re-initialized a sync
agreement that had somehow uninitialized itself. Original issue report
here :
https://www.redhat.com/archives/freeipa-users/2015-June/msg00147.html
Bug report here : https://fedorahosted.org/freeipa/ticket/5054
It appears the same thing may have happened again, one month later, but
this time randomly, as we have not made any changes to our sync agreement
since the initial change in June. it appears to have unitialized itself
without us changing it and managed to crash the directory server in doing
so. Note that during the last week I could still login to the web ui, but
around the time the log entries change, I became unable to.
I tried to login to the web server today and it would not let me login, so
I went to the shell on the server and noticed that ipactl command would
freeze up again. I looked at the logs (which I will paste below) and
restarted the directory server service.
I then found that my sync agreement had become uninitialized.
--- output ---
[root@dc1 slapd-IPADOMAIN-NET]# ldapsearch -xLLL -D cn=directory manager
-W -b cn=config objectclass=nsDSWindowsReplicationAgreement
Enter LDAP Password:
dn: cn=meToofficedc2.office.addomain.net,cn=replica,cn=dc\3Dipadomain
\2Cdc\3Dnet,cn=mapping tree,cn=config
nsds7WindowsReplicaSubtree: OU=Staff,DC=office,DC=addomain,DC=net
nsds7DirectoryReplicaSubtree: cn=users,cn=accounts,dc=ipadomain,dc=net
cn: meToofficedc2.office.addomain.net
nsds7NewWinGroupSyncEnabled: false
objectClass: nsDSWindowsReplicationAgreement
objectClass: top
nsDS5ReplicaTransportInfo: TLS
description: me to officedc2.office.addomain.net
nsDS5ReplicaRoot: dc=ipadomain,dc=net
nsDS5ReplicaHost: officedc2.office.addomain.net
nsds5replicaTimeout: 120
nsDS5ReplicaBindDN: cn=freeipa syncuser,ou=Service
Account,dc=office,dc=addomain,dc=net
nsds7NewWinUserSyncEnabled: true
nsDS5ReplicaPort: 389
nsds7WindowsDomain: ipadomain.net
nsDS5ReplicatedAttributeList: (objectclass=*) $ EXCLUDE memberof
idnssoaserial
entryusn krblastsuccessfulauth krblastfailedauth krbloginfailedcount
nsDS5ReplicaBindMethod: simple
nsDS5ReplicaCredentials:
{AES-TUhNR0NTcUdTSWIzRFFFRkRUQm1NRVVHQ1NxR1NJYjNEUUVG
RERBNEJDUmtOelUzTTJJNVlpMDBaV1EyTTJRMQ0KWXkwNU0yTm1aV05sTVMxbU5qRXpaak5oTlFBQ
0FRSUNBU0F3Q2dZSUtvWklodmNOQWdjd0hRWUpZSVpJQVdVRA0KQkFFcUJCQ2k0N0NxRGZFd2JIdm
I0MFVFZVI3MA==}gWI9NIB8lbt9tmNszzbBFCAe4Vs/e0sMyn5+NZPJg9E=
nsds7DirsyncCookie::
TVNEUwMAAABoJPGME7jQAQAAYAEAAPc1qQAAA
AD3NakAAMUjuImqVZhBkOkdt24C0IsBAA4AY4GwFkVcvEmMMExrVon4d6
13PwAADGzFNzznrESIxHzA74fbsz4HUgAAOnFoO5OE2E27lR/g4EcjQTLbIwAAuEm
PWjYok0qGS0HM/+TDmK7FgAMA6PTFXvAdnkaJSIkZT1lS+/FcJAAA4qTQaC46/Ua4KXgP
/ixNcbK4dgAAWowbgYD1akibZ+sCul5C4VNsMQAAxSO4iapVmEGQ6R23bgLQi/c1qQAAA
AAAogC6jFcyFUmhBp4B7FkaBcRHwwEAyhKMxsP0uUKGEnG2lsyA8eTUwgYA4n8Xx1bAlU
mBUl3zhlZ9WBngDAAA71vM2ebFEkCJkBaLjB4CGU+4CQMAGfO+4ndZCkaVKnwZNlNsf90
NDAAAgD6n+M2bcUGkOwo5gPLx7IOjAwAA
nsds50ruv: {replicageneration} 553fe9bb0004
nsds50ruv: {replica 4 ldap://dc1.ipadomain.net:389} 553fe9c9
0004 557f49fb00020004
nsds50ruv: {replica 3 ldap://dc2.ipadomain.net:389} 553fe9c
40003 557f3e4a00020003
nsruvReplicaLastModified: {replica 4 ldap://dc1.ipadomain.ne
t:389} 557f494a
nsruvReplicaLastModified: {replica 3 ldap://dc2.ipadomain.n
et:389} 557f3d95
oneWaySync: fromWindows
nsds5ReplicaEnabled: on
nsds5replicareapactive: 0
nsds5replicaLastUpdateStart: 0
nsds5replicaLastUpdateEnd: 0
nsds5replicaChangesSentSinceStartup:
nsds5replicaLastUpdateStatus: -1 - LDAP error: Can't contact LDAP server
nsds5replicaUpdateInProgress: FALSE
nsds5replicaLastInitStart: 0
nsds5replicaLastInitEnd: 0
--- output ---
Here are the error logs for the last month for the directory server. They
are totally empty until July 2.
---output---
389-Directory/1.3.3.8 B2015.040.128
dc1.ipadomain.net:636 (/etc/dirsrv/slapd-IPADOMAIN-NET)
[02/Jul/2015:03:19:02 +] NSMMReplicationPlugin - windows sync - failed
to send dirsync search request: 2
[02/Jul/2015:06:10:29 +] - Entry
uid=jenkinsdev,cn=users,cn=accounts,dc=ipadomain,dc=net missing
attribute sn required by object class person
[03/Jul/2015:02:04:02 +] NSMMReplicationPlugin - windows sync - failed
to send dirsync search request: 2
[03/Jul/2015:05:39:01 +] NSMMReplicationPlugin - windows sync - failed
to send dirsync search request: 2
[03/Jul/2015:17:09:00 +] NSMMReplicationPlugin - windows sync - failed
to send dirsync search request: 2
[03/Jul/2015:22:41:32 +] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (Cannot contact any KDC
for realm
