Hello,
Thanks for your instructions, step to step seems it is improving. Unfoutunately
it is not solve yet, now I have this when I do an upgrade following your
instructions:
# ipa-server-upgrade
Upgrading IPA:. Estimated time: 1 minute 30 seconds
[1/8]: saving configuration
[2/8]: disabling listeners
[3/8]: enabling DS global lock
[4/8]: starting directory server
[5/8]: updating schema
[6/8]: upgrading server
[7/8]: stopping directory server
[8/8]: restoring configuration
Done.
Update complete
Upgrading IPA services
Upgrading the configuration of the IPA services
IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command
ipa-server-upgrade manually.
Timeout exceeded
The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for more
information
On ipaupgrade.log I have:
....
2018-05-04T06:54:01Z DEBUG wait_for_open_ports: localhost [389] timeout 300
2018-05-04T06:54:01Z DEBUG waiting for port: 389
2018-05-04T06:54:01Z DEBUG Failed to connect to port 389 tcp on 127.0.0.1
2018-05-04T06:59:01Z ERROR IPA server upgrade failed: Inspect
/var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
2018-05-04T06:59:01Z DEBUG File
"/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 172, in execute
return_value = self.run()
File
"/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrade.py",
line 46, in run
server.upgrade()
File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py",
line 1919, in upgrade
upgrade_configuration()
File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py",
line 1586, in upgrade_configuration
ds.start(ds_serverid)
File "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", line
641, in start
super(DsInstance, self).start(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line
401, in start
self.service.start(instance_name, capture_output=capture_output, wait=wait)
File "/usr/lib/python2.7/site-packages/ipaplatform/redhat/services.py", line
161, in start
instance_name, capture_output=capture_output, wait=wait)
File "/usr/lib/python2.7/site-packages/ipaplatform/base/services.py", line
300, in start
self.wait_for_open_ports(self.service_instance(instance_name))
File "/usr/lib/python2.7/site-packages/ipaplatform/base/services.py", line
270, in wait_for_open_ports
self.api.env.startup_timeout)
File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 1256, in
wait_for_open_ports
raise socket.timeout("Timeout exceeded")
2018-05-04T06:59:01Z DEBUG The ipa-server-upgrade command failed, exception:
timeout: Timeout exceeded
2018-05-04T06:59:01Z ERROR Timeout exceeded
2018-05-04T06:59:01Z ERROR The ipa-server-upgrade command failed. See
/var/log/ipaupgrade.log for more information
And if I get the ports listening by ns-slapd, only upd port is listening:
# netstat -putan | grep 389
udp6 0 0 :::389 :::*
17469/ns-slapd
Thanks & Regards.
______________________________
-----Original Message-----
From: Rob Crittenden <[email protected]>
Sent: Thursday, May 03, 2018 21:16
To: thierry bordaz <[email protected]>; SOLER SANGUESA Miguel
<[email protected]>; FreeIPA users list <[email protected]>
Cc: Ludwig Krispenz <[email protected]>
Subject: Re: [Freeipa-users] Re: Problem on dirsrv when updating from 4.5.0
(RHEL 7.4) to 4.5.4 (RHEL 7.5)
thierry bordaz wrote:
> Hi Soler,
>
> Thanks for the information.
> So indexing is hanging because SC cache_init is running, the SC
> cache_init is hanging because SSSD is not started, SSSD is not started
> possibly because indexing prevents to get read access to the backend
> ("Backend is offline" TBC).
>
> An option would be to disable SC during upgrade phase ? Rob ?
Sure, disabling SC would be a way to get things running again.
To do this:
1. Be sure dirsrv is not running
2. edit /etc/dirsrv/slapd-YOUR-REALM/dse.ldif
3. find the entry for cn=Schema Compatibility,cn=plugins,cn=config
4. change nsslapd-pluginEnabled from on to off 5. try the upgrade again
Assuming the upgrade is successful you can re-enable it post-install with:
# ipa-compat-manage enable
rob
>
> regards
> thierry
>
> On 05/03/2018 11:35 AM, SOLER SANGUESA Miguel wrote:
>> Hello Thierry,
>>
>> The version is: slapi-nis-0.56.0-8.el7.x86_64
>>
>> And the errors are:
>> [02/May/2018:13:04:30.089731032 +0200] - ERR - schema-compat-plugin -
>> group "[email protected]" does not exist because SSSD is offline.
>> [02/May/2018:13:04:30.093169411 +0200] - ERR - schema-compat-plugin -
>> waiting for SSSD to become online...
>>
>> SSSD service is up, but as the ipa services are not (all) running,
>> the ipa domain is down, logs when started sssd shows:
>> sssd[be[ipa.example.org]][823]: Backend is offline
>>
>> and logs sssd_nss.log & sssd_sudo.log have hundreds of:
>> [sss_dp_get_reply] (0x0010): The Data Provider returned an error
>> [org.freedesktop.sssd.Error.DataProvider.Offline]
>>
>> Thanks & Regards.
>>
>> -----Original Message-----
>> From: thierry bordaz <[email protected]>
>> Sent: Thursday, May 03, 2018 11:25
>> To: SOLER SANGUESA Miguel <[email protected]>; Rob Crittenden
>> <[email protected]>; FreeIPA users list
>> <[email protected]>
>> Cc: Ludwig Krispenz <[email protected]>
>> Subject: Re: [Freeipa-users] Re: Problem on dirsrv when updating from
>> 4.5.0 (RHEL 7.4) to 4.5.4 (RHEL 7.5)
>>
>> me again ...
>>
>> If it exists some logs (/var/log/dirsrv/slapd-<xx>/errors) during the
>> hanging period it could also indicate the reason what Schema compat
>> was waiting for (SSSD ?)
>>
>> On 05/03/2018 10:38 AM, SOLER SANGUESA Miguel wrote:
>>> hello,
>>>
>>> Yesterday my ssh console closed the connection, so I had to start
>>> again the "ipa-server-upgrade", but the result is more or less the same:
>>> # ipa-server-upgrade
>>> Upgrading IPA:. Estimated time: 1 minute 30 seconds
>>> [1/10]: stopping directory server
>>> [2/10]: saving configuration
>>> [3/10]: disabling listeners
>>> [4/10]: enabling DS global lock
>>> [5/10]: starting directory server
>>> [6/10]: updating schema
>>> [7/10]: upgrading server
>>>
>>> But now, the lines that are repeated on the access log are:
>>> [03/May/2018:10:33:27.969889221 +0200] conn=6 op=79094 SRCH
>>> base="cn=indextask_l_137445500911864330_4055,cn=index,cn=tasks,cn=config"
>>> scope=0 filter="(objectClass=*)" attrs="nstaskstatus nstaskexitcode"
>>> [03/May/2018:10:33:27.970146545 +0200] conn=6 op=79094 RESULT err=0
>>> tag=101 nentries=1 etime=1.0000687740
>>>
>>> And it is on the same state than the other:
>>> # ldapsearch -H ldapi://%2fvar%2frun%2fslapd-IPA-EXAMPLE-ORG.socket
>>> -b
>>> cn=indextask_l_137445500911864330_4055,cn=index,cn=tasks,cn=config -s base
>>> SASL/EXTERNAL authentication started SASL username:
>>> gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
>>> SASL SSF: 0
>>> # extended LDIF
>>> #
>>> # LDAPv3
>>> # base
>>> <cn=indextask_l_137445500911864330_4055,cn=index,cn=tasks,cn=config>
>>> with scope baseObject # filter: (objectclass=*) # requesting: ALL #
>>>
>>> # indextask_l_137445500911864330_4055, index, tasks, config
>>> dn:
>>> cn=indextask_l_137445500911864330_4055,cn=index,cn=tasks,cn=config
>>> objectClass: top
>>> objectClass: extensibleObject
>>> nsindexattribute: l
>>> nsinstance: userRoot
>>> cn: indextask_l_137445500911864330_4055
>>> nstaskcurrentitem: 0
>>> nstasktotalitems: 1
>>>
>>> # search result
>>> search: 3
>>> result: 0 Success
>>>
>>> # numResponses: 2
>>> # numEntries: 1
>>>
>>> The pstack of the proces: /usr/bin/python2
>>> /usr/sbin/ipa-server-upgrade # pstack 9805
>>> #0 0x00007fb906629be3 in __select_nocancel () from /lib64/libc.so.6
>>> #1 0x00007fb8ff5e0000 in time_sleep () from
>>> /usr/lib64/python2.7/lib-dynload/timemodule.so
>>> #2 0x00007fb90730acf0 in PyEval_EvalFrameEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #3 0x00007fb90730a6bd in PyEval_EvalFrameEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #4 0x00007fb90730a6bd in PyEval_EvalFrameEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #5 0x00007fb90730a6bd in PyEval_EvalFrameEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #6 0x00007fb90730d03d in PyEval_EvalCodeEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #7 0x00007fb90730a53c in PyEval_EvalFrameEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #8 0x00007fb90730a6bd in PyEval_EvalFrameEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #9 0x00007fb90730d03d in PyEval_EvalCodeEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #10 0x00007fb90730a53c in PyEval_EvalFrameEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #11 0x00007fb90730d03d in PyEval_EvalCodeEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #12 0x00007fb90730a53c in PyEval_EvalFrameEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #13 0x00007fb90730a6bd in PyEval_EvalFrameEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #14 0x00007fb90730d03d in PyEval_EvalCodeEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #15 0x00007fb90730a53c in PyEval_EvalFrameEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #16 0x00007fb90730a6bd in PyEval_EvalFrameEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #17 0x00007fb90730a6bd in PyEval_EvalFrameEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #18 0x00007fb90730a6bd in PyEval_EvalFrameEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #19 0x00007fb90730a6bd in PyEval_EvalFrameEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #20 0x00007fb90730d03d in PyEval_EvalCodeEx () from
>>> /lib64/libpython2.7.so.1.0
>>> #21 0x00007fb90730d142 in PyEval_EvalCode () from
>>> /lib64/libpython2.7.so.1.0
>>> #22 0x00007fb90732657f in run_mod () from /lib64/libpython2.7.so.1.0
>>> #23 0x00007fb90732773e in PyRun_FileExFlags () from
>>> /lib64/libpython2.7.so.1.0
>>> #24 0x00007fb9073289c9 in PyRun_SimpleFileExFlags () from
>>> /lib64/libpython2.7.so.1.0
>>> #25 0x00007fb907339b7f in Py_Main () from /lib64/libpython2.7.so.1.0
>>> #26 0x00007fb9065563d5 in __libc_start_main () from /lib64/libc.so.6
>>> #27 0x000000000040066e in _start ()
>>>
>>>
>>> And the pstack from the process: /usr/sbin/ns-slapd -D
>>> /etc/dirsrv/slapd-IPA-EXAMPLE-ORG -i
>>> /var/run/dirsrv/slapd-IPA-EXAMPLE-ORG.pid
>>> # pstack 15536
>>> Thread 50 (Thread 0x7f5bc0ae9700 (LWP 15539)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d33e63 in worker_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 49
>>> (Thread 0x7f5bc02e8700 (LWP 15540)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d33e63 in worker_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 48
>>> (Thread 0x7f5bbfae7700 (LWP 15541)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d33e63 in worker_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 47
>>> (Thread 0x7f5bbf2e6700 (LWP 15542)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d33e63 in worker_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 46
>>> (Thread 0x7f5bbeae5700 (LWP 15543)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d33e63 in worker_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 45
>>> (Thread 0x7f5bbe2e4700 (LWP 15544)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d33e63 in worker_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 44
>>> (Thread 0x7f5bbdae3700 (LWP 15545)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d33e63 in worker_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 43
>>> (Thread 0x7f5bbd2e2700 (LWP 15546)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d33e63 in worker_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 42
>>> (Thread 0x7f5bbcae1700 (LWP 15547)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d33e63 in worker_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 41
>>> (Thread 0x7f5bbc2e0700 (LWP 15548)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d33e63 in worker_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 40
>>> (Thread 0x7f5bbbadf700 (LWP 15549)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d33e63 in worker_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 39
>>> (Thread 0x7f5bbb2de700 (LWP 15550)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d33e63 in worker_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 38
>>> (Thread 0x7f5bbaadd700 (LWP 15551)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d33e63 in worker_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 37
>>> (Thread 0x7f5bba2dc700 (LWP 15552)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d33e63 in worker_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 36
>>> (Thread 0x7f5bb9adb700 (LWP 15553)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d33e63 in worker_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 35
>>> (Thread 0x7f5bb92da700 (LWP 15554)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d33e63 in worker_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 34
>>> (Thread 0x7f5bb8ad9700 (LWP 15555)):
>>> #0 0x00007f5bd18a6113 in epoll_wait () at /lib64/libc.so.6
>>> #1 0x00007f5bd1d9c803 in epoll_dispatch () at
>>> /lib64/libevent-2.0.so.5
>>> #2 0x00007f5bd1d883ea in event_base_loop () at
>>> /lib64/libevent-2.0.so.5
>>> #3 0x00007f5bd4d34eae in ns_event_fw_loop () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #4 0x00007f5bd4d33ac9 in event_loop_thread_func () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 33
>>> (Thread 0x7f5bb2b84700 (LWP 15559)):
>>> #0 0x00007f5bd189cc03 in select () at /lib64/libc.so.6
>>> #1 0x00007f5bd4acc599 in DS_Sleep () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #2 0x00007f5bc3e41e77 in deadlock_threadmain () at
>>> /usr/lib64/dirsrv/plugins/libback-ldbm.so
>>> #3 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #4 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #5 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 32
>>> (Thread 0x7f5bb2383700 (LWP 15560)):
>>> #0 0x00007f5bd189cc03 in select () at /lib64/libc.so.6
>>> #1 0x00007f5bd4acc599 in DS_Sleep () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #2 0x00007f5bc3e45a87 in checkpoint_threadmain () at
>>> /usr/lib64/dirsrv/plugins/libback-ldbm.so
>>> #3 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #4 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #5 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 31
>>> (Thread 0x7f5bb1b82700 (LWP 15561)):
>>> #0 0x00007f5bd189cc03 in select () at /lib64/libc.so.6
>>> #1 0x00007f5bd4acc599 in DS_Sleep () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #2 0x00007f5bc3e420c7 in trickle_threadmain () at
>>> /usr/lib64/dirsrv/plugins/libback-ldbm.so
>>> #3 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #4 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #5 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 30
>>> (Thread 0x7f5bb1381700 (LWP 15562)):
>>> #0 0x00007f5bd189cc03 in select () at /lib64/libc.so.6
>>> #1 0x00007f5bd4acc599 in DS_Sleep () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #2 0x00007f5bc3e3ce47 in perf_threadmain () at
>>> /usr/lib64/dirsrv/plugins/libback-ldbm.so
>>> #3 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #4 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #5 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 29
>>> (Thread 0x7f5bb0b80700 (LWP 15563)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x00007f5bd4ab8eb8 in slapi_wait_condvar () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #3 0x00007f5bc920213e in cos_cache_wait_on_change () at
>>> /usr/lib64/dirsrv/plugins/libcos-plugin.so
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 28
>>> (Thread 0x7f5bb037f700 (LWP 15564)):
>>> #0 0x00007f5bd189ae9d in poll () at /lib64/libc.so.6
>>> #1 0x00007f5bc6d7e44c in ipa_cldap_worker () at
>>> /usr/lib64/dirsrv/plugins/libipa_cldap.so
>>> #2 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #3 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 27
>>> (Thread 0x7f5bd5132700 (LWP 15566)):
>>> #0 0x00007f5bd21fccf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2852d27 in pt_TimedWait () at /lib64/libnspr4.so
>>> #2 0x00007f5bd285320e in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #3 0x00007f5bc3572033 in _cl5TrimMain () at
>>> /usr/lib64/dirsrv/plugins/libreplication-plugin.so
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 26
>>> (Thread 0x7f5bafb7e700 (LWP 15567)):
>>> #0 0x00007f5bd21fccf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2852d27 in pt_TimedWait () at /lib64/libnspr4.so
>>> #2 0x00007f5bd285320e in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #3 0x00007f5bc3587fa4 in protocol_sleep () at
>>> /usr/lib64/dirsrv/plugins/libreplication-plugin.so
>>> #4 0x00007f5bc358b75a in repl5_inc_run () at
>>> /usr/lib64/dirsrv/plugins/libreplication-plugin.so
>>> #5 0x00007f5bc358fbec in prot_thread_main () at
>>> /usr/lib64/dirsrv/plugins/libreplication-plugin.so
>>> #6 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #7 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #8 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 25
>>> (Thread 0x7f5baf37d700 (LWP 15568)):
>>> #0 0x00007f5bd21fccf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2852d27 in pt_TimedWait () at /lib64/libnspr4.so
>>> #2 0x00007f5bd285320e in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #3 0x00007f5bc3587fa4 in protocol_sleep () at
>>> /usr/lib64/dirsrv/plugins/libreplication-plugin.so
>>> #4 0x00007f5bc358b75a in repl5_inc_run () at
>>> /usr/lib64/dirsrv/plugins/libreplication-plugin.so
>>> #5 0x00007f5bc358fbec in prot_thread_main () at
>>> /usr/lib64/dirsrv/plugins/libreplication-plugin.so
>>> #6 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #7 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #8 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 24
>>> (Thread 0x7f5bae759700 (LWP 15569)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x00007f5bd4ab8eb8 in slapi_wait_condvar () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #3 0x00007f5bc22d11ad in roles_cache_wait_on_change () at
>>> /usr/lib64/dirsrv/plugins/libroles-plugin.so
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 23
>>> (Thread 0x7f5badf58700 (LWP 15570)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x00007f5bd4ab8eb8 in slapi_wait_condvar () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #3 0x00007f5bc22d11ad in roles_cache_wait_on_change () at
>>> /usr/lib64/dirsrv/plugins/libroles-plugin.so
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 22
>>> (Thread 0x7f5bad757700 (LWP 15571)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x00007f5bd4ab8eb8 in slapi_wait_condvar () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #3 0x00007f5bc22d11ad in roles_cache_wait_on_change () at
>>> /usr/lib64/dirsrv/plugins/libroles-plugin.so
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 21
>>> (Thread 0x7f5bacd4f700 (LWP 15572)):
>>> #0 0x00007f5bd21fccf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2852d27 in pt_TimedWait () at /lib64/libnspr4.so
>>> #2 0x00007f5bd285320e in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #3 0x0000564591e10b08 in housecleaning ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 20
>>> (Thread 0x7f5bac54e700 (LWP 15573)):
>>> #0 0x00007f5bd21fccf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2852d27 in pt_TimedWait () at /lib64/libnspr4.so
>>> #2 0x00007f5bd285320e in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #3 0x00007f5bd4a5ba63 in eq_loop () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 19
>>> (Thread 0x7f5babd4d700 (LWP 15574)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x0000564591e05fce in [IDLE THREAD] connection_wait_for_new_work
>>> ()
>>> #3 0x0000564591e075d7 in connection_threadmain ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 18
>>> (Thread 0x7f5bab54c700 (LWP 15575)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x0000564591e05fce in [IDLE THREAD] connection_wait_for_new_work
>>> ()
>>> #3 0x0000564591e075d7 in connection_threadmain ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 17
>>> (Thread 0x7f5baad4b700 (LWP 15576)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x0000564591e05fce in [IDLE THREAD] connection_wait_for_new_work
>>> ()
>>> #3 0x0000564591e075d7 in connection_threadmain ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 16
>>> (Thread 0x7f5baa54a700 (LWP 15577)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x0000564591e05fce in [IDLE THREAD] connection_wait_for_new_work
>>> ()
>>> #3 0x0000564591e075d7 in connection_threadmain ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 15
>>> (Thread 0x7f5ba9d49700 (LWP 15578)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x0000564591e05fce in [IDLE THREAD] connection_wait_for_new_work
>>> ()
>>> #3 0x0000564591e075d7 in connection_threadmain ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 14
>>> (Thread 0x7f5ba9548700 (LWP 15579)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x0000564591e05fce in [IDLE THREAD] connection_wait_for_new_work
>>> ()
>>> #3 0x0000564591e075d7 in connection_threadmain ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 13
>>> (Thread 0x7f5ba8d47700 (LWP 15580)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x0000564591e05fce in [IDLE THREAD] connection_wait_for_new_work
>>> ()
>>> #3 0x0000564591e075d7 in connection_threadmain ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 12
>>> (Thread 0x7f5ba8546700 (LWP 15581)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x0000564591e05fce in [IDLE THREAD] connection_wait_for_new_work
>>> ()
>>> #3 0x0000564591e075d7 in connection_threadmain ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 11
>>> (Thread 0x7f5ba7d45700 (LWP 15582)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x0000564591e05fce in [IDLE THREAD] connection_wait_for_new_work
>>> ()
>>> #3 0x0000564591e075d7 in connection_threadmain ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 10
>>> (Thread 0x7f5ba7544700 (LWP 15583)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x0000564591e05fce in [IDLE THREAD] connection_wait_for_new_work
>>> ()
>>> #3 0x0000564591e075d7 in connection_threadmain ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 9
>>> (Thread 0x7f5ba6d43700 (LWP 15584)):
>>> #0 0x00007f5bd189ae9d in poll () at /lib64/libc.so.6
>>> #1 0x00007f5bd2854bc7 in _pr_poll_with_poll () at
>>> /lib64/libnspr4.so
>>> #2 0x0000564591e07049 in connection_read_operation ()
>>> #3 0x0000564591e0765d in connection_threadmain ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 8
>>> (Thread 0x7f5ba6542700 (LWP 15585)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x0000564591e05fce in [IDLE THREAD] connection_wait_for_new_work
>>> ()
>>> #3 0x0000564591e075d7 in connection_threadmain ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 7
>>> (Thread 0x7f5ba5d41700 (LWP 15586)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x0000564591e05fce in [IDLE THREAD] connection_wait_for_new_work
>>> ()
>>> #3 0x0000564591e075d7 in connection_threadmain ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 6
>>> (Thread 0x7f5ba5540700 (LWP 15587)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x0000564591e05fce in [IDLE THREAD] connection_wait_for_new_work
>>> ()
>>> #3 0x0000564591e075d7 in connection_threadmain ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 5
>>> (Thread 0x7f5ba4d3f700 (LWP 15588)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x0000564591e05fce in [IDLE THREAD] connection_wait_for_new_work
>>> ()
>>> #3 0x0000564591e075d7 in connection_threadmain ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 4
>>> (Thread 0x7f5ba453e700 (LWP 15589)):
>>> #0 0x00007f5bd21fc945 in pthread_cond_wait@@GLIBC_2.3.2 () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd2853290 in PR_WaitCondVar () at /lib64/libnspr4.so
>>> #2 0x0000564591e05fce in [IDLE THREAD] connection_wait_for_new_work
>>> ()
>>> #3 0x0000564591e075d7 in connection_threadmain ()
>>> #4 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #5 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #6 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 3
>>> (Thread 0x7f5ba3900700 (LWP 15590)):
>>> #0 0x00007f5bd189cc03 in select () at /lib64/libc.so.6
>>> #1 0x00007f5bd4acc599 in DS_Sleep () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #2 0x00007f5bc1ea5c6e in backend_set_entry_from () at
>>> /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
>>> #3 0x00007f5bc1ea8906 in backend_shr_set_entry_cb () at
>>> /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
>>> #4 0x00007f5bd4aac614 in send_ldap_search_entry_ext () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #5 0x00007f5bd4aace0c in send_ldap_search_entry () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #6 0x00007f5bd4a87ca4 in send_entry.isra.0 () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #7 0x00007f5bd4a88208 in send_results_ext.constprop.4 () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #8 0x00007f5bd4a8a0d1 in op_shared_search () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #9 0x00007f5bd4a9b507 in search_internal_callback_pb () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #10 0x00007f5bc1ead018 in backend_shr_set_config_entry_add () at
>>> /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
>>> #11 0x00007f5bc1ea7a2d in backend_set_config_entry_add_cb () at
>>> /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
>>> #12 0x00007f5bc1ea8ac1 in backend_shr_data_initialize_thread_cb ()
>>> at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
>>> #13 0x00007f5bc1eba597 in wrap_pthread_starter () at
>>> /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
>>> #14 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #15 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #16 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 2
>>> (Thread 0x7f5ba30ff700 (LWP 15606)):
>>> #0 0x00007f5bd21fc2ae in pthread_rwlock_wrlock () at
>>> /lib64/libpthread.so.0
>>> #1 0x00007f5bd4a7f0e5 in slapi_mtn_be_set_readonly () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #2 0x00007f5bc3e921ac in instance_set_busy_and_readonly () at
>>> /usr/lib64/dirsrv/plugins/libback-ldbm.so
>>> #3 0x00007f5bc3e8d5d1 in ldbm_back_ldbm2index () at
>>> /usr/lib64/dirsrv/plugins/libback-ldbm.so
>>> #4 0x00007f5bd4ac5942 in task_index_thread () at
>>> /usr/lib64/dirsrv/libslapd.so.0
>>> #5 0x00007f5bd2858c8b in _pt_root () at /lib64/libnspr4.so
>>> #6 0x00007f5bd21f8dd5 in start_thread () at /lib64/libpthread.so.0
>>> #7 0x00007f5bd18a5b3d in clone () at /lib64/libc.so.6 Thread 1
>>> (Thread 0x7f5bd5135940 (LWP 15536)):
>>> #0 0x00007f5bd21f9f47 in pthread_join () at /lib64/libpthread.so.0
>>> #1 0x00007f5bd4d34b2f in ns_thrpool_wait () at
>>> /usr/lib64/dirsrv/libnunc-stans.so.0
>>> #2 0x0000564591e0d862 in slapd_daemon ()
>>> #3 0x0000564591dfe86f in main ()
>>>
>>> Thanks & Regards.
>>> ______________________________
>>>
>>> -----Original Message-----
>>> From: thierry bordaz <[email protected]>
>>> Sent: Thursday, May 03, 2018 9:31
>>> To: Rob Crittenden <[email protected]>; FreeIPA users list
>>> <[email protected]>; SOLER SANGUESA Miguel
>>> <[email protected]>
>>> Cc: Ludwig Krispenz <[email protected]>
>>> Subject: Re: [Freeipa-users] Re: Problem on dirsrv when updating
>>> from
>>> 4.5.0 (RHEL 7.4) to 4.5.4 (RHEL 7.5)
>>>
>>> Hi,
>>>
>>> During indexing task we should see in the task status the periodic
>>> progression of the indexing.
>>> May be the indexing is hanging somewhere. When the problem occurs
>>> could you provide a pstack of the dirsrv server ?
>>>
>>> best regards
>>> thierry
>>>
>>> On 05/02/2018 10:27 PM, Rob Crittenden wrote:
>>>> SOLER SANGUESA Miguel via FreeIPA-users wrote:
>>>>> Hello,
>>>>>
>>>>> This is the output of the command (seems that is not complete):
>>>>>
>>>>> # ldapsearch -H
>>>>> ldapi://%2fvar%2frun%2fslapd-IPA-EXAMPLE-ORG.socket
>>>>> -b
>>>>> cn=indextask_description_137444551994158920_5958,cn=index,cn=tasks
>>>>> ,c
>>>>> n
>>>>> =config
>>>>> -s base
>>>>> SASL/EXTERNAL authentication started SASL username:
>>>>> gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
>>>>> SASL SSF: 0
>>>>> # extended LDIF
>>>>> #
>>>>> # LDAPv3
>>>>> # base
>>>>> <cn=indextask_description_137444551994158920_5958,cn=index,cn=task
>>>>> s,
>>>>> c
>>>>> n=config>
>>>>> with scope baseObject
>>>>> # filter: (objectclass=*)
>>>>> # requesting: ALL
>>>>> #
>>>>>
>>>>> # indextask_description_137444551994158920_5958, index, tasks,
>>>>> config
>>>>> dn:
>>>>> cn=indextask_description_137444551994158920_5958,cn=index,cn=tasks
>>>>> ,c
>>>>> n
>>>>> =conf
>>>>> ig
>>>>> objectClass: top
>>>>> objectClass: extensibleObject
>>>>> nsindexattribute: description
>>>>> nsinstance: userRoot
>>>>> cn: indextask_description_137444551994158920_5958
>>>>> nstaskcurrentitem: 0
>>>>> nstasktotalitems: 1
>>>> IPA is looking for nstaskstatus and since that isn't there it is
>>>> looping forever.
>>>>
>>>> I've cc'd a couple of the 389-ds devs to see what they think about it.
>>>>
>>>> rob
>>>>
>>>>> # search result
>>>>> search: 3
>>>>> result: 0 Success
>>>>>
>>>>> # numResponses: 2
>>>>> # numEntries: 1
>>>>>
>>>>> Thanks & Regards.
>>>>> ______________________________
>>>>> -----Original Message-----
>>>>> From: Rob Crittenden <[email protected]>
>>>>> Sent: Tuesday, May 01, 2018 15:18
>>>>> To: FreeIPA users list <[email protected]>
>>>>> Cc: SOLER SANGUESA Miguel <[email protected]>
>>>>> Subject: Re: [Freeipa-users] Re: Problem on dirsrv when updating
>>>>> from
>>>>> 4.5.0 (RHEL 7.4) to 4.5.4 (RHEL 7.5)
>>>>>
>>>>> SOLER SANGUESA Miguel via FreeIPA-users wrote:
>>>>>> Hello,
>>>>>>
>>>>>> Thank you for your answer, now dirsrv can start, but after
>>>>>> running the "
>>>>>> [email protected] " it hungs:
>>>>>>
>>>>>> # ipa-server-upgrade
>>>>>>
>>>>>> Upgrading IPA:. Estimated time: 1 minute 30 seconds
>>>>>>
>>>>>> [1/8]: saving configuration
>>>>>>
>>>>>> [2/8]: disabling listeners
>>>>>>
>>>>>> [3/8]: enabling DS global lock
>>>>>>
>>>>>> [4/8]: starting directory server
>>>>>>
>>>>>> [5/8]: updating schema
>>>>>>
>>>>>> [6/8]: upgrading server
>>>>>>
>>>>>> The latest messages on the log is (there are no logs on the lasts
>>>>>> 2h):
>>>>>>
>>>>>> 2018-05-01T08:13:14Z DEBUG
>>>>>> ---------------------------------------------
>>>>>>
>>>>>> 2018-05-01T08:13:14Z DEBUG Final value after applying updates
>>>>>>
>>>>>> 2018-05-01T08:13:14Z DEBUG dn:
>>>>>> cn=description,cn=index,cn=userroot,cn=ldbm
>>>>>> database,cn=plugins,cn=config
>>>>>>
>>>>>> 2018-05-01T08:13:14Z DEBUG objectclass:
>>>>>>
>>>>>> 2018-05-01T08:13:14Z DEBUG top
>>>>>>
>>>>>> 2018-05-01T08:13:14Z DEBUG nsindex
>>>>>>
>>>>>> 2018-05-01T08:13:14Z DEBUG nsindextype:
>>>>>>
>>>>>> 2018-05-01T08:13:14Z DEBUG eq
>>>>>>
>>>>>> 2018-05-01T08:13:14Z DEBUG sub
>>>>>>
>>>>>> 2018-05-01T08:13:14Z DEBUG cn:
>>>>>>
>>>>>> 2018-05-01T08:13:14Z DEBUG description
>>>>>>
>>>>>> 2018-05-01T08:13:14Z DEBUG nssystemindex:
>>>>>>
>>>>>> 2018-05-01T08:13:14Z DEBUG false
>>>>>>
>>>>>> 2018-05-01T08:13:19Z DEBUG Creating task to index attribute:
>>>>>> description
>>>>>>
>>>>>> 2018-05-01T08:13:19Z DEBUG Task id:
>>>>>> cn=indextask_description_137444551994158920_5958,cn=index,cn=task
>>>>>> s,
>>>>>> c
>>>>>> n=
>>>>>> config
>>>>>>
>>>>>> If I check the ipa services, dirsrv is the only one running:
>>>>>>
>>>>>> # ipactl status
>>>>>>
>>>>>> Directory Service: RUNNING
>>>>>>
>>>>>> krb5kdc Service: STOPPED
>>>>>>
>>>>>> kadmin Service: STOPPED
>>>>>>
>>>>>> named Service: STOPPED
>>>>>>
>>>>>> httpd Service: STOPPED
>>>>>>
>>>>>> ipa-custodia Service: STOPPED
>>>>>>
>>>>>> ntpd Service: RUNNING
>>>>>>
>>>>>> pki-tomcatd Service: STOPPED
>>>>>>
>>>>>> smb Service: STOPPED
>>>>>>
>>>>>> winbind Service: STOPPED
>>>>>>
>>>>>> ipa-otpd Service: STOPPED
>>>>>>
>>>>>> ipa-dnskeysyncd Service: STOPPED
>>>>>>
>>>>>> ipa: INFO: The ipactl command was successful
>>>>>>
>>>>>> Should I stop the ipa-server-upgrade and start ipa services?
>>>>> It depends on how many entries you have. This is an indexing task
>>>>> and while 2 hours seems a bit excessive, it depends on how much
>>>>> work it has to do.
>>>>>
>>>>> The 389-ds access and/or error logs may provide details.
>>>>>
>>>>> rob
>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Alexander Bokovoy <[email protected]>
>>>>>> Sent: Tuesday, May 01, 2018 9:56
>>>>>> To: FreeIPA users list <[email protected]>
>>>>>> Cc: SOLER SANGUESA Miguel <[email protected]>
>>>>>> Subject: Re: [Freeipa-users] Problem on dirsrv when updating from
>>>>>> 4.5.0 (RHEL 7.4) to 4.5.4 (RHEL 7.5)
>>>>>>
>>>>>> On ti, 01 touko 2018, SOLER SANGUESA Miguel via FreeIPA-users wrote:
>>>>>>
>>>>>> >hello,
>>>>>>
>>>>>> >
>>>>>>
>>>>>> >I have an IPA master that updated from 4.5.0 (RHEL 7.4) to
>>>>>> 4.5.4 (RHEL 7.5). An hour later I tried to do the same with the
>>>>>> unique replica I have, but after update dirsrv is not starting.
>>>>>>
>>>>>> >It says it is needed run "ipa-server-upgrade", but it also
>>>>>> fails:
>>>>>>
>>>>>> ># ipactl start
>>>>>>
>>>>>> >Upgrade required: please run ipa-server-upgrade command
>>>>>> Aborting ipactl
>>>>>>
>>>>>> >
>>>>>>
>>>>>> ># ipa-server-upgrade
>>>>>>
>>>>>> >Upgrading IPA:. Estimated time: 1 minute 30 seconds
>>>>>>
>>>>>> > [1/8]: saving configuration
>>>>>>
>>>>>> > [2/8]: disabling listeners
>>>>>>
>>>>>> > [3/8]: enabling DS global lock
>>>>>>
>>>>>> > [4/8]: starting directory server
>>>>>>
>>>>>> > [error] CalledProcessError: Command '/bin/systemctl start
>>>>>>
>>>>>> >[email protected]
>>>>>> <mailto:[email protected]>' returned non-zero exit
>>>>>> status
>>>>>> 1
>>>>>>
>>>>>> > [cleanup]: stopping directory server
>>>>>>
>>>>>> > [cleanup]: restoring configuration
>>>>>>
>>>>>> >IPA server upgrade failed: Inspect /var/log/ipaupgrade.log
>>>>>> and run command ipa-server-upgrade manually.
>>>>>>
>>>>>> >Unexpected error - see /var/log/ipaupgrade.log for details:
>>>>>>
>>>>>> >CalledProcessError: Command '/bin/systemctl start
>>>>>>
>>>>>> >[email protected]
>>>>>> <mailto:[email protected]>' returned non-zero exit
>>>>>> status
>>>>>> 1 The
>>>>>>
>>>>>> >ipa-server-upgrade command failed. See
>>>>>> /var/log/ipaupgrade.log for more
>>>>>>
>>>>>> >information
>>>>>>
>>>>>> >
>>>>>>
>>>>>> >On the log I can see:
>>>>>>
>>>>>> >2018-04-30T14:36:15Z DEBUG Starting external process
>>>>>>
>>>>>> >2018-04-30T14:36:15Z DEBUG args=/bin/systemctl is-active
>>>>>>
>>>>>> >[email protected]
>>>>>> <mailto:[email protected]>
>>>>>> 2018-04-30T14:36:15Z DEBUG Process
>>>>>>
>>>>>> >finished, return code=3 2018-04-30T14:36:15Z DEBUG
>>>>>> stdout=failed ...
>>>>>>
>>>>>> >2018-04-30T14:36:15Z DEBUG [4/8]: starting directory
>>>>>> server
>>>>>>
>>>>>> >2018-04-30T14:36:15Z DEBUG Starting external process
>>>>>>
>>>>>> >2018-04-30T14:36:15Z DEBUG args=/bin/systemctl start
>>>>>>
>>>>>> >[email protected]
>>>>>> <mailto:[email protected]>
>>>>>> 2018-04-30T14:36:15Z DEBUG Process
>>>>>>
>>>>>> >finished, return code=1 2018-04-30T14:36:15Z DEBUG stdout=
>>>>>>
>>>>>> >2018-04-30T14:36:15Z DEBUG stderr=Job for
>>>>>> [email protected]
>>>>>> <mailto:[email protected]>
>>>>>> failed because the control process exited with error code. See
>>>>>> "systemctl status [email protected]
>>>>>> <mailto:[email protected]>" and "journalctl -xe" for
>>>>>> details.
>>>>>>
>>>>>> >
>>>>>>
>>>>>> >2018-04-30T14:36:15Z DEBUG Traceback (most recent call last):
>>>>>>
>>>>>> > File
>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/service.py",
>>>>>> line 504, in start_creation
>>>>>>
>>>>>> > run_step(full_msg, method)
>>>>>>
>>>>>> > File
>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/service.py",
>>>>>> line 494, in run_step
>>>>>>
>>>>>> > method()
>>>>>>
>>>>>> > File
>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/upgradeinstance.
>>>>>> py
>>>>>> ",
>>>>>> line 95, in __start
>>>>>>
>>>>>> > srv.start(self.serverid, ldapi=True)
>>>>>>
>>>>>> > File
>>>>>> "/usr/lib/python2.7/site-packages/ipaplatform/redhat/services.py"
>>>>>> ,
>>>>>> line 161, in start
>>>>>>
>>>>>> > instance_name, capture_output=capture_output, wait=wait)
>>>>>>
>>>>>> > File
>>>>>> "/usr/lib/python2.7/site-packages/ipaplatform/base/services.py",
>>>>>> line 294, in start
>>>>>>
>>>>>> > skip_output=not capture_output)
>>>>>>
>>>>>> > File
>>>>>> "/usr/lib/python2.7/site-packages/ipapython/ipautil.py",
>>>>>> line 542, in run
>>>>>>
>>>>>> > raise CalledProcessError(p.returncode, arg_string,
>>>>>> str(output))
>>>>>>
>>>>>> >CalledProcessError: Command '/bin/systemctl start
>>>>>>
>>>>>> >[email protected]
>>>>>> <mailto:[email protected]>' returned non-zero exit
>>>>>> status
>>>>>> 1
>>>>>>
>>>>>> >
>>>>>>
>>>>>> >2018-04-30T14:36:15Z DEBUG [error] CalledProcessError:
>>>>>> Command '/bin/systemctl start [email protected]
>>>>>> <mailto:[email protected]>' returned non-zero exit
>>>>>> status
>>>>>> 1
>>>>>>
>>>>>> >
>>>>>>
>>>>>> >Checking /var/log/dirsrv/slapd-IPA-EXAMPLE-ORG/errors I show:
>>>>>>
>>>>>> >[30/Apr/2018:16:04:52.584220922 +0200] - ERR -
>>>>>> slapd_bootstrap_config
>>>>>> - The default password storage scheme could not be read or was
>>>>>> not found in the file /etc/dirsrv/slapd-IPA-EXAMPLE-ORG/dse.ldif.
>>>>>> It is mandatory.
>>>>>>
>>>>>> >
>>>>>>
>>>>>> >Checking on internet I show that "dse.ldif" could be
>>>>>> corrupted, so I changed with "dse.ldif.startOK" without any
>>>>>> change and then I changed with "dse.ldif.bak". The problem
>>>>>> persist but the error has
>>>>>> changed:
>>>>>>
>>>>>> >[30/Apr/2018:16:32:13.435210918 +0200] - NOTICE -
>>>>>> config_set_port
>>>>>> -
>>>>>>
>>>>>> >Non-Secure Port Disabled
>>>>>>
>>>>>> >[30/Apr/2018:16:32:13.556581301 +0200] - ERR -
>>>>>> symload_report_error
>>>>>> -
>>>>>>
>>>>>> >Netscape Portable Runtime error -5975:
>>>>>>
>>>>>> >/usr/lib64/dirsrv/plugins/libreplication-plugin.so:
>>>>>> undefined
>>>>>> symbol:
>>>>>>
>>>>>> >replication_legacy_plugin_init
>>>>>>
>>>>>> >[30/Apr/2018:16:32:13.561590553 +0200] - ERR -
>>>>>> symload_report_error
>>>>>> -
>>>>>>
>>>>>> >Could not load symbol "replication_legacy_plugin_init" from
>>>>>>
>>>>>> >"/usr/lib64/dirsrv/plugins/libreplication-plugin.so" for
>>>>>> plugin Legacy
>>>>>>
>>>>>> >Replication Plugin
>>>>>>
>>>>>> >[30/Apr/2018:16:32:13.564590264 +0200] - ERR -
>>>>>> load_plugin_entry
>>>>>> - Unable to load plugin "cn=Legacy Replication
>>>>>> Plugin,cn=plugins,cn=config"
>>>>>>
>>>>>> >
>>>>>>
>>>>>> >I saw a bug about this problem, but it is still opened:
>>>>>>
>>>>>>> https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=1529
>>>>>>> 44
>>>>>>> 2
>>>>>> >
>>>>>>
>>>>>> >Any idea how to fix the issue?
>>>>>>
>>>>>> >
>>>>>>
>>>>>> >If it is not possible to fix it, can I remove the replica
>>>>>> from IPA and install it again with the same name?
>>>>>>
>>>>>> A quick fix could be to remove an entry for cn=Legacy Replication
>>>>>> Plugin,cn=plugins,cn=config from
>>>>>> /etc/dirsrv/slapd-IPA-EXAMPLE-ORG/dse/ldif
>>>>>>
>>>>>> when dirsrv is down.
>>>>>>
>>>>>> --
>>>>>>
>>>>>> / Alexander Bokovoy
>>>>>>
>>>>>> Sr. Principal Software Engineer
>>>>>>
>>>>>> Security / Identity Management Engineering Red Hat Limited,
>>>>>> Finland
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> FreeIPA-users mailing list --
>>>>>> [email protected]
>>>>>> To unsubscribe send an email to
>>>>>> [email protected]
>>>>>>
>>>>> _______________________________________________
>>>>> FreeIPA-users mailing list -- [email protected]
>>>>> To unsubscribe send an email to
>>>>> [email protected]
>>>>>
>
_______________________________________________
FreeIPA-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
