On 03/09/2016 05:51 PM, Andrew E. Bruno wrote:
On Wed, Mar 09, 2016 at 05:21:50PM +0100, Ludwig Krispenz wrote:
On 03/09/2016 04:46 PM, Andrew E. Bruno wrote:
On Wed, Mar 09, 2016 at 10:37:05AM -0500, Andrew E. Bruno wrote:
On Wed, Mar 09, 2016 at 04:13:28PM +0100, Ludwig Krispenz wrote:
if the process hangs, could you get a pstack from the process ?
I'd be happy to provide a pstack but can't seem to get the correct debuginfo
packages installed.. we're running centos7 and 389-ds-base 1.3.3.1. We haven't
upgraded to 1.3.4.0. How can I get the debuginfo packages installed for that
specific version.
Nevermind.. i got the debuginfo packages. Attached is the stacktrace of
our second failed replicate that's currently hung.
not sure, but the process could be in a deadlock, there are threads in the
retro cl and memberof plugin and we have seen deadlocks there. In that case
restart or stop would not be able to stop the ds process. You can try ipactl
stop and if the ds process is still running, you have to kill it
Our first master came back up after the restart and appears to be
working again. The second replica that was hung, we did a ipactl stop
and it killed the ds process. Running a ipactl start now. We got the
same error:
[09/Mar/2016:11:33:03 -0500] NSMMReplicationPlugin - changelog program -
_cl5NewDBFile: PR_DeleteSemaphore:
/var/lib/dirsrv/slapd-CBLS-CCR-BUFFALO-EDU/cldb/ed35d212-2cb811e5-af63d574-de3f6355.sema;
NSPR error - -5943
if ds is cleanly shutdown this file should be removed, if ds is killed
it remains and should be recreated at restart, which fails. could you
try another stop, remove the file manually and start again ?
We're going to let this run and hopefully will come back up.
Just want to confirm again that these can can be safely ignored:
[09/Mar/2016:10:23:10 -0500] DSRetroclPlugin - delete_changerecord: could not
delete change record 11272989 (rc: 32)
[09/Mar/2016:10:23:10 -0500] DSRetroclPlugin - delete_changerecord: could not
delete change record 11272990 (rc: 32)
there is something wrong with defining the starting point for changelog
trimming, so it will skip many entries, this is annoying and we will
have to fix it. Apart from spamming the logs and keeping the retor cl
busy for a while it should not do any harm.
They fill up the logs when bringing the ds back up.
We seem to keep getting bit by this deadlock [1,2]. Replicas become
unresponsive, file descriptor counts increase. Other than a pstack, if there's
any other info we can provide/check let us know.
We'll be upgrading to centos 7.2 and 389-ds-base 1.3.4 next week.
As always, thanks again for the help and quick responses.
Best,
--Andrew
[1] https://www.redhat.com/archives/freeipa-users/2015-September/msg00006.html
[2] https://www.redhat.com/archives/freeipa-users/2015-June/msg00389.html
--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham, Michael
O'Neill
--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project
