Re: [Puppet Users] Puppet Crashing, libldap error?
Belated update on this one from another 'victim'...
Our CentOS and RHEL hosts don't have ruby-ldap, but we had the same problem,
with ldap-related assertion failures, including a couple of times outside of
a puppet run where ssh sessions would be terminated.
The fix: upgrade the version of nss_ldap to 253-37.el5_6.1 .
Bug details: https://bugzilla.redhat.com/show_bug.cgi?id=684889 and
https://bugzilla.redhat.com/show_bug.cgi?id=703831
Bug fix advisory:
http://rhn.redhat.com/errata/RHBA-2011-0514.html
On 26 April 2011 12:02, Matt Wise wrote:
> Just following up on this a bit more. One additional thing I needed to do
> as well was install ruby-ldap. It wasnt installed by default, and caused all
> kinds of random failures that were hard to troubleshoot. Once installed, all
> the warning messages went away and the failures stopped.
>
> On Apr 19, 2011, at 8:14 AM, Matt Wise wrote:
>
> > Ok, for what its worth, I think I solved this a while back.. but I ran
> into it again the other day, and couldn't remember the fix. I found this
> thread again while searching around — so I figure I should update it so that
> everyone knows what went wrong.
> >
> > We are using not only local files, and LDAP for auth.. but we're using
> 'nsscache' as a backup if the LDAP service is down for any extended period
> of time. The problem seems to be that occasionally nsscache writes out its
> DB files in a way that upsets ruby's ldap library significantly. Disabling
> nsscache, or rebuilding its DB files from scratch seems to solve this
> problem.
> >
> > —Matt
> >
> > On Mar 16, 2011, at 8:46 PM, Daniel Pittman wrote:
> >
> >> On Wed, Mar 16, 2011 at 20:16, Matt Wise wrote:
> >>
> >>> I've got a handful of nodes (3?) out of about 400 that are giving me
> grief... puppet will run either manually or in the service mode. However, in
> the service mode the puppet process dies after an hour or so. I got an
> strace of the failure:
> >>>
> >>> rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
> >>> rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
> >>> read(8, "", 4096) = 0
> >>> rt_sigaction(SIGPIPE, {0x1, [], SA_RESTORER, 0x31af00eb10},
> {0x31af88c700, [], SA_RESTORER|SA_RESTART, 0x31af00eb10}, 8) = 0
> >>> write(2, "ruby: ../../../libraries/libldap"..., 98ruby:
> ../../../libraries/libldap/result.c:113: ldap_result: Assertion `ld !=
> ((void *)0)' failed.
> >>> ) = 98
> >>> rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
> >>> write(4, ":XMLRPC::WEBrickServlet:0x2afa17"..., 495) = 495
> >>> tgkill(1267, 1267, SIGABRT) = 0
> >>> --- SIGABRT (Aborted) @ 0 (0) ---
> >>> +++ killed by SIGABRT (core dumped) +++
> >>> [root@bastion100 ~]#
> >>>
> >>> I'm running Puppet 2.6.5 on CentOS 5.5 x86_64... any thoughts?
> >>
> >> Something is making libldap unhappy, which is being used by Ruby, used
> >> in turn by Puppet. Which triggers the unfortunate error handling
> >> behaviour of "abort the entire process now!!!" that I just *love* from
> >> libraries I depend on.
> >>
> >> So, I would go hunting for details about what causes that particular
> >> assertion to fire in the LDAP library you are using. My guess would
> >> be that there is a bug in there, tickled by some particular data set,
> >> that those hosts hit, and that a newer release would probably fix it.
> >>
> >> Other that turning off LDAP in the Puppet code, there isn't likely
> >> much we can do about it.
> >>
> >> Daniel
> >> --
> >> ⎋ Puppet Labs Developer – http://puppetlabs.com
> >> ✉ Daniel Pittman
> >> ✆ Contact me via gtalk, email, or phone: +1 (877) 575-9775
> >> ♲ Made with 100 percent post-consumer electrons
> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Puppet Crashing, libldap error?
Just following up on this a bit more. One additional thing I needed to do as
well was install ruby-ldap. It wasnt installed by default, and caused all kinds
of random failures that were hard to troubleshoot. Once installed, all the
warning messages went away and the failures stopped.
On Apr 19, 2011, at 8:14 AM, Matt Wise wrote:
> Ok, for what its worth, I think I solved this a while back.. but I ran into
> it again the other day, and couldn't remember the fix. I found this thread
> again while searching around — so I figure I should update it so that
> everyone knows what went wrong.
>
> We are using not only local files, and LDAP for auth.. but we're using
> 'nsscache' as a backup if the LDAP service is down for any extended period of
> time. The problem seems to be that occasionally nsscache writes out its DB
> files in a way that upsets ruby's ldap library significantly. Disabling
> nsscache, or rebuilding its DB files from scratch seems to solve this
> problem.
>
> —Matt
>
> On Mar 16, 2011, at 8:46 PM, Daniel Pittman wrote:
>
>> On Wed, Mar 16, 2011 at 20:16, Matt Wise wrote:
>>
>>> I've got a handful of nodes (3?) out of about 400 that are giving me
>>> grief... puppet will run either manually or in the service mode. However,
>>> in the service mode the puppet process dies after an hour or so. I got an
>>> strace of the failure:
>>>
>>> rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
>>> rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
>>> read(8, "", 4096) = 0
>>> rt_sigaction(SIGPIPE, {0x1, [], SA_RESTORER, 0x31af00eb10}, {0x31af88c700,
>>> [], SA_RESTORER|SA_RESTART, 0x31af00eb10}, 8) = 0
>>> write(2, "ruby: ../../../libraries/libldap"..., 98ruby:
>>> ../../../libraries/libldap/result.c:113: ldap_result: Assertion `ld !=
>>> ((void *)0)' failed.
>>> ) = 98
>>> rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
>>> write(4, ":XMLRPC::WEBrickServlet:0x2afa17"..., 495) = 495
>>> tgkill(1267, 1267, SIGABRT) = 0
>>> --- SIGABRT (Aborted) @ 0 (0) ---
>>> +++ killed by SIGABRT (core dumped) +++
>>> [root@bastion100 ~]#
>>>
>>> I'm running Puppet 2.6.5 on CentOS 5.5 x86_64... any thoughts?
>>
>> Something is making libldap unhappy, which is being used by Ruby, used
>> in turn by Puppet. Which triggers the unfortunate error handling
>> behaviour of "abort the entire process now!!!" that I just *love* from
>> libraries I depend on.
>>
>> So, I would go hunting for details about what causes that particular
>> assertion to fire in the LDAP library you are using. My guess would
>> be that there is a bug in there, tickled by some particular data set,
>> that those hosts hit, and that a newer release would probably fix it.
>>
>> Other that turning off LDAP in the Puppet code, there isn't likely
>> much we can do about it.
>>
>> Daniel
>> --
>> ⎋ Puppet Labs Developer – http://puppetlabs.com
>> ✉ Daniel Pittman
>> ✆ Contact me via gtalk, email, or phone: +1 (877) 575-9775
>> ♲ Made with 100 percent post-consumer electrons
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Puppet Crashing, libldap error?
Ok, for what its worth, I think I solved this a while back.. but I ran into it
again the other day, and couldn't remember the fix. I found this thread again
while searching around — so I figure I should update it so that everyone knows
what went wrong.
We are using not only local files, and LDAP for auth.. but we're using
'nsscache' as a backup if the LDAP service is down for any extended period of
time. The problem seems to be that occasionally nsscache writes out its DB
files in a way that upsets ruby's ldap library significantly. Disabling
nsscache, or rebuilding its DB files from scratch seems to solve this problem.
—Matt
On Mar 16, 2011, at 8:46 PM, Daniel Pittman wrote:
> On Wed, Mar 16, 2011 at 20:16, Matt Wise wrote:
>
>> I've got a handful of nodes (3?) out of about 400 that are giving me
>> grief... puppet will run either manually or in the service mode. However, in
>> the service mode the puppet process dies after an hour or so. I got an
>> strace of the failure:
>>
>> rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
>> rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
>> read(8, "", 4096) = 0
>> rt_sigaction(SIGPIPE, {0x1, [], SA_RESTORER, 0x31af00eb10}, {0x31af88c700,
>> [], SA_RESTORER|SA_RESTART, 0x31af00eb10}, 8) = 0
>> write(2, "ruby: ../../../libraries/libldap"..., 98ruby:
>> ../../../libraries/libldap/result.c:113: ldap_result: Assertion `ld !=
>> ((void *)0)' failed.
>> ) = 98
>> rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
>> write(4, ":XMLRPC::WEBrickServlet:0x2afa17"..., 495) = 495
>> tgkill(1267, 1267, SIGABRT) = 0
>> --- SIGABRT (Aborted) @ 0 (0) ---
>> +++ killed by SIGABRT (core dumped) +++
>> [root@bastion100 ~]#
>>
>> I'm running Puppet 2.6.5 on CentOS 5.5 x86_64... any thoughts?
>
> Something is making libldap unhappy, which is being used by Ruby, used
> in turn by Puppet. Which triggers the unfortunate error handling
> behaviour of "abort the entire process now!!!" that I just *love* from
> libraries I depend on.
>
> So, I would go hunting for details about what causes that particular
> assertion to fire in the LDAP library you are using. My guess would
> be that there is a bug in there, tickled by some particular data set,
> that those hosts hit, and that a newer release would probably fix it.
>
> Other that turning off LDAP in the Puppet code, there isn't likely
> much we can do about it.
>
> Daniel
> --
> ⎋ Puppet Labs Developer – http://puppetlabs.com
> ✉ Daniel Pittman
> ✆ Contact me via gtalk, email, or phone: +1 (877) 575-9775
> ♲ Made with 100 percent post-consumer electrons
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Puppet Crashing, libldap error?
On Wed, Mar 16, 2011 at 20:16, Matt Wise wrote:
> I've got a handful of nodes (3?) out of about 400 that are giving me grief...
> puppet will run either manually or in the service mode. However, in the
> service mode the puppet process dies after an hour or so. I got an strace of
> the failure:
>
> rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
> rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
> read(8, "", 4096) = 0
> rt_sigaction(SIGPIPE, {0x1, [], SA_RESTORER, 0x31af00eb10}, {0x31af88c700,
> [], SA_RESTORER|SA_RESTART, 0x31af00eb10}, 8) = 0
> write(2, "ruby: ../../../libraries/libldap"..., 98ruby:
> ../../../libraries/libldap/result.c:113: ldap_result: Assertion `ld != ((void
> *)0)' failed.
> ) = 98
> rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
> write(4, ":XMLRPC::WEBrickServlet:0x2afa17"..., 495) = 495
> tgkill(1267, 1267, SIGABRT) = 0
> --- SIGABRT (Aborted) @ 0 (0) ---
> +++ killed by SIGABRT (core dumped) +++
> [root@bastion100 ~]#
>
> I'm running Puppet 2.6.5 on CentOS 5.5 x86_64... any thoughts?
Something is making libldap unhappy, which is being used by Ruby, used
in turn by Puppet. Which triggers the unfortunate error handling
behaviour of "abort the entire process now!!!" that I just *love* from
libraries I depend on.
So, I would go hunting for details about what causes that particular
assertion to fire in the LDAP library you are using. My guess would
be that there is a bug in there, tickled by some particular data set,
that those hosts hit, and that a newer release would probably fix it.
Other that turning off LDAP in the Puppet code, there isn't likely
much we can do about it.
Daniel
--
⎋ Puppet Labs Developer – http://puppetlabs.com
✉ Daniel Pittman
✆ Contact me via gtalk, email, or phone: +1 (877) 575-9775
♲ Made with 100 percent post-consumer electrons
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Puppet Crashing, libldap error?
I've got a handful of nodes (3?) out of about 400 that are giving me grief...
puppet will run either manually or in the service mode. However, in the service
mode the puppet process dies after an hour or so. I got an strace of the
failure:
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
read(8, "", 4096) = 0
rt_sigaction(SIGPIPE, {0x1, [], SA_RESTORER, 0x31af00eb10}, {0x31af88c700, [],
SA_RESTORER|SA_RESTART, 0x31af00eb10}, 8) = 0
write(2, "ruby: ../../../libraries/libldap"..., 98ruby:
../../../libraries/libldap/result.c:113: ldap_result: Assertion `ld != ((void
*)0)' failed.
) = 98
rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
write(4, ":XMLRPC::WEBrickServlet:0x2afa17"..., 495) = 495
tgkill(1267, 1267, SIGABRT) = 0
--- SIGABRT (Aborted) @ 0 (0) ---
+++ killed by SIGABRT (core dumped) +++
[root@bastion100 ~]#
I'm running Puppet 2.6.5 on CentOS 5.5 x86_64... any thoughts?
—Matt
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
