Robert Joly wrote:
This appeared on the console.
*** glibc detected *** /usr/bin/sipXproxy: malloc(): memory
corruption: 0xb3ce5408 *** ======= Backtrace: =========
/lib/libc.so.6[0xa70883] /lib/libc.so.6(__libc_malloc+0x7b)[0xa723ab]
/usr/lib/libstdc++.so.6(_Znwj+0x27)[0x583aa7]
/usr/lib/libstdc++.so.6(_Znaj+0x1d)[0x583bdd]
/usr/lib/libsipXport.so.2(_ZN10UtlHashBagC1Ev+0x47)[0x65f1d7]
/usr/lib/libsipXtack.so.2(_ZN10SipMessage9getViaTagEPKcS1_R9Ut
lString+0x57)[0x26c657]
/usr/lib/libsipXtack.so.2(_ZN8BranchId12loopDetectedERK10SipMe
ssage+0x113)[0x2199d3]
/usr/lib/libsipXtack.so.2(_ZN14SipTransaction15recurseChildren
ER12SipUserAgentR18SipTransactionList+0x57b)[0x2ae36b]
/usr/lib/libsipXtack.so.2(_ZN14SipTransaction21startSequential
SearchER12SipUserAgentR18SipTransactionList+0x1cb)[0x2aed8b]
/usr/lib/libsipXtack.so.2(_ZN14SipTransaction21startSequential
SearchER12SipUserAgentR18SipTransactionList+0x181)[0x2aed41]
/usr/lib/libsipXtack.so.2(_ZN14SipTransaction19handleChildInco
mingER10SipMessageR12SipUserAgentNS_19messageRelationshipER18S
ipTransactionListiRPS0_+0x37f)[0x2b0a0f]
/usr/lib/libsipXtack.so.2(_ZN14SipTransaction19handleChildInco
mingER10SipMessageR12SipUserAgentNS_19messageRelationshipER18S
ipTransactionListiRPS0_+0x1e8)[0x2b0878]
/usr/lib/libsipXtack.so.2(_ZN14SipTransaction19handleChildInco
mingER10SipMessageR12SipUserAgentNS_19messageRelationshipER18S
ipTransactionListiRPS0_+0x1e8)[0x2b0878]
/usr/lib/libsipXtack.so.2(_ZN14SipTransaction14handleIncomingE
R10SipMessageR12SipUserAgentNS_19messageRelationshipER18SipTra
nsactionListRPS0_+0x180)[0x2b1170]
/usr/lib/libsipXtack.so.2(_ZN12SipUserAgent8dispatchEP10SipMes
sagei+0xd02)[0x2c0512]
/usr/lib/libsipXtack.so.2(_ZN9SipClient3runEPv+0x509)[0x248ae9]
/usr/lib/libsipXport.so.2(_ZN11OsTaskLinux9taskEntryEPv+0xa9)[
0x6926d9]
/lib/libpthread.so.0[0xb8149b]
/lib/libc.so.6(clone+0x5e)[0xad842e]
======= Memory map: ========
00110000-0017a000 r-xp 00000000 fd:00 8473422
/usr/lib/libodbc.so.1.0.0
0017a000-0017f000 rwxp 00069000 fd:00 8473422
/usr/lib/libodbc.so.1.0.0
0017f000-001ac000 r-xp 00000000 fd:00 8473426
/usr/lib/libgssapi_krb5.so.2.2
001ac000-001ad000 rwxp 0002d000 fd:00 8473426
/usr/lib/libgssapi_krb5.so.2.2
001ad000-001d2000 r-xp 00000000 fd:00 8473424
/usr/lib/libk5crypto.so.3.1
001d2000-001d3000 rwxp 00025000 fd:00 8473424
/usr/lib/libk5crypto.so.3.1
001d3000-001d4000 r-xp 00000000 fd:00 8520106
/usr/lib/gconv/ISO8859-1.so
001d4000-001d6000 rwxp 00000000 fd:00 8520106
/usr/lib/gconv/ISO8859-1.so
001d6000-001d9000 r-xp 00000000 fd:00 8816799
/usr/lib/authplugins/libTransferControl.so.1.0.0
001d9000-001da000 rwxp 00002000 fd:00 8816799
/usr/lib/authplugins/libTransferControl.so.1.0.0
001da000-001dc000 r-xp 00000000 fd:00 8816789
/usr/lib/authplugins/libRequestLinter.so.1.0.0
001dc000-001dd000 rwxp 00001000 fd:00 8816789
/usr/lib/authplugins/libRequestLinter.so.1.0.0
001dd000-0030e000 r-xp 00000000 fd:00 8459383
/usr/lib/libsipXtack.so.2.0.0.#prelink#.lPk4Ah (deleted)
0030e000-00313000 rwxp 00131000 fd:00 8459383
/usr/lib/libsipXtack.so.2.0.0.#prelink#.lPk4Ah (deleted)
00313000-00326000 r-xp 00000000 fd:00 2130125 /lib/libnsl-2.5.so
00326000-00327000 r-xp 00012000 fd:00 2130125 /lib/libnsl-2.5.so
00327000-00328000 rwxp 00013000 fd:00 2130125 /lib/libnsl-2.5.so
00328000-0032a000 rwxp 00328000 00:00 0
0032a000-0032d000 r-xp 00000000 fd:00 8816779
/usr/lib/authplugins/libMSFT_ExchangeTransferHack.so.1.0.0
0032d000-0032e000 rwxp 00002000 fd:00 8816779
/usr/lib/authplugins/libMSFT_ExchangeTransferHack.so.1.0.0
00330000-00414000 r-xp 00000000 fd:00 8466619
/usr/lib/libsipXcommserver.so.2.0.0.#prelink#.JnWlwk (deleted)
00414000-00417000 rwxp 000e4000 fd:00 8466619
/usr/lib/libsipXcommserver.so.2.0.0.#prelink#.JnWlwk (deleted)
00417000-0041b000 rwxp 00417000 00:00 0
0041b000-00424000 r-xp 00000000 fd:00 2130116 /lib/libcrypt-2.5.so
00424000-00425000 r-xp 00008000 fd:00 2130116 /lib/libcrypt-2.5.so
00425000-00426000 rwxp 00009000 fd:00 2130116 /lib/libcrypt-2.5.so
00426000-0044d000 rwxp 00426000 00:00 0
0044d000-00451000 r-xp 00000
Calling still works so I guess it restarted itself ok.
I also see a core file in /var/log/sipxpbx (not sure if it's
related,or a different crash) but gdb seems to spit too many
errors to be useful
gdb /usr/bin/sipXproxy core.15120
<snip copyright, etc>
This GDB was configured as "i386-redhat-linux-gnu"...
(no debugging symbols found)
warning: Can't read pathname for load map: Input/output error.
warning: .dynamic section for
"/usr/lib/libsipXcommserver.so.2" is not at the expected address
warning: difference appears to be caused by prelink,
adjusting expectations
warning: .dynamic section for
"/usr/lib/libsipXtack.so.2" is not at the expected address
(wrong library or version mismatch?)
warning: .dynamic section for
"/usr/lib/libsipXport.so.2" is not at the expected address
warning: difference appears to be caused by prelink,
adjusting expectations
I can do a backtrace, but about 2/3 of the lines look like
"#nn memory address in ?? ()".
Highest sipx library related line is
#8 0x0065f1d7 in UtlHashBag::UtlHashBag () from
/usr/lib/libsipXport.so.2
Hi Chirs, do you mind posting the full backtrace even if it mostly
contains garbage. Also, some questions about your system:
1- what version of sipXecs are you running?
2- on which OS?
3- Is it an HA system?
4- Do you have NAT Traversal feature enabled?
5- Does the console output coincide with some event (incoming call,
outgoing call, ..., ?)
Thanks
1. sipXconfig (4.0.1-015823 2009-06-19T07:16:14 ecs-centos5)
2. RHEL 5.3 (fully up to date according to yum). Linux pbx1.smartt.com
2.6.18-128.1.14.el5 #1 SMP Mon Jun 1 15:52:36 EDT 2009 i686 i686 i386
GNU/Linux
3. Yes. Slave server is same sipx version. It's a VPS running CentOS 5.3
image: Linux pbx3.smartt.com 2.6.18-53.1.13.el5.028stab053.10 #1 SMP Tue
Apr 1 14:45:45 MSD 2008 i686 i686 i386 GNU/Linux
4. The servers and clients are not behind nat -- they all have real IPs.
However under system => servers => pbx1.smartt.com => NAT. Address type
is the default value of "use stun". There doesn't seem to be an off
option for it. I guess I can use "Specify IP address" and enter the
server's address, but IMO there really should be an option to just use
the server's main ip address.
5. Not that I am aware of. I noticed it because I used that ssh windows
for an unrelated dns lookup shortly before it happened. However this is
a production pbx and gets a decent call volume so it wouldn't surprise me.
_______________________________________________
sipx-dev mailing list [email protected]
List Archive: http://list.sipfoundry.org/archive/sipx-dev
Unsubscribe: http://list.sipfoundry.org/mailman/listinfo/sipx-dev
sipXecs IP PBX -- http://www.sipfoundry.org/
