Hi Ben,
According to the backtrace, the memory debugger was not activated. Do an
"opensips -V" to check the resulting compile flags - do you see the
DBG_MALLOC and QM_MALLOC ?
Regards,
Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
http://www.opensips-solutions.com
OpenSIPS Bootcamp 2018
http://opensips.org/training/OpenSIPS_Bootcamp_2018/
On 10/31/2018 05:04 PM, Ben Newlin wrote:
Bogdan,
I was able to compile with those options and the crash has occurred
again. Backtrace is here: https://pastebin.com/dezi9xUU.
Even though I had `memdump=1` set in my script, there was no extra
memory debugging information in the logs prior to or at the time of
the crash. I’m not sure if that is expected or not.
Ben Newlin
*From: *Bogdan-Andrei Iancu <bog...@opensips.org>
*Date: *Monday, October 29, 2018 at 8:11 AM
*To: *Ben Newlin <ben.new...@genesys.com>, OpenSIPS devel mailling
list <devel@lists.opensips.org>
*Subject: *Re: [OpenSIPS-Devel] OpenSIPS Crash
Hi Ben,
You can change the compile flags via the Makefile.conf file - the
menuconfig is also updating that file. So during your build you can
simply push a pre-modified Makefile.conf file with the options needed
for memory debugging.
Regards,
Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
http://www.opensips-solutions.com
OpenSIPS Bootcamp 2018
http://opensips.org/training/OpenSIPS_Bootcamp_2018/
On 10/26/2018 05:14 PM, Ben Newlin wrote:
Bogdan,
Unfortunately, we have run into a similar issue before. Our build
system is completely automated and there is no way to inject the
`make menuconfig` interactive step into that process. If I were to
be testing this locally I might be able to work something out, but
I could never get such a build into our testing environment which
is where the crashes are occurring.
Do you have instructions for enabling memory debugging that do not
require using the interactive TUI tool? What does the menuconfig
program do when these options are selected? Are there some defines
or other settings we can change ourselves and bypass menuconfig?
Ben Newlin
*From: *Bogdan-Andrei Iancu <bog...@opensips.org>
<mailto:bog...@opensips.org>
*Date: *Friday, October 26, 2018 at 4:59 AM
*To: *OpenSIPS devel mailling list <devel@lists.opensips.org>
<mailto:devel@lists.opensips.org>, Ben Newlin
<ben.new...@genesys.com> <mailto:ben.new...@genesys.com>
*Subject: *Re: [OpenSIPS-Devel] OpenSIPS Crash
Hi Ben,
all the BT's points to crashes while doing memory ops. I suspect a
memory corruption that randomly triggers crashes in different
parts of the code.
Could you try to re-compile with memory debugging support ? See
http://www.opensips.org/Documentation/TroubleShooting-OutOfMem,
the "How to handle it" section.
Regards,
Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
http://www.opensips-solutions.com
OpenSIPS Bootcamp 2018
http://opensips.org/training/OpenSIPS_Bootcamp_2018/
On 10/24/2018 04:28 AM, Ben Newlin wrote:
We have had 2 more crashes today.
Crash 2: https://pastebin.com/rMruBQcZ
<https://pastebin.com/rMruBQcZ>
This crash appears to have occurred while processing an
initial INVITE request. I could not see anything unusual about
the request. I cannot tell if this crash is related to the others.
Crash 3: https://pastebin.com/Gmk1m4NT
<https://pastebin.com/Gmk1m4NT>
This crash follows the pattern of the original crash I reported.
Ben Newlin
*From: *Devel <devel-boun...@lists.opensips.org>
<mailto:devel-boun...@lists.opensips.org> on behalf of Ben
Newlin <ben.new...@genesys.com> <mailto:ben.new...@genesys.com>
*Reply-To: *OpenSIPS devel mailling list
<devel@lists.opensips.org> <mailto:devel@lists.opensips.org>
*Date: *Monday, October 22, 2018 at 4:45 PM
*To: *OpenSIPS devel mailling list <devel@lists.opensips.org>
<mailto:devel@lists.opensips.org>
*Subject: *Re: [OpenSIPS-Devel] OpenSIPS Crash
Here is a better trace of the call:
https://pastebin.com/gWpQR8E7 <https://pastebin.com/gWpQR8E7>
Ben Newlin
*From: *Ben Newlin <ben.new...@genesys.com>
<mailto:ben.new...@genesys.com>
*Date: *Monday, October 22, 2018 at 4:34 PM
*To: *OpenSIPS devel mailling list <devel@lists.opensips.org>
<mailto:devel@lists.opensips.org>
*Subject: *OpenSIPS Crash
Hello,
We have been having sporadic crashes and I was recently able
to recover a core dump for one. I have uploaded it here:
https://pastebin.com/ABktcYcH <https://pastebin.com/ABktcYcH>.
I picked out a Call-ID from the crash data and took a look in
our tracing. I have uploaded it here:
https://pastebin.com/ZEzUUKZ5 <https://pastebin.com/ZEzUUKZ5>.
It appears that a downstream server was extremely lagged and
failed to respond to an INVITE. We sent the INVITE to another
server and the call was connected, but then eventually the
original server “caught up” and sent a burst of 200 OK
responses. The crash seems to have occurred processing the ACK
to one of these responses.
Ben Newlin
_______________________________________________
Devel mailing list
Devel@lists.opensips.org <mailto:Devel@lists.opensips.org>
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
_______________________________________________
Devel mailing list
Devel@lists.opensips.org
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
