Bogdan,
I was able to reproduce this crash on the newer 2.4 commit.
Backtrace: https://pastebin.com/RaMXTZA4 <https://pastebin.com/RaMXTZA4>
# opensips -V
version: opensips 2.4.6 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC,
QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll, sigio_rt, select.
git revision: 714afb97f
main.c compiled on 17:22:40 Jun 26 2019 with gcc 7
Ben Newlin
*From: *Bogdan-Andrei Iancu <bog...@opensips.org>
*Date: *Wednesday, June 26, 2019 at 11:34 AM
*To: *Ben Newlin <ben.new...@genesys.com>, OpenSIPS devel mailling
list <devel@lists.opensips.org>
*Subject: *Re: [OpenSIPS-Devel] OpenSIPS Crash
As per ticket, the issue you reported was already fixed, so going for
the latest version in 2.4 branch should be fine with you.
Regards,
Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
https://www.opensips-solutions.com
OpenSIPS Summit 2019
https://www.opensips.org/events/Summit-2019Amsterdam/
On 06/26/2019 06:21 PM, Ben Newlin wrote:
Bogdan,
I will test both with the latest and with that specific commit,
but I cannot use the latest in production due to the issue I
reported in 2.4.6. [1] Perhaps that specific commit is before
whatever change is causing that issue.
[1] https://github.com/OpenSIPS/opensips/issues/1736
<https://github.com/OpenSIPS/opensips/issues/1736>
Ben Newlin
*From: *Bogdan-Andrei Iancu <bog...@opensips.org>
<mailto:bog...@opensips.org>
*Date: *Wednesday, June 26, 2019 at 10:56 AM
*To: *OpenSIPS devel mailling list <devel@lists.opensips.org>
<mailto:devel@lists.opensips.org>, Ben Newlin
<ben.new...@genesys.com> <mailto:ben.new...@genesys.com>
*Subject: *Re: [OpenSIPS-Devel] OpenSIPS Crash
Hi Ben,
While re-checking some of some data related to your report, I
noticed that your opensips version (revision d025b4f61) is a bit
old and it is missing some fixes that might be relevant for you
(mainly 3a994ceeafc6830a1c3b3f2699a7dc1ea915c83a).
Could you grab the latest 2.4 from git and give it a try please.
Regards,
Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
https://www.opensips-solutions.com
OpenSIPS Summit 2019
https://www.opensips.org/events/Summit-2019Amsterdam/
On 06/26/2019 02:37 PM, Bogdan-Andrei Iancu wrote:
Thank you Ben,
This backtrace confirms the initial suspicion - I'm trying to
find out a way to get it fixed in a more generic way; what you
experience is just one face of the problem (with many faces
:P) and there are other related reports.
Regards,
Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
https://www.opensips-solutions.com
OpenSIPS Summit 2019
https://www.opensips.org/events/Summit-2019Amsterdam/
On 06/26/2019 02:34 AM, Ben Newlin wrote:
Bogdan,
I have now been able to reproduce this crash reliably, I
believe. Please see the backtrace [1]. Is there some other
information you would like me to collect when this occurs?
[1] https://pastebin.com/n0Ph8XH5
<https://pastebin.com/n0Ph8XH5>
Ben Newlin
*From: *Bogdan-Andrei Iancu <bog...@opensips.org>
<mailto:bog...@opensips.org>
*Date: *Friday, June 7, 2019 at 9:15 AM
*To: *Ben Newlin <ben.new...@genesys.com>
<mailto:ben.new...@genesys.com>, OpenSIPS devel mailling
list <devel@lists.opensips.org>
<mailto:devel@lists.opensips.org>
*Subject: *Re: [OpenSIPS-Devel] OpenSIPS Crash
Hi Ben,
How often/easy is to reproduce this crash (if possible) ?
Brainstorming with Razvan, we suspect a race (on the msg
save in shmem in transaction) between the process doing
the cleanup after the async resume and the process running
the failure route (due th 503).
But this is just a supposition, eventually you can
validate it or not by removing the async ??
And on the double ACK - I'm not 100% it is a actually a
double one, as the second has a smaller MF value (69,
versus the 70 on the first ACK).
Regards,
Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
https://www.opensips-solutions.com
OpenSIPS Summit 2019
https://www.opensips.org/events/Summit-2019Amsterdam/
On 06/07/2019 03:52 PM, Ben Newlin wrote:
Bogdan,
Sorry, I should have thought to actually look at the
trace and examine this call.
1) Yes
2) The Called Party is 10.32.20.60, which is another
OpenSIPS instance. The crashed instance received the
"503 Service Unavailable" approximately 8-10 ms after
sending the INVITE.
There is a SIP trace of the exchange here:
https://pastebin.com/6bttsSVD
<https://pastebin.com/6bttsSVD>.
One oddity I saw is that the crashed process appears
to send (or at least siptrace) the ACK twice.
Ben Newlin
*From: *Bogdan-Andrei Iancu <bog...@opensips.org>
<mailto:bog...@opensips.org>
*Date: *Thursday, June 6, 2019 at 11:42 AM
*To: *OpenSIPS devel mailling list
<devel@lists.opensips.org>
<mailto:devel@lists.opensips.org>, Ben Newlin
<ben.new...@genesys.com> <mailto:ben.new...@genesys.com>
*Subject: *Re: [OpenSIPS-Devel] OpenSIPS Crash
Hi Ben,
Thanks for "another" report :).
Questions:
1) do you do any async for the INVITE in this crash ?
2) if it is an YES to (1), is the caller party
generating the "503 Service Unavailable" (which
triggers the crash) - 10.32.20.60 ?? - a really close
(from net delay perspective) and fast to answer party ?
Regards,
Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
https://www.opensips-solutions.com
OpenSIPS Summit 2019
https://www.opensips.org/events/Summit-2019Amsterdam/
On 06/05/2019 10:02 PM, Ben Newlin wrote:
We have had another crash today.
Backtrace is here: https://pastebin.com/q4RQC7kS
<https://pastebin.com/q4RQC7kS>
I found this in the log at the time of the crash:
Jun 5 17:54:10 [4978] CRITICAL:core:sig_usr:
segfault in process pid: 4978, id: 8
Please let me know if any further information can
be useful.
Ben Newlin
*From: *Devel <devel-boun...@lists.opensips.org>
<mailto:devel-boun...@lists.opensips.org> on
behalf of Ben Newlin <ben.new...@genesys.com>
<mailto:ben.new...@genesys.com>
*Reply-To: *OpenSIPS devel mailling list
<devel@lists.opensips.org>
<mailto:devel@lists.opensips.org>
*Date: *Friday, May 10, 2019 at 6:31 PM
*To: *OpenSIPS devel mailling list
<devel@lists.opensips.org>
<mailto:devel@lists.opensips.org>
*Subject: *Re: [OpenSIPS-Devel] OpenSIPS Crash
I found this in the log at the time of the crash:
kernel: opensips[5003]: segfault at 30 ip
00007fbd4c8f59d0 sp 00007ffcaa850c80 error 6 in
tm.so[7fbd4c887000+8e000]
Ben Newlin
*From: *Devel <devel-boun...@lists.opensips.org>
<mailto:devel-boun...@lists.opensips.org> on
behalf of Ben Newlin <ben.new...@genesys.com>
<mailto:ben.new...@genesys.com>
*Reply-To: *OpenSIPS devel mailling list
<devel@lists.opensips.org>
<mailto:devel@lists.opensips.org>
*Date: *Friday, May 10, 2019 at 5:44 PM
*To: *OpenSIPS devel mailling list
<devel@lists.opensips.org>
<mailto:devel@lists.opensips.org>
*Subject: *[OpenSIPS-Devel] OpenSIPS Crash
Hello,
We had a crash today of our OpenSIPS instance.
Backtrace is here: https://pastebin.com/QbRJimwx
# opensips -V
version: opensips 2.4.5 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST,
SHM_MMAP, PKG_MALLOC, F_MALLOC,
FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE
262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE
65535
poll method support: poll, epoll, sigio_rt, select.
git revision: d025b4f61
main.c compiled on 20:58:31 May 9 2019 with gcc 7
Ben Newlin
_______________________________________________
Devel mailing list
Devel@lists.opensips.org
<mailto:Devel@lists.opensips.org>
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
_______________________________________________
Devel mailing list
Devel@lists.opensips.org <mailto:Devel@lists.opensips.org>
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
