Re: [sr-dev] [kamailio/kamailio] Regression on app_lua after ASLR enable (Issue #3202)
@miconda, what about to merge f5c98a49c98aedcf6e1afec3c42dd862d0eeb9a3 and 69ba64e26e3876ce84053a691dee2f2ad9bb6185 to 5.6? -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/3202#issuecomment-1260918081 You are receiving this because you are subscribed to this thread. Message ID: ___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] pike: fixed regression (#2744)
@miconda, yep, it's the fix for regression. Pike (`pike_check_req()`) doesn't work without the fix at all. :( It has nothing to do with previous PR (str case search). I've investigated the regression and found two issues (str case search and unnecessary NULL-return). -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/2744#issuecomment-848223214___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
[sr-dev] [kamailio/kamailio] pike: fixed regression (#2744)
Pre-Submission Checklist - [x] Commit message has the format required by CONTRIBUTING guide - [x] Commits are split per component (core, individual modules, libs, utils, ...) - [x] Each component has a single commit (if not, squash them into one commit) - [x] No commits to README files for modules (changes must be done to docbook files in `doc/` subfolder, the README file is autogenerated) Type Of Change - [x] Small bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds new functionality) - [ ] Breaking change (fix or feature that would change existing functionality) Checklist: - [x] PR should be backported to stable branches - [x] Tested changes locally - [ ] Related to issue # (replace with an open issue number) Description There's an regression in 5.5 on `pike.so`. After c9dc0336a33a0ecfe776975be7fbfab8f3c91b48 the module has stopped to work. `mark_node()` had returned a NULL on every IP-address. You can view, comment on, or merge this pull request online at: https://github.com/kamailio/kamailio/pull/2744 -- Commit Summary -- * pike: fixed regression -- File Changes -- M src/modules/pike/ip_tree.c (4) -- Patch Links -- https://github.com/kamailio/kamailio/pull/2744.patch https://github.com/kamailio/kamailio/pull/2744.diff -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/2744 ___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
[sr-dev] [kamailio/kamailio] core: ut - fixed str_casesearch() (#2743)
Pre-Submission Checklist - [x] Commit message has the format required by CONTRIBUTING guide - [x] Commits are split per component (core, individual modules, libs, utils, ...) - [x] Each component has a single commit (if not, squash them into one commit) - [x] No commits to README files for modules (changes must be done to docbook files in `doc/` subfolder, the README file is autogenerated) Type Of Change - [x] Small bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds new functionality) - [ ] Breaking change (fix or feature that would change existing functionality) Checklist: - [x] PR should be backported to stable branches - [x] Tested changes locally - [ ] Related to issue # Description There's an regression after 42228552b72267786561704f120e3da3aac5fd89 `pike.so` was affected by the commit. RPC `pike.top` and `pike.list` commands didn't work. `str_casesearch()` had returned NULL while text and needle were equal. You can view, comment on, or merge this pull request online at: https://github.com/kamailio/kamailio/pull/2743 -- Commit Summary -- * core: ut - fixed str_casesearch() -- File Changes -- M src/core/ut.c (4) -- Patch Links -- https://github.com/kamailio/kamailio/pull/2743.patch https://github.com/kamailio/kamailio/pull/2743.diff -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/2743 ___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] nathelper: fix_nated_sdp added ignoring RFC3605-param if omitted (#2737)
@miconda the fix takes care IPv4 (AF_INET socket type). We've tested it on IPv4, but we cannot test it on IPv6. :( -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/2737#issuecomment-843832197___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
[sr-dev] [kamailio/kamailio] Topos: critical error after upgrade to 5.5 (#2734)
### Description I've faced a critical error in the log after upgrade to 5.5 on _topos_ module. ### Troubleshooting It looks like _ki_tps_set_context()_ tries to free incorrectly defined variable __tps_context_value_. Reproduction ``` loadmodule "topos.so" modparam("topos", "storage", "db") modparam("topos", "db_url", DBURL_PRIMARY) modparam("topos", "sanity_checks", 1) modparam("topos", "branch_expire", 300) modparam("topos", "dialog_expire", 7200) modparam("topos", "clean_interval", 120) ``` Log Messages ``` 20:59:43.951476 kamailio 81412 CRITICAL: [core/mem/q_malloc.c:502]: qm_free(): BUG: bad pointer 0x82310fbdc (out of memory block!) called from topos: topos_mod.c: ki_tps_set_context(315) - ignoring 20:59:58.925224 kamailio 81413 CRITICAL: [core/mem/q_malloc.c:502]: qm_free(): BUG: bad pointer 0x82310fbdc (out of memory block!) called from topos: topos_mod.c: ki_tps_set_context(315) - ignoring 21:00:13.930896 kamailio 81414 CRITICAL: [core/mem/q_malloc.c:502]: qm_free(): BUG: bad pointer 0x82310fbdc (out of memory block!) called from topos: topos_mod.c: ki_tps_set_context(315) - ignoring 21:00:28.929303 kamailio 81415 CRITICAL: [core/mem/q_malloc.c:502]: qm_free(): BUG: bad pointer 0x82310fbdc (out of memory block!) called from topos: topos_mod.c: ki_tps_set_context(315) - ignoring 21:00:43.925256 kamailio 81416 CRITICAL: [core/mem/q_malloc.c:502]: qm_free(): BUG: bad pointer 0x82310fbdc (out of memory block!) called from topos: topos_mod.c: ki_tps_set_context(315) - ignoring 21:00:58.925535 kamailio 81417 CRITICAL: [core/mem/q_malloc.c:502]: qm_free(): BUG: bad pointer 0x82310fbdc (out of memory block!) called from topos: topos_mod.c: ki_tps_set_context(315) - ignoring 21:01:13.925081 kamailio 81418 CRITICAL: [core/mem/q_malloc.c:502]: qm_free(): BUG: bad pointer 0x82310fbdc (out of memory block!) called from topos: topos_mod.c: ki_tps_set_context(315) - ignoring 21:01:28.925306 kamailio 81419 CRITICAL: [core/mem/q_malloc.c:502]: qm_free(): BUG: bad pointer 0x82310fbdc (out of memory block!) called from topos: topos_mod.c: ki_tps_set_context(315) - ignoring ``` ### Additional Information ``` version: kamailio 5.5.0 (x86_64/freebsd) d4c1a1 flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, select, kqueue. id: d4c1a1 compiled on 13:02:36 May 13 2021 with cc FreeBSD clang version 10.0.1 (g...@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2) ``` * **Operating System**: FreeBSD 13 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2734___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
[sr-dev] [kamailio/kamailio] CORE_TLS option in 5.4 (#2720)
### Description There's a `CORE_TLS` option in _Makefile.defs_. But Kamailio isn't build with the option. https://github.com/kamailio/kamailio/blob/cf105d5af78963759825f5eaf9feb767c047a49c/src/main.c#L104 There is no `core/tls/tls_init.h` for include. ### Additional Information Kamailio 5.4.5 * **Operating System**: FreeBSD 12.2 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2720___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Random crashes on tcp_main.c:handle_tcp_child() on FreeBSD (#2638)
Closed #2638. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2638#event-4663472651___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Random crashes on tcp_main.c:handle_tcp_child() on FreeBSD (#2638)
I'm closing the issue as a resolved main annoying problem. I will open new issue if new problem will have occured (ex. on syslog() function). -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2638#issuecomment-829328416___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Random crashes on tcp_main.c:handle_tcp_child() on FreeBSD (#2638)
@miconda ``` (lldb) p *tcp_children (tcp_child) $0 = { pid = 51051 proc_no = 25 unix_sock = 40 busy = 0 mysocket = 0x n_reqs = 56 } ``` ``` (lldb) p *tcpconn (tcp_connection) $2 = { s = -1 fd = -1 write_lock = 0 id = 29 reader_pid = 0 rcv = { src_ip = { af = 2 len = 4 u = { addrl = ([0] = 1404229979, [1] = 0) addr32 = ([0] = 1404229979, [1] = 0, [2] = 0, [3] = 0) addr16 = ([0] = 55643, [1] = 21426, [2] = 0, [3] = 0, [4] = 0, [5] = 0, [6] = 0, [7] = 0) addr = { [0] = '[' [1] = '\xd9' [2] = '\xb2' [3] = 'S' [4] = '\0' [5] = '\0' [6] = '\0' [7] = '\0' [8] = '\0' [9] = '\0' [10] = '\0' [11] = '\0' [12] = '\0' [13] = '\0' [14] = '\0' [15] = '\0' } } } dst_ip = { af = 2 len = 4 u = { addrl = ([0] = 1555224923, [1] = 0) addr32 = ([0] = 1555224923, [1] = 0, [2] = 0, [3] = 0) addr16 = ([0] = 55643, [1] = 23730, [2] = 0, [3] = 0, [4] = 0, [5] = 0, [6] = 0, [7] = 0) addr = { [0] = '[' [1] = '\xd9' [2] = '\xb2' [3] = '\' [4] = '\0' [5] = '\0' [6] = '\0' [7] = '\0' [8] = '\0' [9] = '\0' [10] = '\0' [11] = '\0' [12] = '\0' [13] = '\0' [14] = '\0' [15] = '\0' } } } src_port = 5071 dst_port = 0 proto_reserved1 = 29 proto_reserved2 = 0 src_su = { s = (sa_len = '\x10', sa_family = '\x02', sa_data = char [14] @ 0x00083078485a) sin = { sin_len = '\x10' sin_family = '\x02' sin_port = 53011 sin_addr = (s_addr = 1404229979) sin_zero = { [0] = '\0' [1] = '\0' [2] = '\0' [3] = '\0' [4] = '\0' [5] = '\0' [6] = '\0' [7] = '\0' } } sin6 = { sin6_len = '\x10' sin6_family = '\x02' sin6_port = 53011 sin6_flowinfo = 1404229979 sin6_addr = { __u6_addr = { __u6_addr8 = { [0] = '\0' [1] = '\0' [2] = '\0' [3] = '\0' [4] = '\0' [5] = '\0' [6] = '\0' [7] = '\0' [8] = '\0' [9] = '\0' [10] = '\0' [11] = '\0' [12] = '\0' [13] = '\0' [14] = '\0' [15] = '\0' } __u6_addr16 = ([0] = 0, [1] = 0, [2] = 0, [3] = 0, [4] = 0, [5] = 0, [6] = 0, [7] = 0) __u6_addr32 = ([0] = 0, [1] = 0, [2] = 0, [3] = 0) } } sin6_scope_id = 0 } sas = (ss_len = '\x10', ss_family = '\x02', __ss_pad1 = char [6] @ 0x00083078485a, __ss_align = 0, __ss_pad2 = char [112] @ 0x000830784868) } bind_address = 0x000801492170 proto = '\x03' proto_pad0 = '\0' proto_pad1 = 0 } cinfo = { src_ip = { af = 0 len = 0 u = { addrl = ([0] = 0, [1] = 0) addr32 = ([0] = 0, [1] = 0, [2] = 0, [3] = 0) addr16 = ([0] = 0, [1] = 0, [2] = 0, [3] = 0, [4] = 0, [5] = 0, [6] = 0, [7] = 0) addr = { [0] = '\0' [1] = '\0' [2] = '\0' [3] = '\0' [4] = '\0' [5] = '\0' [6] = '\0' [7] = '\0' [8] = '\0' [9] = '\0' [10] = '\0' [11] = '\0' [12] = '\0' [13] = '\0' [14] = '\0' [15] = '\0' } } } dst_ip = { af = 0 len = 0 u = { addrl = ([0] = 0, [1] = 0) addr32 = ([0] = 0, [1] = 0, [2] = 0, [3] = 0) addr16 = ([0] = 0, [1] = 0, [2] = 0, [3] = 0, [4] = 0, [5] = 0, [6] = 0, [7] = 0) addr = { [0] = '\0' [1] = '\0' [2] = '\0' [3] = '\0' [4] = '\0' [5] = '\0' [6] = '\0' [7] = '\0' [8] = '\0' [9] = '\0' [10] = '\0' [11] = '\0' [12] = '\0' [13] = '\0' [14] = '\0' [15] = '\0' } } } src_port = 0 dst_port = 0 proto = 0 csocket = 0x } req = { next = 0x buf = 0x000802e93610 "\r\n\r\n2.0 487 Request Terminated\r\nVia: SIP/2.0/TLS 91.217.xx.yy:5070;rport=12181;received=91.217.xx.yy;branch=z9hG4bK6b3e.a1465b8bd9ff395e0dd82d3998ac2841.0;i=1\r\nVia: SIP/2.0/UDP 127.0.0.127;branch=z9hG4bKsr-TU0.jAsiBomT.bdNBXQqjzxIGo33Go3q8XebBLYqG-n0YIKJVvKQlXxrGg3-Bg3C8AQiBz.qGbpslupJVzWqwhmqVLUb8Lxr8zWRwv4iYINmFz0epApRgAxUGL.q8zx-BAd*\r\nRecord-Route: \r\nRecord-Rout
Re: [sr-dev] [kamailio/kamailio] Random crashes on tcp_main.c:handle_tcp_child() on FreeBSD (#2638)
@miconda, do you prefer a libressl? Yep, Kamailio is with a tls module. ``` (lldb) f 17 frame #17: 0x00668966 kamailio`handle_tcp_child(tcp_c=0x0008014dcf68, fd_i=-1) at tcp_main.c:3706:5 (lldb) fr v (tcp_child *) tcp_c = 0x0008014dcf68 (int) fd_i = -1 (tcp_connection *) tcpconn = 0x000802e93270 (long [2]) response = ([0] = 34408575600, [1] = -1) (int) cmd = -1 (int) bytes = 16 (int) n = 0 (ticks_t) t = 745497382 (ticks_t) crt_timeout = 28880 (ticks_t) con_lifetime = 28880 (lldb) ta v tcp_children (tcp_child *) tcp_children = 0x0008014dcec8 ``` there're no sources for the list/source list command -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2638#issuecomment-781926443___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Random crashes on tcp_main.c:handle_tcp_child() on FreeBSD (#2638)
@miconda, we use openssl. We can try to rebuild the kamailio with libressl support. I've investigated the problem much more and found out the crash has appeared on handle_tcp_child() instead tls_h_tcpconn_clean_f(). -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2638#issuecomment-781489573___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Core dumps possibly related to #2616 or http_async_query (#2632)
I've have similar crashes on syslog(). But I don't use HTTP(S). ### Troubleshooting Debugging Data ``` (lldb) target create "kamailio" --core "/var/coredump/986/kamailio.4087.core" Core file '/var/coredump/986/kamailio.4087.core' (x86_64) was loaded. (lldb) bt all * thread #1, name = 'kamailio', stop reason = signal SIGSEGV * frame #0: 0x0008009fa56d libc.so.7`__je_tcache_bin_flush_small [inlined] extent_arena_get(extent=0x) at extent_inlines.h:39:43 frame #1: 0x0008009fa56d libc.so.7`__je_tcache_bin_flush_small(tsd=, tcache=, tbin=0x000800d1a2a8, binind=, rem=33) at jemalloc_tcache.c:123 frame #2: 0x0008009fa17c libc.so.7`__je_tcache_event_hard(tsd=, tcache=0x000800d1a250) at jemalloc_tcache.c:54:4 frame #3: 0x000800a37934 libc.so.7`__malloc [inlined] arena_malloc(tsdn=0x000800d1a090, arena=0x, size=, ind=, zero=false, tcache=, slow_path=false) at arena_inlines_b.h:0 frame #4: 0x000800a378a7 libc.so.7`__malloc [inlined] iallocztm(tsdn=0x000800d1a090, size=, ind=, zero=false, tcache=, is_internal=false, arena=0x, slow_path=false) at jemalloc_internal_inlines_c.h:53 frame #5: 0x000800a378a7 libc.so.7`__malloc [inlined] imalloc_no_sample(sopts=, dopts=, tsd=, size=, usize=, ind=) at jemalloc_jemalloc.c:1713 frame #6: 0x000800a378a7 libc.so.7`__malloc [inlined] imalloc_body(sopts=, dopts=, tsd=) at jemalloc_jemalloc.c:1909 frame #7: 0x000800a378a7 libc.so.7`__malloc [inlined] imalloc(sopts=, dopts=) at jemalloc_jemalloc.c:2009 frame #8: 0x000800a3768b libc.so.7`__malloc(size=) at jemalloc_jemalloc.c:2042 frame #9: 0x000800acaf34 libc.so.7`__smakebuf(fp=0x000800d06010) at makebuf.c:73:11 frame #10: 0x000800acae39 libc.so.7`__swsetup(fp=0x000800d06010) at wsetup.c:82:3 frame #11: 0x000800a67b53 libc.so.7`__vfprintf(fp=0x000800d06010, locale=0x000800ade698, fmt0="", ap=0x7fff8cf0) at vfprintf.c:462:6 frame #12: 0x000800a678c5 libc.so.7`vfprintf_l(fp=0x000800d06010, locale=0x000800ade698, fmt0="", ap=0x7fff8cf0) at vfprintf.c:285:9 frame #13: 0x000800a6f113 libc.so.7`fprintf(fp=, fmt=) at fprintf.c:57:8 frame #14: 0x000800aa8f01 libc.so.7`vsyslog [inlined] vsyslog1(pri=135, fmt="", ap=0x7fff9c50) at syslog.c:173:8 frame #15: 0x000800aa8e4c libc.so.7`vsyslog(pri=, fmt="", ap=0x7fff9c50) at syslog.c:363 frame #16: 0x000800aa8dcd libc.so.7`syslog(pri=, fmt=) at syslog.c:129:2 frame #17: 0x006655dc kamailio`handle_tcp_child(tcp_c=0x0008014dcf08, fd_i=-1) at tcp_main.c:3559:2 frame #18: 0x0065b7e9 kamailio`handle_io(fm=0x0008014e6080, ev=1, idx=-1) at tcp_main.c:4523:8 frame #19: 0x00644502 kamailio`io_wait_loop_kqueue(h=0x008dbf50, t=5, repeat=0) at io_wait.h:1187:9 frame #20: 0x0063cb62 kamailio`tcp_main_loop at tcp_main.c:4824:5 frame #21: 0x002e973a kamailio`main_loop at main.c:1779:5 frame #22: 0x002f7650 kamailio`main(argc=7, argv=0x7fffec70) at main.c:2856:6 frame #23: 0x002d1af0 kamailio`_start(ap=, cleanup=) at crt1.c:76:7 ``` Log Messages ``` 03:20:21.349512 4087 - - DEBUG: [core/tcp_main.c:4130]: send2child(): selected tcp worker idx:1 proc:26 pid:4080 for activity on [tcp:91.217.xx.yy:5060], 0x802e70718 03:20:27.364691 4087 - - DEBUG: [core/tcp_main.c:3560]: handle_tcp_child(): reader response= 802e70718, 1 from 1 03:20:27.364727 4087 - - DEBUG: [core/io_wait.h:375]: io_watch_add(): DBG: io_watch_add(0x8dbf50, 53, 2, 0x802e70718), fd_no=43 03:20:27.364746 4087 - - DEBUG: [core/tcp_main.c:3687]: handle_tcp_child(): CONN_RELEASE 0x802e70718 refcnt= 1 03:20:29.740878 4062 - - DEBUG: {1 1001 SUBSCRIBE ceccb6ad-add35f9c@10.10.10.251} [core/socket_info.c:646]: grep_sock_info(): checking if host==us: 13==13 && [78.37.aa.bb] == [91.217.xx.yy] 03:20:51.496077 4087 - - DEBUG: [core/io_wait.h:600]: io_watch_del(): DBG: io_watch_del (0x8dbf50, 53, -1, 0x0) fd_no=44 called 03:20:51.496127 4087 - - DEBUG: [core/tcp_main.c:4457]: handle_tcpconn_ev(): sending to child, events 1 03:20:51.496147 4087 - - DEBUG: [core/tcp_main.c:4130]: send2child(): selected tcp worker idx:2 proc:27 pid:4081 for activity on [tcp:91.217.xx.yy:5060], 0x802e70718 03:20:58.190139 4054 - - ALERT: [main.c:777]: handle_sigs(): child process 4087 exited by a signal 11 ``` ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` kamailio 5.4.3 (x86_64/freebsd) e19ae3 flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_
[sr-dev] [kamailio/kamailio] Random crashes on tls_server.c:tls_h_tcpconn_clean_f() (#2638)
### Description We've random crashes (approx one occurrence per day) on tls_h_tcpconn_clean_f(). ### Troubleshooting Reproduction The crash appears **sometimes** on closing TLS session (it doesn't have to be gracefully) initiated by remote edge. Debugging Data ``` (lldb) target create "/usr/local/sbin/kamailio" --core "/var/coredump/986/kamailio.51059.core" Core file '/var/coredump/986/kamailio.51059.core' (x86_64) was loaded. (lldb) bt * thread #1, name = 'kamailio', stop reason = signal SIGSEGV * frame #0: 0x000800a13cd4 libc.so.7`__je_large_dalloc(tsdn=0x000800d1a090, extent=0x) at jemalloc_large.c:346 frame #1: 0x000800a3acb7 libc.so.7`__free [inlined] rtree_leaf_elm_lookup(tsdn=0x000800d1a090, rtree=, rtree_ctx=, key=, dependent=true, init_missing=false) at rtree.h:337:6 frame #2: 0x000800a3acb2 libc.so.7`__free [inlined] rtree_read(tsdn=0x000800d1a090, rtree=, rtree_ctx=, key=, dependent=true) at rtree.h:406 frame #3: 0x000800a3acb2 libc.so.7`__free [inlined] rtree_extent_read(tsdn=0x000800d1a090, rtree=, rtree_ctx=, key=, dependent=true) at rtree.h:418 frame #4: 0x000800a3acb2 libc.so.7`__free [inlined] iealloc(tsdn=0x000800d1a090, ptr=0x0008235782f0) at jemalloc_internal_inlines_b.h:82 frame #5: 0x000800a3acb2 libc.so.7`__free [inlined] arena_dalloc(tsdn=0x000800d1a090, ptr=0x0008235782f0, tcache=0x000800cf4c80, alloc_ctx=, slow_path=false) at arena_inlines_b.h:236 frame #6: 0x000800a3ac91 libc.so.7`__free [inlined] idalloctm(tsdn=0x000800d1a090, ptr=0x0008235782f0, tcache=0x000800cf4c80, alloc_ctx=, is_internal=false, slow_path=false) at jemalloc_internal_inlines_c.h:118 frame #7: 0x000800a3ac91 libc.so.7`__free [inlined] ifree(tsd=, ptr=0x0008235782f0, tcache=0x000800cf4c80, slow_path=false) at jemalloc_jemalloc.c:2226 frame #8: 0x000800a3ac91 libc.so.7`__free(ptr=) at jemalloc_jemalloc.c:2382 frame #9: 0x000800d9ed0b libthr.so.3`_thr_rwlock_destroy(rwlock=) at thr_rwlock.c:136:3 frame #10: 0x0008027feba3 libcrypto.so.111`CRYPTO_THREAD_lock_free(lock=0x000802e87e20) at threads_pthread.c:107:5 frame #11: 0x0008027ec676 libcrypto.so.111`BIO_free(a=0x000802e88920) at bio_lib.c:136:5 frame #12: 0x0008027ed572 libcrypto.so.111`BIO_free_all(bio=0x) at bio_lib.c:691:9 frame #13: 0x000802540d0b libssl.so.111`SSL_free(s=0x000802e799a8) at ssl_lib.c:1160:5 frame #14: 0x000802482fdd tls.so`tls_h_tcpconn_clean_f(c=0x000802e93270) at tls_server.c:655:3 frame #15: 0x00618bd8 kamailio`_tcpconn_free(c=0x000802e93270) at tcp_main.c:1528:58 frame #16: 0x00659534 kamailio`tcpconn_put_destroy(tcpconn=0x000802e93270) at tcp_main.c:3269:3 frame #17: 0x00668966 kamailio`handle_tcp_child(tcp_c=0x0008014dcf68, fd_i=-1) at tcp_main.c:3706:5 frame #18: 0x0065b7e9 kamailio`handle_io(fm=0x0008014e6110, ev=1, idx=-1) at tcp_main.c:4523:8 frame #19: 0x00644502 kamailio`io_wait_loop_kqueue(h=0x008dbf50, t=5, repeat=0) at io_wait.h:1187:9 frame #20: 0x0063cb62 kamailio`tcp_main_loop at tcp_main.c:4824:5 frame #21: 0x002e973a kamailio`main_loop at main.c:1779:5 frame #22: 0x002f7650 kamailio`main(argc=7, argv=0x7fffec68) at main.c:2856:6 frame #23: 0x002d1af0 kamailio`_start(ap=, cleanup=) at crt1.c:76:7 ``` All backtraces are differ from other on frames downed SSL_free() (frame #13 on the backtrace). Log Messages ``` 17:40:34.305319 51059 - - DEBUG: [core/io_wait.h:600]: io_watch_del(): DBG: io_watch_del (0x8dbf50, 55, -1, 0x0) fd_no=46 called 17:40:34.30536351059 - - DEBUG: [core/tcp_main.c:4457]: handle_tcpconn_ev(): sending to child, events 11 17:40:34.305382 51059 - - DEBUG: [core/tcp_main.c:4130]: send2child(): selected tcp worker idx:5 proc:30 pid:51056 for activity on [tls:91.217.xx.yy:5070], 0x802e93270 17:40:34.305430 51059 - - DEBUG: [core/io_wait.h:600]: io_watch_del(): DBG: io_watch_del (0x8dbf50, 56, -1, 0x0) fd_no=45 called 17:40:34.305446 51059 - - DEBUG: [core/tcp_main.c:4457]: handle_tcpconn_ev(): sending to child, events 11 17:40:34.305461 51059 - - DEBUG: [core/tcp_main.c:4130]: send2child(): selected tcp worker idx:6 proc:31 pid:51057 for activity on [tls:91.217.xx.yy:5070], 0x802ea7430 17:40:34.305678 51059 - - DEBUG: [core/tcp_main.c:3560]: handle_tcp_child(): reader response= 802e93270, -1 from 5 17:40:34.305700 51059 - - DEBUG: tls [tls_server.c:683]: tls_h_tcpconn_close_f(): Closing SSL connection 0x802e9b3a8 17:40:35.024013 51026 - - ALERT: [main.c:777]: handle_sigs(): child process 51059 exited by a signal 11 ``` ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` kamailio 5.4.3 (x86_64/freebsd) e19ae3 flags: USE_TCP, USE_TLS, USE_S