Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
It was a good catch after all, indeed it should not segfault and now should be fixed. Yesterday I didn't have time to add here the comments about it, but as jsonrpcs replaced the mi_fifo, its default transports were changed in 5.0, given that xhttp is not loaded by default. I also added a note in the upgrade guide from 4.4 to 5.0. Thanks for assisting with troubleshooting. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1030#issuecomment-287521722___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
Closed #1030. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1030#event-1005402855___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
Well, I'll be damned. Your log message gave me the clue. I was simply missing
`modparam("jsonrpcs", "transport", 1)`. I must have missed that in the upgrade.
Maybe that parameter should be added to the default kamailio.cfg since the
`Default value is '6' (fifo and datagram transport).`
Sorry @miconda. But I guess it shouldn't segfault either ;) Thanks again.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1030#issuecomment-287517767___
sr-dev mailing list
sr-dev@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
After applying f81c128, no coredump, but I get `jsonrpcs [jsonrpcs_mod.c:1174]: jsonrpc_dispatch(): jsonrpc over http not initialized - check transport param` FYI, The request is sent in via 127.0.0.1:5060 TCP, with nexthop set as 127.0.0.1:5060 so the request will route through the script and eventually gets looked up by lookup_branches--finding the contact which is a SIP/TLS endpoint. Attached log. [kamailio-coredump-jsonrpcs-message_3.txt](https://github.com/kamailio/kamailio/files/852183/kamailio-coredump-jsonrpcs-message_3.txt) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1030#issuecomment-287517348___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
Just in case, these are some of the script settings:
```
# - jsonrpcs params -
modparam("jsonrpcs", "pretty_format", 1)
/* set the path to RPC fifo control file */
# modparam("jsonrpcs", "fifo_name", "/run/kamailio/kamailio_rpc.fifo")
/* set the path to RPC unix socket control file */
# modparam("jsonrpcs", "dgram_socket", "/run/kamailio/kamailio_rpc.sock")
# - ctl params -
/* set the path to RPC unix socket control file */
# modparam("ctl", "binrpc", "unix:/run/kamailio/kamailio_ctl")
```
Probably unrelated, but a few xhttp-related compiler warnings:
```
make[2]: Entering directory
'/builddir/build/BUILD/kamailio-5.0.0/src/modules/xhttp_rpc'
Makefile.defs defs skipped
Makefile.defs defs skipped
gcc -fPIC -DPIC -g -funroll-loops -Wcast-align -m64 -minline-all-stringops
-falign-loops -ftree-vectorize -fno-strict-overflow -Wall
-DNAME='"kamailio"' -DVERSION='"5.0.0"' -DARCH='"x86_64"' -DOS='linux_'
-DOS_QUOTED='"linux"' -DCOMPILER='"gcc 6.3.1"' -D__CPU_x86_64 -D__OS_linux
-DSER_VER=500 -DCFG_DIR='"/etc/kamailio/"' -DRUN_DIR='"/var/run/kamailio/"'
-DPKG_MALLOC -DSHM_MEM -DSHM_MMAP -DDNS_IP_HACK -DUSE_MCAST -DUSE_TCP
-DDISABLE_NAGLE -DHAVE_RESOLV_RES -DUSE_DNS_CACHE -DUSE_DNS_FAILOVER
-DUSE_DST_BLACKLIST -DUSE_NAPTR -DWITH_XAVP -DMEM_JOIN_FREE -DF_MALLOC
-DQ_MALLOC -DTLSF_MALLOC -DDBG_SR_MEMORY -DUSE_TLS -DTLS_HOOKS -DUSE_CORE_STATS
-DSTATISTICS -DMALLOC_STATS -DWITH_AS_SUPPORT -DUSE_SCTP -DFAST_LOCK
-DADAPTIVE_WAIT -DADAPTIVE_WAIT_LOOPS=1024 -DCC_GCC_LIKE_ASM
-DHAVE_GETHOSTBYNAME2 -DHAVE_UNION_SEMUN -DHAVE_SCHED_YIELD -DHAVE_MSG_NOSIGNAL
-DHAVE_MSGHDR_MSG_CONTROL -DHAVE_ALLOCA_H -DHAVE_TIMEGM
-DHAVE_SCHED_SETSCHEDULER -DHAVE_IP_MREQN -DHAVE_EPOLL -DHAVE_SIGIO_RT
-DSIGINFO64_WORKARROUND -DUSE_FUTEX -DHAVE_SELECT -DKAMAILIO_MOD_INTERFACE
-DMOD_NAME='"xhttp_rpc"' -c xhttp_rpc.c -o xhttp_rpc.o -MMD -MP
gcc -fPIC -DPIC -g -funroll-loops -Wcast-align -m64 -minline-all-stringops
-falign-loops -ftree-vectorize -fno-strict-overflow -Wall
-DNAME='"kamailio"' -DVERSION='"5.0.0"' -DARCH='"x86_64"' -DOS='linux_'
-DOS_QUOTED='"linux"' -DCOMPILER='"gcc 6.3.1"' -D__CPU_x86_64 -D__OS_linux
-DSER_VER=500 -DCFG_DIR='"/etc/kamailio/"' -DRUN_DIR='"/var/run/kamailio/"'
-DPKG_MALLOC -DSHM_MEM -DSHM_MMAP -DDNS_IP_HACK -DUSE_MCAST -DUSE_TCP
-DDISABLE_NAGLE -DHAVE_RESOLV_RES -DUSE_DNS_CACHE -DUSE_DNS_FAILOVER
-DUSE_DST_BLACKLIST -DUSE_NAPTR -DWITH_XAVP -DMEM_JOIN_FREE -DF_MALLOC
-DQ_MALLOC -DTLSF_MALLOC -DDBG_SR_MEMORY -DUSE_TLS -DTLS_HOOKS -DUSE_CORE_STATS
-DSTATISTICS -DMALLOC_STATS -DWITH_AS_SUPPORT -DUSE_SCTP -DFAST_LOCK
-DADAPTIVE_WAIT -DADAPTIVE_WAIT_LOOPS=1024 -DCC_GCC_LIKE_ASM
-DHAVE_GETHOSTBYNAME2 -DHAVE_UNION_SEMUN -DHAVE_SCHED_YIELD -DHAVE_MSG_NOSIGNAL
-DHAVE_MSGHDR_MSG_CONTROL -DHAVE_ALLOCA_H -DHAVE_TIMEGM
-DHAVE_SCHED_SETSCHEDULER -DHAVE_IP_MREQN -DHAVE_EPOLL -DHAVE_SIGIO_RT
-DSIGINFO64_WORKARROUND -DUSE_FUTEX -DHAVE_SELECT -DKAMAILIO_MOD_INTERFACE
-DMOD_NAME='"xhttp_rpc"' -c xhttp_rpc_fnc.c -o xhttp_rpc_fnc.o -MMD -MP
xhttp_rpc_fnc.c:188:18: warning: 'XHTTP_RPC_Post_1a' defined but not used
[-Wunused-const-variable=]
static const str XHTTP_RPC_Post_1a = str_init("\n"\
^
xhttp_rpc_fnc.c:176:18: warning: 'XHTTP_RPC_ATTR_VAL_SEPARATOR' defined but not
used [-Wunused-const-variable=]
static const str XHTTP_RPC_ATTR_VAL_SEPARATOR = str_init("=");
^~~~
xhttp_rpc_fnc.c:175:18: warning: 'XHTTP_RPC_ATTR_SEPARATOR' defined but not
used [-Wunused-const-variable=]
static const str XHTTP_RPC_ATTR_SEPARATOR = str_init(" ");
^~~~
xhttp_rpc_fnc.c:174:18: warning: 'XHTTP_RPC_NODE_SEPARATOR' defined but not
used [-Wunused-const-variable=]
static const str XHTTP_RPC_NODE_SEPARATOR = str_init(":: ");
^~~~
gcc -shared -m64 -Wl,-O2 -Wl,-E xhttp_rpc.o xhttp_rpc_fnc.o -o
xhttp_rpc.so
make[2]: Leaving directory
'/builddir/build/BUILD/kamailio-5.0.0/src/modules/xhttp_rpc'
```
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1030#issuecomment-287205186___
sr-dev mailing list
sr-dev@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
The new log is from a rebuild of the 5.0 branch as before with 0350885c076fa691de6106eb821ec5890572cee6, fa50c85a9e61dd6b91ce88f3f74c433af5d414e2, and a97701575863993f8a19f9dd611156b37ffca247 cherry-picked onto it. Still dumps. [kamailio.script.stateless.debug.3.txt](https://github.com/kamailio/kamailio/files/849124/kamailio.script.stateless.debug.3.txt) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1030#issuecomment-287204075___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
The log messages are not from syslog, so they are missing date/time and which PID printed them. Can you redo the test and grab via syslog? You can also test with a977015 . -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1030#issuecomment-287201871___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
@miconda attached is the debug log using the 5.0 branch at f5889fcc9e42cae01c58357a622d2270f994b8d7. [kamailio.script.debug.3.txt](https://github.com/kamailio/kamailio/files/849081/kamailio.script.debug.3.txt) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1030#issuecomment-287197443___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
I see a97701575863993f8a19f9dd611156b37ffca247 in the master branch, but not the 5.0 branch. Do you want me to include that in the testing as well? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1030#issuecomment-287192751___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
Can you try with latest branch 5.0 and grab again the log messages with the debug=3 in case the issue is still there. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1030#issuecomment-287010991___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
The strange thing is, the MESSAGE is received and transmitted to the recipient, then Kamailio fails, apparently on the reply. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1030#issuecomment-286579620___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
I had to send another message to do this, but here it is: [kamailio-coredump-jsonrpcs-message-script-debug.txt](https://github.com/kamailio/kamailio/files/842912/kamailio-coredump-jsonrpcs-message-script-debug.txt) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1030#issuecomment-286578930___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
Couldn't spot anything relevant wrong for sending the response. Can you grab all the log messages with debug=3 in kamailio.cfg? Hopefully it will show more leads about what frame 0 can be. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1030#issuecomment-286417462___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
See attached. I have replaced my domain with "example.com" and my username with "user". [kamailio-coredump-jsonrpcs-message_2.txt](https://github.com/kamailio/kamailio/files/839849/kamailio-coredump-jsonrpcs-message_2.txt) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1030#issuecomment-286272391___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
Can you also get the content of msg field, respectively: ``` p *ctx->msg ``` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1030#issuecomment-286126055___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
This is reproducable every time. Here is the output you requested:
```
(gdb) frame 1
#1 0x7f1426c9f5d1 in jsonrpc_send (ctx=0x7f1426ec88e0 <_jsonrpc_ctx>) at
jsonrpcs_mod.c:395
395 xhttp_api.reply(ctx->msg, ctx->http_code,
>http_text,
(gdb) p *ctx
$1 = {msg = 0x7ffdd76deda0, msg_shm_block_size = 0, method = 0x23fe850
"tm.t_uac_start", flags = 0, jreq = 0x2402810, req_node = 0x0, jrpl =
0x2402f40, rpl_node = 0x0, reply_sent = 1, error_code = 0, http_code = 200,
http_text = {
s = 0x7f1426cbec14 "OK", len = 2}, transport = 1}
```
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1030#issuecomment-285867501___
sr-dev mailing list
sr-dev@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Kamailio 5 coredump 11 (SEGV) upcon receiving MESSAGE via jsonrpcs (#1030)
Can this be reproduced? If you get other corefile, are the details of frame 0 missing again? Without it, no details of the code it effective crashed are shown. >From this corefile, can you get the output of: ``` frame 0 p *ctx ``` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1030#issuecomment-285852714___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
