Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-09-08 Thread zxcpoiu
Closed #296. --- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/296#event-403343460___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-09-08 Thread zxcpoiu
Thanks, we're trying to serve as large amount as we can on a single kamailio server. Anyway, after some test, I found the problem may related to debug level. If I switch on memdbg, it'll have a HUGE impact to kamailio and cause TCP connection drop. If I leave debug=-2, then it runs well. Normal

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-09-07 Thread Daniel-Constantin Mierla
It looks like a lot of tcp connections are open (over 15 000) -- for each of them is allocated the read/write buffer. Do you expect so many tcp connections? If yes, then you need to allocate more memory for kamailio, it runs out of it. --- Reply to this email directly or view it on GitHub: https

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-09-05 Thread zxcpoiu
Looks like problems not in jsonrpc-c now? points of the command `kamcmd corex.shm_summary` issued ``` root@SIP-B03:/home/pkg/kamailio/etc/kamailio# kamcmd core.shmmem { total: 268435456 free: 7586408 used: 259640648 real_used: 260849048 max_used: 2608

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-09-04 Thread Daniel-Constantin Mierla
Back to this after some time caught with other stuff. The status with the list of free chunks doesn't help at all. To get the summary, try: ``` memlog=1 ``` in kamailio.cfg Then, at runtime issue the command: ``` kamcmd corex.shm_summary ``` --- Reply to this email directly or view it on Git

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-29 Thread zxcpoiu
I played some human reaction game with kamailio, try to issue `kamailio stop` when I saw shm go down near 0 to get the shm status. hope this helps. I can play more games if you need : ) ### FIRST: missed the point, syslog AFTER kamailio just free shm **below is time sequence where I stop kamai

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-28 Thread zxcpoiu
There are lots of ``` Aug 28 23:04:05 SIP-B03 /home/pkg/kamailio/sbin/kamailio[35260]: : [mem/f_malloc.c:431]: fm_malloc(): fm_malloc(0x7f3b170e, 64) called from jsonrpc-c: jsonrpc_request.c: jsonrpc_notification(135) ``` but no statistics, since it will not shutdown or SIGUSR1. I test un

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-28 Thread Daniel-Constantin Mierla
I was looking for the summary printed in syslog, like you attached in a comment before, eg: ``` Aug 24 14:39:41 SIP-6 /home/pkg/kamailio/sbin/kamailio[263619]: : fm_status: - Aug 24 14:39:41 SIP-6 /home/pkg/kamailio/sbin/kamailio[263619]: : [main.c:569]: cleanup():

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-28 Thread zxcpoiu
This is initial state kamailio just restart ``` root@SIP-B03:~# kamcmd core.shmmem { total: 536870912 free: 531530384 used: 5295024 real_used: 5340528 max_used: 5340528 fragments: 2 } ``` begin test: issue command `kamcmd.shmmem` withi

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-28 Thread Daniel-Constantin Mierla
Can you get the shm summary again, when shm free is very low? --- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/296#issuecomment-135701765___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.si

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-27 Thread zxcpoiu
I compiled brand new kamailio, and use my kamailio.cfg ``` $> git branch -v * master ec0ba37 jsonrpc-c: proper error handling for pipe cmd callback $> kamcmd core.info { version: kamailio 4.4.0-dev2 id: ec0ba3 compiler: gcc 4.7.2 compiled: 22:33:06 Aug 27 2015

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-27 Thread zxcpoiu
I will recompile and test it from the latest master branch ec0ba37b9f2b6c60303227ea0527d69896f2930e --- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/296#issuecomment-135450282___ sr-dev mailing list sr-d

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-27 Thread Daniel-Constantin Mierla
Pushed a commit for it, including some other cleanup operations. --- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/296#issuecomment-135443860___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-27 Thread Daniel-Constantin Mierla
@que273 great hit, thanks! It is missing a return there ... --- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/296#issuecomment-135440046___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-27 Thread Daniel-Constantin Mierla
Are the backtrace and the syslog messages from the same case? One is reporting a crash at line 551 and the other at 553. ``` (gdb) frame 3 #3 0x7fe5d724f9ea in free_pipe_cmd (cmd=0x7fe5dca4a1d0) at jsonrpc_io.c:551 ``` ``` Aug 26 13:53:49 SIP-B03 /home/pkg/kamailio/sbin/kamailio[108633]: :

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-27 Thread Hugh Waite
Hi, I think your problem is at line 238 ``` @@ -237,6 +236,10 @@ pkg_free(ns); json_object_put(payload); + if (cmd->notify_only) free_pipe_cmd(cmd); +error: + if (cmd->notify_only) free_pipe_cmd(cmd); + return; } ``` You are calling free_pipe_cmd before and after the error: label H

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-26 Thread zxcpoiu
oh, sorry for the chaos. here it is: bt_AFTER_patch.corefile ``` (gdb) frame 3 #3 0x7fe5d724f9ea in free_pipe_cmd (cmd=0x7fe5dca4a1d0) at jsonrpc_io.c:551 551 shm_free(cmd->method); (gdb) l 546 547 548 void free_pipe_cmd(struct jsonrpc_pipe_cmd *cmd) 549 { 5

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-26 Thread Daniel-Constantin Mierla
Can you get what is at modules/jsonrpc-c/jsonrpc_io.c line 551? You applied the patch to your version and I cannot match in the source code. --- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/296#issuecomment-134951398___

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-26 Thread zxcpoiu
Attach **kamailio syslog** when crash: BEFORE_patch.syslog ``` Aug 26 13:45:42 SIP-B03 kernel: [29523793.119992] kamailio[106903]: segfault at 0 ip 7fdc4c1e03bf sp 751d9450 error 4 in libjson.so.0.1.0[7fdc4c1dc000+9000] Aug 26 13:45:42 SIP-B03 /home/pkg/kamailio/sbin/kamailio[

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-26 Thread zxcpoiu
bt_BEFORE_patch.corefile with frame3 p *cmd **Addition gdb command** ``` frame 3 p *cmd ``` **Output** ``` #3 0x7fdc4c3eae1a in cmd_pipe_cb (fd=6, event=2, arg=0x751d9750) at jsonrpc_io.c:152 152 json_object *payload = NULL; $1 = {method = 0x0, params = 0x0, cb_route = 0x

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-25 Thread zxcpoiu
I'm afraid that core files has been deleted. Anyway, after test your patch, it seems there are still have issues. So, let's from the ground up again: I will attach **two** back trace below, one is crashed **BEFORE your patch** that I reproduced, the other one is crashed **AFTER your patch**

Re: [sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-25 Thread Daniel-Constantin Mierla
I pushed a patch to fix the leak -- it is in master branch 79459d3f87731309641794a2b881ee1cacc5e1f2 If you test it and report it solves the leak, I will backport. For the crash, not sure it is related, can you get the output in gdb for the next commands? ``` frame 4 p *cmd ``` --- Reply to th

[sr-dev] [kamailio] jsonrpc-c notification does not free shm memory (#296)

2015-08-24 Thread zxcpoiu
Use Case Description: I'm trying to log "user last register status" using function `jsonrpc_notification` from module `jsonrpc-c` # invoke in kamailio.cfg: `jsonrpc_notification("last_reg", "{'user': '$au', 'data':['$Ts', '$au', '$ct', '$ua', '$sel(received.proto_ip_port)']}");` **acc