[Freeswitch-users] Memory leak with mod_local_stream
Hey guys, Having a problem with mod_local_stream. I recently did a make current from 15334 to the latest trunk (15630). After restarting, there now appears to be a memory leak. On a test system (CentOS 5.4, 64-bit) with no calls or registrations, Freeswitch gradually consumes all of the host memory (rate of about 200K/second), then swaps out, eventually rendering the system useless. I isolated it to mod_local_stream. If I unload mod_local_stream, the memory use stops climbing. If I re-load mod_local_stream, it starts again. I would submit the logs except they aren't any besides it starting. The system is just sitting there idle. Even valgrind didn't show much (http://pastebin.freeswitch.org/11238). Maybe I'm using it wrong? I ran it: valgrind --tool=memcheck --log-file-exactly=vg.log --leak- check=full --leak-resolution=high --show-reachable=yes .libs/ freeswitch -vg Questions: * has anyone else seen this? * what is the best way I can assist troubleshooting this? I saw a patch to mod_local_stream (rev 15431) a few weeks back. Could that have anything to do with it? Rob ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] Memory leak with mod_local_stream
if you suspect 15431 to have caused this, then revert to 15430 and see if the problem exists. if you can narrow do the bug to a specific svn revision, then you greatly assist in the resolution of the issue. apart from that im not much help sorry. maybe someone else can lab it up and see if they get the same result. ( Im on a train now, so not so easy :P ) J On 24/11/2009, at 7:53 AM, Rob Forman wrote: Hey guys, Having a problem with mod_local_stream. I recently did a make current from 15334 to the latest trunk (15630). After restarting, there now appears to be a memory leak. On a test system (CentOS 5.4, 64-bit) with no calls or registrations, Freeswitch gradually consumes all of the host memory (rate of about 200K/second), then swaps out, eventually rendering the system useless. I isolated it to mod_local_stream. If I unload mod_local_stream, the memory use stops climbing. If I re-load mod_local_stream, it starts again. I would submit the logs except they aren't any besides it starting. The system is just sitting there idle. Even valgrind didn't show much (http://pastebin.freeswitch.org/11238). Maybe I'm using it wrong? I ran it: valgrind --tool=memcheck --log-file-exactly=vg.log --leak- check=full --leak-resolution=high --show-reachable=yes .libs/ freeswitch -vg Questions: * has anyone else seen this? * what is the best way I can assist troubleshooting this? I saw a patch to mod_local_stream (rev 15431) a few weeks back. Could that have anything to do with it? Rob ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] Memory leak with mod_local_stream
That rev should have fixed that memory leak, could you test mod_local_stream.c from rev 15430 (http://fisheye.freeswitch.org/browse/~raw,r=15430/FreeSWITCH/src/mod/formats/mod_local_stream/mod_local_stream.c) with your current fs version to confirm this is the cause please? Mike On Nov 23, 2009, at 4:53 PM, Rob Forman wrote: Hey guys, Having a problem with mod_local_stream. I recently did a make current from 15334 to the latest trunk (15630). After restarting, there now appears to be a memory leak. On a test system (CentOS 5.4, 64-bit) with no calls or registrations, Freeswitch gradually consumes all of the host memory (rate of about 200K/second), then swaps out, eventually rendering the system useless. I isolated it to mod_local_stream. If I unload mod_local_stream, the memory use stops climbing. If I re-load mod_local_stream, it starts again. I would submit the logs except they aren't any besides it starting. The system is just sitting there idle. Even valgrind didn't show much (http://pastebin.freeswitch.org/11238). Maybe I'm using it wrong? I ran it: valgrind --tool=memcheck --log-file-exactly=vg.log --leak- check=full --leak-resolution=high --show-reachable=yes .libs/ freeswitch -vg Questions: * has anyone else seen this? * what is the best way I can assist troubleshooting this? I saw a patch to mod_local_stream (rev 15431) a few weeks back. Could that have anything to do with it? Rob ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] Memory leak with mod_local_stream
I tried mod_local_stream.c from rev 15430, did a make clean make all make install-- but it didn't fix it so it wasn't that patch. I'll make current and try valgrind again unless someone has other ideas. Rob On Nov 23, 2009, at 4:15 PM, Michael Jerris wrote: That rev should have fixed that memory leak, could you test mod_local_stream.c from rev 15430 (http://fisheye.freeswitch.org/browse/ ~raw,r=15430/FreeSWITCH/src/mod/formats/mod_local_stream/ mod_local_stream.c) with your current fs version to confirm this is the cause please? Mike On Nov 23, 2009, at 4:53 PM, Rob Forman wrote: Hey guys, Having a problem with mod_local_stream. I recently did a make current from 15334 to the latest trunk (15630). After restarting, there now appears to be a memory leak. On a test system (CentOS 5.4, 64-bit) with no calls or registrations, Freeswitch gradually consumes all of the host memory (rate of about 200K/second), then swaps out, eventually rendering the system useless. I isolated it to mod_local_stream. If I unload mod_local_stream, the memory use stops climbing. If I re-load mod_local_stream, it starts again. I would submit the logs except they aren't any besides it starting. The system is just sitting there idle. Even valgrind didn't show much (http://pastebin.freeswitch.org/11238). Maybe I'm using it wrong? I ran it: valgrind --tool=memcheck --log-file-exactly=vg.log --leak- check=full --leak-resolution=high --show-reachable=yes .libs/ freeswitch -vg Questions: * has anyone else seen this? * what is the best way I can assist troubleshooting this? I saw a patch to mod_local_stream (rev 15431) a few weeks back. Could that have anything to do with it? Rob ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] Memory leak with mod_local_stream
Ignore this. I'm an idiot. Rob :) On Nov 23, 2009, at 4:15 PM, Michael Jerris wrote: That rev should have fixed that memory leak, could you test mod_local_stream.c from rev 15430 (http://fisheye.freeswitch.org/browse/ ~raw,r=15430/FreeSWITCH/src/mod/formats/mod_local_stream/ mod_local_stream.c) with your current fs version to confirm this is the cause please? Mike On Nov 23, 2009, at 4:53 PM, Rob Forman wrote: Hey guys, Having a problem with mod_local_stream. I recently did a make current from 15334 to the latest trunk (15630). After restarting, there now appears to be a memory leak. On a test system (CentOS 5.4, 64-bit) with no calls or registrations, Freeswitch gradually consumes all of the host memory (rate of about 200K/second), then swaps out, eventually rendering the system useless. I isolated it to mod_local_stream. If I unload mod_local_stream, the memory use stops climbing. If I re-load mod_local_stream, it starts again. I would submit the logs except they aren't any besides it starting. The system is just sitting there idle. Even valgrind didn't show much (http://pastebin.freeswitch.org/11238). Maybe I'm using it wrong? I ran it: valgrind --tool=memcheck --log-file-exactly=vg.log --leak- check=full --leak-resolution=high --show-reachable=yes .libs/ freeswitch -vg Questions: * has anyone else seen this? * what is the best way I can assist troubleshooting this? I saw a patch to mod_local_stream (rev 15431) a few weeks back. Could that have anything to do with it? Rob ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak - outbound socket
This should now be resolved in svn trunk. Mike On Sep 16, 2009, at 11:39 AM, Christian Löschenkohl wrote: as a good fs user - of course i am :-) - i made a jira on this MODAPP-336 to be precise i hope this helps to solve my problem br On 2009-09-16 17:05, Rupa Schomaker wrote: Either: 1) Provide a simple self-contained example that demonstrates the leak or 2) Run your application with FreeSWITCH under valgrind and provide the final output. To run freeswitch under valgrind: http://wiki.freeswitch.org/wiki/Reporting_Bugs#Collection_Information_With_Valgrind_.28Linux.2FUnix.29 You should not have to run with high load to capture the behavior. Try with just 5 (in series) and then stop freeswitch. 2009/9/16 Christian Löschenkohlchristian.loeschenk...@xpirio.com: hello version : 1.0.4 std. tarball - the wiki example for php outbound socket connection leaks memory without the async option - the memory used is never given back - async isn't that usefull for us - we want to query databases, set variables and so on no wait statements are possible extension name=outbound-socket condition field=destination_number expression=^55(522)$ action application=set data=ivr_path=/usr/local/freeswitch/ scripts/ivrd-demo.php/ action application=socket data=127.0.0.1:8084 full/ no async /condition /extension the script is on the site http://wiki.freeswitch.org/wiki/PHP_ESL --- what can i do? on our production server we use outbound socket connection and the 4 gig of memory are eaten up in less than a day br ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
[Freeswitch-users] memory leak - outbound socket
hello version : 1.0.4 std. tarball - the wiki example for php outbound socket connection leaks memory without the async option - the memory used is never given back - async isn't that usefull for us - we want to query databases, set variables and so on no wait statements are possible extension name=outbound-socket condition field=destination_number expression=^55(522)$ action application=set data=ivr_path=/usr/local/freeswitch/scripts/ivrd-demo.php/ action application=socket data=127.0.0.1:8084 full/ no async /condition /extension the script is on the site http://wiki.freeswitch.org/wiki/PHP_ESL --- what can i do? on our production server we use outbound socket connection and the 4 gig of memory are eaten up in less than a day br -- Ing. Christian Löschenkohl Technische Leitung, Forschung Entwicklung VoIP xpirio Telekommunikation Service GmbH Lakeside B04 9020 Klagenfurt Austria T +43 (0) 5 77 11 - 1000 F +43 (0) 5 77 11 - 1002 E christian.loeschenk...@xpirio.com ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak - outbound socket
Either: 1) Provide a simple self-contained example that demonstrates the leak or 2) Run your application with FreeSWITCH under valgrind and provide the final output. To run freeswitch under valgrind: http://wiki.freeswitch.org/wiki/Reporting_Bugs#Collection_Information_With_Valgrind_.28Linux.2FUnix.29 You should not have to run with high load to capture the behavior. Try with just 5 (in series) and then stop freeswitch. 2009/9/16 Christian Löschenkohl christian.loeschenk...@xpirio.com: hello version : 1.0.4 std. tarball - the wiki example for php outbound socket connection leaks memory without the async option - the memory used is never given back - async isn't that usefull for us - we want to query databases, set variables and so on no wait statements are possible extension name=outbound-socket condition field=destination_number expression=^55(522)$ action application=set data=ivr_path=/usr/local/freeswitch/scripts/ivrd-demo.php/ action application=socket data=127.0.0.1:8084 full/ no async /condition /extension the script is on the site http://wiki.freeswitch.org/wiki/PHP_ESL --- what can i do? on our production server we use outbound socket connection and the 4 gig of memory are eaten up in less than a day br -- Ing. Christian Löschenkohl Technische Leitung, Forschung Entwicklung VoIP xpirio Telekommunikation Service GmbH Lakeside B04 9020 Klagenfurt Austria T +43 (0) 5 77 11 - 1000 F +43 (0) 5 77 11 - 1002 E christian.loeschenk...@xpirio.com ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- -Rupa ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak - outbound socket
as a good fs user - of course i am :-) - i made a jira on this MODAPP-336 to be precise i hope this helps to solve my problem br On 2009-09-16 17:05, Rupa Schomaker wrote: Either: 1) Provide a simple self-contained example that demonstrates the leak or 2) Run your application with FreeSWITCH under valgrind and provide the final output. To run freeswitch under valgrind: http://wiki.freeswitch.org/wiki/Reporting_Bugs#Collection_Information_With_Valgrind_.28Linux.2FUnix.29 You should not have to run with high load to capture the behavior. Try with just 5 (in series) and then stop freeswitch. 2009/9/16 Christian Löschenkohlchristian.loeschenk...@xpirio.com: hello version : 1.0.4 std. tarball - the wiki example for php outbound socket connection leaks memory without the async option - the memory used is never given back - async isn't that usefull for us - we want to query databases, set variables and so on no wait statements are possible extension name=outbound-socket condition field=destination_number expression=^55(522)$ action application=set data=ivr_path=/usr/local/freeswitch/scripts/ivrd-demo.php/ action application=socket data=127.0.0.1:8084 full/ no async /condition /extension the script is on the site http://wiki.freeswitch.org/wiki/PHP_ESL --- what can i do? on our production server we use outbound socket connection and the 4 gig of memory are eaten up in less than a day br -- Ing. Christian Löschenkohl Technische Leitung, Forschung Entwicklung VoIP xpirio Telekommunikation Service GmbH Lakeside B04 9020 Klagenfurt Austria T +43 (0) 5 77 11 - 1000 F +43 (0) 5 77 11 - 1002 E christian.loeschenk...@xpirio.com ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- Ing. Christian Löschenkohl Technische Leitung, Forschung Entwicklung VoIP xpirio Telekommunikation Service GmbH Lakeside B04 9020 Klagenfurt Austria T +43 (0) 5 77 11 - 1000 F +43 (0) 5 77 11 - 1002 E christian.loeschenk...@xpirio.com ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Hello *, sched_api ... works, too. Thx again and looking forward to the next bug :) Beni. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Next Bug? Huh? :P /b On Sep 11, 2009, at 2:32 AM, Benedikt Fraunhofer wrote: Thx again and looking forward to the next bug :) ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Hello *, the latest bugfixes for luarun (pool allocation) fixed it. It's working now with luarun (friday-monday) and bgapi (monday-today). The last test with the new operator for sched_api is currently running. Thx! Beni. attachment: mem-sz_luarun_bgapi.png___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Hello Anthony, 2009/9/2 Anthony Minessale anthony.miness...@gmail.com: yes if you have a version that only has log-file you can use that. if you find me on irc and send me the credentials privately I will examine your box for you. thanks for that offer, but the box is pretty deep inside our internal network with no routing to the outside, several stepping-stones in between and all that security stuff. I finally found the right amount of load where the memory leak builds up quickly enough and was able to stop freeswitch before it started swapping. The result is available on http://ns42.ath.cx/B0GdWh/vg-2.log.bz2 (19k) Thx in advance Beni. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Worst offenders (leakers over 100K). The last one is the worst (672M) -- looks like a lua script. What are you doing in lua again? ==28624== 105,725 bytes in 1,804 blocks are still reachable in loss record 497 of 529 ==28624==at 0x4022AB8: malloc (vg_replace_malloc.c:207) ==28624==by 0x444AFCF: strdup (in /lib/tls/i686/cmov/libc-2.7.so) ==28624==by 0x50384F2: xmlrpc_strdupnull (asprintf.c:92) ==28624==by 0x503F86D: RequestRead (http.c:57) ==28624==by 0x5044413: ??? (server.c:538) ==28624==by 0x5039FAF: ??? (conn.c:37) ==28624==by 0x50486F1: ??? (thread_pthread.c:48) ==28624==by 0x42114FA: start_thread (in /lib/tls/i686/cmov/libpthread-2.7.so) ==28624==by 0x44AFE5D: clone (in /lib/tls/i686/cmov/libc-2.7.so) ==28624== ==28624== ==28624== 116,772 bytes in 3,156 blocks are definitely lost in loss record 498 of 529 ==28624==at 0x4022AB8: malloc (vg_replace_malloc.c:207) ==28624==by 0x444AFCF: strdup (in /lib/tls/i686/cmov/libc-2.7.so) ==28624==by 0x40E4EA9: switch_log_vprintf (switch_log.c:438) ==28624==by 0x40E5130: switch_log_printf (switch_log.c:308) ==28624==by 0x4088664: switch_channel_perform_hangup (switch_channel.c:1674) ==28624==by 0x64A12EA: ??? (mod_dptools.c:633) ==28624==by 0x409AA45: switch_core_session_exec (switch_core_session.c:1476) ==28624==by 0x409AF88: switch_core_session_execute_application (switch_core_session.c:1398) ==28624==by 0x409E674: switch_core_session_run (switch_core_state_machine.c:166) ==28624==by 0x409A48E: switch_core_session_thread (switch_core_session.c:1066) ==28624==by 0x4110E05: dummy_worker (thread.c:138) ==28624==by 0x42114FA: start_thread (in /lib/tls/i686/cmov/libpthread-2.7.so) ==28624== ==28624== ==28624== 119,658 (119,621 direct, 37 indirect) bytes in 3,233 blocks are definitely lost in loss record 499 of 529 ==28624==at 0x4022AB8: malloc (vg_replace_malloc.c:207) ==28624==by 0x444AFCF: strdup (in /lib/tls/i686/cmov/libc-2.7.so) ==28624==by 0x40E4EA9: switch_log_vprintf (switch_log.c:438) ==28624==by 0x40E5130: switch_log_printf (switch_log.c:308) ==28624==by 0x4088664: switch_channel_perform_hangup (switch_channel.c:1674) ==28624==by 0x50B6790: sofia_event_callback (sofia.c:3863) ==28624==by 0x5146787: nua_application_event (nua_stack.c:393) ==28624==by 0x519DB28: su_base_port_execute_msgs (su_base_port.c:280) ==28624==by 0x519D8CF: su_base_port_getmsgs (su_base_port.c:202) ==28624==by 0x519DD75: su_base_port_step (su_base_port.c:454) ==28624==by 0x5190968: su_port_step (su_port.h:340) ==28624==by 0x5190938: su_root_step (su_root.c:858) ==28624== ==28624== ==28624== 124,209 bytes in 3,357 blocks are still reachable in loss record 500 of 529 ==28624==at 0x4022AB8: malloc (vg_replace_malloc.c:207) ==28624==by 0x444AFCF: strdup (in /lib/tls/i686/cmov/libc-2.7.so) ==28624==by 0x40E4EA9: switch_log_vprintf (switch_log.c:438) ==28624==by 0x40E5130: switch_log_printf (switch_log.c:308) ==28624==by 0x409A5EC: switch_core_session_thread (switch_core_session.c:1086) ==28624==by 0x4110E05: dummy_worker (thread.c:138) ==28624==by 0x42114FA: start_thread (in /lib/tls/i686/cmov/libpthread-2.7.so) ==28624==by 0x44AFE5D: clone (in /lib/tls/i686/cmov/libc-2.7.so) ==28624== ==28624== ==28624== 124,290 bytes in 4,143 blocks are still reachable in loss record 501 of 529 ==28624==at 0x4022AB8: malloc (vg_replace_malloc.c:207) ==28624==by 0x443B957: vasprintf (in /lib/tls/i686/cmov/libc-2.7.so) ==28624==by 0x5038532: xmlrpc_vasprintf (asprintf.c:61) ==28624==by 0x5038581: xmlrpc_asprintf (asprintf.c:81) ==28624==by 0x503B881: DateToString (date.c:43) ==28624==by 0x5036D09: handler_hook (mod_xml_rpc.c:733) ==28624==by 0x504456F: ??? (server.c:515) ==28624==by 0x5039FAF: ??? (conn.c:37) ==28624==by 0x50486F1: ??? (thread_pthread.c:48) ==28624==by 0x42114FA: start_thread (in /lib/tls/i686/cmov/libpthread-2.7.so) ==28624==by 0x44AFE5D: clone (in /lib/tls/i686/cmov/libc-2.7.so) ==28624== ==28624== ==28624== 137,085 bytes in 3,705 blocks are still reachable in loss record 502 of 529 ==28624==at 0x4022AB8: malloc (vg_replace_malloc.c:207) ==28624==by 0x444AFCF: strdup (in /lib/tls/i686/cmov/libc-2.7.so) ==28624==by 0x40E4EA9: switch_log_vprintf (switch_log.c:438) ==28624==by 0x40E5130: switch_log_printf (switch_log.c:308) ==28624==by 0x409921F: switch_core_session_perform_destroy (switch_core_session.c:947) ==28624==by 0x409A60D: switch_core_session_thread (switch_core_session.c:1088) ==28624==by 0x4110E05: dummy_worker (thread.c:138) ==28624==by 0x42114FA: start_thread (in /lib/tls/i686/cmov/libpthread-2.7.so) ==28624==by 0x44AFE5D: clone (in /lib/tls/i686/cmov/libc-2.7.so) ==28624== ==28624== ==28624== 145,589 bytes in 1,837 blocks are possibly lost in loss record 503 of 529 ==28624==at 0x4022AB8: malloc (vg_replace_malloc.c:207) ==28624==
Re: [Freeswitch-users] memory leak
2009/9/4 Rupa Schomaker r...@rupa.com: Worst offenders (leakers over 100K). The last one is the worst (672M) -- looks like a lua script. What are you doing in lua again? i feel kinda dumb to double post, but here it is again :) the setup is the same as in http://jira.freeswitch.org/browse/MODSOFIA-22 one is - local reason = session:getVariable(originate_disposition); session:setAutoHangup(false); if(reason) then if(reason == NO_ANSWER) then -- nothing end if(reason == USER_BUSY) then -- nothing end end freeswitch.consoleLog(... -- anotherone is local sess = nil; if(argv[1]) then sess=argv[1]; end freeswitch.consoleLog(... api = freeswitch.API(); local res = api:execute(sched_api ... freeswitch.consoleLog(... and the scheduled script does --- function log(msg) freeswitch.consoleLog(notice, c2c-hangup-timeout.lua: .. msg .. \n); end local sess = argv[1]; if(sess) then freeswitch.consoleLog(INFO, hangup-timeout.lua for uuid .. sess .. \n); api = freeswitch.API(); local stillValid = api:execute(uuid_getvar, sess .. Dummy-DoesChannelExists); if(stillValid:sub(1,4) == -ERR) then log(session uuid .. sess .. disappeared (nothing bad)); else -- this is important!!! Otherwise the aleg get's just hung up! api:execute(uuid_media, sess); api:execute(uuid_transfer, sess .. -both timeout); end else -- /if(sess) log(called with nil session?); end -- /if(sess) --- at least there's no fancy db-connection-thingi which could make debugging harder :) Cheers Beni. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Doesn't that look like a pool that isn't being destroyed? On Fri, Sep 4, 2009 at 9:10 AM, Rupa Schomakerr...@rupa.com wrote: Worst offenders (leakers over 100K). The last one is the worst (672M) -- looks like a lua script. What are you doing in lua again? ==28624== 672,268,288 bytes in 82,064 blocks are still reachable in loss record 529 of 529 ==28624== at 0x4022AB8: malloc (vg_replace_malloc.c:207) ==28624== by 0x410BB2F: apr_pool_create_ex (apr_pools.c:300) ==28624== by 0x476: apr_thread_create (thread.c:171) ==28624== by 0x4080878: switch_thread_create (switch_apr.c:631) ==28624== by 0x6C278E9: lua_thread (mod_lua.cpp:372) ==28624== by 0x6C27948: ??? (mod_lua.cpp:407) ==28624== by 0x40AADFC: switch_api_execute (switch_loadable_module.c:1567) ==28624== by 0x583A7FC: ??? (mod_commands.c:2426) ==28624== by 0x40A8881: switch_scheduler_execute (switch_scheduler.c:61) ==28624== by 0x40A8DE0: task_thread_loop (switch_scheduler.c:127) ==28624== by 0x40A8EA3: switch_scheduler_task_thread (switch_scheduler.c:168) ==28624== by 0x4110E05: dummy_worker (thread.c:138) ==28624== ==28624== LEAK SUMMARY: ==28624== definitely lost: 63,113,740 bytes in 1,690,880 blocks. ==28624== indirectly lost: 35,632 bytes in 491 blocks. ==28624== possibly lost: 645,758 bytes in 9,150 blocks. ==28624== still reachable: 681,849,684 bytes in 113,077 blocks. ==28624== suppressed: 0 bytes in 0 blocks. -- -Rupa ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
personally i would blame xmlrpc (which is no xml :) for it. Just my 2cent Beni. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
There are other smaller leakers. xmlrpc is leaking, but the leaks are very small compared to the lua leak. Same with spidermonkey_curl - it is leaking but not too terribly much. I'll hop on #freeswitch in a bit and see if anyone has an idea. On Fri, Sep 4, 2009 at 9:35 AM, Benedikt Fraunhoferfraunhofer.lists.freeswitch-...@traced.net wrote: personally i would blame xmlrpc (which is no xml :) for it. Just my 2cent Beni. http://www.freeswitch.org -- -Rupa ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
that looks to me like luarun being called on a script that never terminates. could your script be ending up caught in an endless loop or blocking on something? On Fri, Sep 4, 2009 at 9:42 AM, Rupa Schomaker r...@rupa.com wrote: There are other smaller leakers. xmlrpc is leaking, but the leaks are very small compared to the lua leak. Same with spidermonkey_curl - it is leaking but not too terribly much. I'll hop on #freeswitch in a bit and see if anyone has an idea. On Fri, Sep 4, 2009 at 9:35 AM, Benedikt Fraunhoferfraunhofer.lists.freeswitch-...@traced.net wrote: personally i would blame xmlrpc (which is no xml :) for it. Just my 2cent Beni. http://www.freeswitch.org -- -Rupa ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_miness...@hotmail.com msn%3aanthony_miness...@hotmail.com GTALK/JABBER/PAYPAL:anthony.miness...@gmail.compaypal%3aanthony.miness...@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:8...@conference.freeswitch.org sip%3a...@conference.freeswitch.org iax:gu...@conference.freeswitch.org/888 googletalk:conf+...@conference.freeswitch.orggoogletalk%3aconf%2b...@conference.freeswitch.org pstn:213-799-1400 ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Hello *, 2009/8/31 Rupa Schomaker r...@rupa.com: Isn't there a known issue with lua+sql leaking memory on some platforms? just lua, no sql in use :) On Mon, Aug 31, 2009 at 8:32 AM, Brian Westbr...@freeswitch.org wrote: Use valgrind. i tried that... in the beginning valgrind segfaultet several times with some error message that it's unable to monitor so many threads. vg_alloc_ThreadState: no free slots available Increase VG_N_THREADS, rebuild and try again. valgrind: the 'impossible' happened: VG_N_THREADS is too low it happeneded with 400 concurrent calls pretty soon, with 100 it took a while, but at least it consumed half a gig of ram. Can someone interpret the file it generated so far? i put it at http://ns42.ath.cx/B0GdWh/vg.log.bz2 (5k compressed, 452k uncompressed) or is there just not enough information contained? I'll give it another shot with 50 concurrent calls but i guess this will take ages... Thx in advance Beni. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
What are you doing in these lua scripts? Because there are a few things you can do in the lua script itself that will cause you to leak like crazy due to improper use. /b On Sep 2, 2009, at 7:59 AM, Benedikt Fraunhofer wrote: Hello *, 2009/8/31 Rupa Schomaker r...@rupa.com: Isn't there a known issue with lua+sql leaking memory on some platforms? just lua, no sql in use :) ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Hello Brian, 2009/9/2 Brian West br...@freeswitch.org: What are you doing in these lua scripts? Because there are a few things you can do in the lua script itself that will cause you to leak like crazy due to improper use. /b the setup is the same as in http://jira.freeswitch.org/browse/MODSOFIA-22 one is - local reason = session:getVariable(originate_disposition); session:setAutoHangup(false); if(reason) then if(reason == NO_ANSWER) then -- nothing end if(reason == USER_BUSY) then -- nothing end end freeswitch.consoleLog(... -- anotherone is local sess = nil; if(argv[1]) then sess=argv[1]; end freeswitch.consoleLog(... api = freeswitch.API(); local res = api:execute(sched_api ... freeswitch.consoleLog(... and the scheduled script does --- function log(msg) freeswitch.consoleLog(notice, c2c-hangup-timeout.lua: .. msg .. \n); end local sess = argv[1]; if(sess) then freeswitch.consoleLog(INFO, hangup-timeout.lua for uuid .. sess .. \n); api = freeswitch.API(); local stillValid = api:execute(uuid_getvar, sess .. Dummy-DoesChannelExists); if(stillValid:sub(1,4) == -ERR) then log(session uuid .. sess .. disappeared (nothing bad)); else -- this is important!!! Otherwise the aleg get's just hung up! api:execute(uuid_media, sess); api:execute(uuid_transfer, sess .. -both timeout); end else -- /if(sess) log(called with nil session?); end -- /if(sess) --- i don't have to do some kind of free() on all local variables, do i? Thx. Beni. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
You have to reduce the load when running valgrind. On Wed, Sep 2, 2009 at 7:59 AM, Benedikt Fraunhofer fraunhofer.lists.freeswitch-...@traced.net wrote: Hello *, 2009/8/31 Rupa Schomaker r...@rupa.com: Isn't there a known issue with lua+sql leaking memory on some platforms? just lua, no sql in use :) On Mon, Aug 31, 2009 at 8:32 AM, Brian Westbr...@freeswitch.org wrote: Use valgrind. i tried that... in the beginning valgrind segfaultet several times with some error message that it's unable to monitor so many threads. vg_alloc_ThreadState: no free slots available Increase VG_N_THREADS, rebuild and try again. valgrind: the 'impossible' happened: VG_N_THREADS is too low it happeneded with 400 concurrent calls pretty soon, with 100 it took a while, but at least it consumed half a gig of ram. Can someone interpret the file it generated so far? i put it at http://ns42.ath.cx/B0GdWh/vg.log.bz2 (5k compressed, 452k uncompressed) or is there just not enough information contained? I'll give it another shot with 50 concurrent calls but i guess this will take ages... Thx in advance Beni. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_miness...@hotmail.com msn%3aanthony_miness...@hotmail.com GTALK/JABBER/PAYPAL:anthony.miness...@gmail.compaypal%3aanthony.miness...@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:8...@conference.freeswitch.org sip%3a...@conference.freeswitch.org iax:gu...@conference.freeswitch.org/888 googletalk:conf+...@conference.freeswitch.orggoogletalk%3aconf%2b...@conference.freeswitch.org pstn:213-799-1400 ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Hello Anthony, 2009/9/2 Anthony Minessale anthony.miness...@gmail.com: run it slower and make sure it shuts down clean. i already reduced load 37% but that didnt help, now i'm down to 25% and it's running. valgrind --tool=memcheck --log-file-exactly=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg that's the line in the wiki? I used that but log-file-exactly is not available in my valgrind-3.3.0-Debian, just log-file. Does this make any difference? Also it wont shut down cleanly but oom-coredump (some assertion failed) and if i stop the loadgen after some time and try to 'fsctl shutdown' it wont cleanly bring it down, too. (see first post, last line printed is switch_core_memory.c:567 Stopping memory pool queue. ) Thx. Beni. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
i mean valgrind is very intensive so you must run very slow 1-5cps yes if you have a version that only has log-file you can use that. if you find me on irc and send me the credentials privately I will examine your box for you. On Wed, Sep 2, 2009 at 10:26 AM, Benedikt Fraunhofer fraunhofer.lists.freeswitch-...@traced.net wrote: Hello Anthony, 2009/9/2 Anthony Minessale anthony.miness...@gmail.com: run it slower and make sure it shuts down clean. i already reduced load 37% but that didnt help, now i'm down to 25% and it's running. valgrind --tool=memcheck --log-file-exactly=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg that's the line in the wiki? I used that but log-file-exactly is not available in my valgrind-3.3.0-Debian, just log-file. Does this make any difference? Also it wont shut down cleanly but oom-coredump (some assertion failed) and if i stop the loadgen after some time and try to 'fsctl shutdown' it wont cleanly bring it down, too. (see first post, last line printed is switch_core_memory.c:567 Stopping memory pool queue. ) Thx. Beni. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_miness...@hotmail.com msn%3aanthony_miness...@hotmail.com GTALK/JABBER/PAYPAL:anthony.miness...@gmail.compaypal%3aanthony.miness...@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:8...@conference.freeswitch.org sip%3a...@conference.freeswitch.org iax:gu...@conference.freeswitch.org/888 googletalk:conf+...@conference.freeswitch.orggoogletalk%3aconf%2b...@conference.freeswitch.org pstn:213-799-1400 ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Hello *, still no luck. Removing the default dialplan with all those hash-inserts made the memory-usage-curve flatter but it oom-coredumped stating: freeswitch: src/switch_core_memory.c:443: switch_core_perform_alloc: Assertion `ptr != ((void *)0)' failed. after eating up 3.2 gig ram+swap. Any chance or idea to track that down or any hint where to look at? The other two freeswitch-machines in this scenario play well, they're dumb load-sinks. what makes this machine different is on one hand the lua-scripts that are run on it and on the other hand it's the only machine who gets it's commands via xml_rpc. Cheers Beni. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Use valgrind. /b On Aug 31, 2009, at 7:22 AM, Benedikt Fraunhofer wrote: The other two freeswitch-machines in this scenario play well, they're dumb load-sinks. what makes this machine different is on one hand the lua-scripts that are run on it and on the other hand it's the only machine who gets it's commands via xml_rpc. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Isn't there a known issue with lua+sql leaking memory on some platforms? On Mon, Aug 31, 2009 at 8:32 AM, Brian Westbr...@freeswitch.org wrote: Use valgrind. /b On Aug 31, 2009, at 7:22 AM, Benedikt Fraunhofer wrote: The other two freeswitch-machines in this scenario play well, they're dumb load-sinks. what makes this machine different is on one hand the lua-scripts that are run on it and on the other hand it's the only machine who gets it's commands via xml_rpc. http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- -Rupa ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Cool, feel free to report these bugs in Jira. If they are indeed FreeSWITCH bugs. On Fri, Aug 28, 2009 at 12:10 AM, Jason White ja...@jasonjgw.net wrote: Jay Binks jaybi...@gmail.com wrote: Reason I ask ... I personally only have a preference for debian, but others may have policy mandated Os's For their companies, and it would be great to have some info about this. The only problem I've had with FreeSWITCH under Debian Squeeze and Sid involves TLS-related segmentation faults that appear to be related to something in the version of libssl supplied with Debian. The same problem can't be reproduced on Fedora, for example, but it does occur under Debian Lenny as well as Debian Squeeze and Sid (i.e., testing and unstable, respectively). besides this, all of the issues that I have encountered turned out to be (usually short-lived) bugs in FreeSWITCH or one of the libraries included in the source tree - they're mostly FreeSWITCH issues. I should point out that the FreeSWITCH developers are very good at avoiding the introduction of bugs into their code and that known bugs get fixed. It appears to be an unchanging fact about programming that with a large and complex project, even given highly knowledgeable, experienced, committed and talented developers (as we have with FreeSWITCH), sometimes, bugs do slip through. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
[Freeswitch-users] memory leak
Hello *, a memory leak showed up in our loadtests. It's (still) the same setup as in the http://jira.freeswitch.org/browse/MODSOFIA-22 bugfix. One thing I'd like to add is that fsctl shutdown restart was unable to shutdown freeswitch. The last line printed is switch_core_memory.c:567 Stopping memory pool queue. attached file is a the collected and graphed output of some ps waux command. sz should be in physical pages (2k?). I rerun the test and this time it coredumped trying to malloc() space for some playback. Anything else you need (full backtraces?) to dig into it? Cheers Beni. attachment: mem-sz.png___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
600k is not a leak? FS can use as much as a gig of ram or more depending on what you are doing. you may want to install a fresh copy of FS, removing all your old files etc and make sure they build clean. We also have not had much luck running on ubuntu which is more of a desktop centric OS. I recommend you try your application on 64 bit CentOS which is the platform all of our paid customers use. On Thu, Aug 27, 2009 at 8:15 AM, Benedikt Fraunhofer fraunhofer.lists.freeswitch-...@traced.net wrote: Hello *, a memory leak showed up in our loadtests. It's (still) the same setup as in the http://jira.freeswitch.org/browse/MODSOFIA-22 bugfix. One thing I'd like to add is that fsctl shutdown restart was unable to shutdown freeswitch. The last line printed is switch_core_memory.c:567 Stopping memory pool queue. attached file is a the collected and graphed output of some ps waux command. sz should be in physical pages (2k?). I rerun the test and this time it coredumped trying to malloc() space for some playback. Anything else you need (full backtraces?) to dig into it? Cheers Beni. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_miness...@hotmail.com msn%3aanthony_miness...@hotmail.com GTALK/JABBER/PAYPAL:anthony.miness...@gmail.compaypal%3aanthony.miness...@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:8...@conference.freeswitch.org sip%3a...@conference.freeswitch.org iax:gu...@conference.freeswitch.org/888 googletalk:conf+...@conference.freeswitch.orggoogletalk%3aconf%2b...@conference.freeswitch.org pstn:213-799-1400 ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Hello Anthony, 2009/8/27 Anthony Minessale anthony.miness...@gmail.com: 600k is not a leak? FS can use as much as a gig of ram or more depending on what you are doing. it's 600*1024 * 4* 1024 /1024/1024 = ~ 2.4 Gig that's what i tried to express with the physical pages in my first post. you may want to install a fresh copy of FS, removing all your old files etc and make sure they build clean. i make current to get the SOFIA-22 bugfix. We also have not had much luck running on ubuntu which is more of a desktop centric OS. really? even the server edition? (not the netbook remix :) Is 64bit so grave different if i don't need to address more than 4 Gig of ram per process? I recommend you try your application on 64 bit CentOS which is the platform all of our paid customers use. hmm. our operations-people wont be happy with that. But as i carefully followed the other mailing-list-threads it came to my mind that i could be the culprit: somehow the original example.xml came back into the directory tree and included a lot of things, most notably the hash-inserts for last-dailed-numbers and stuff. I stripped it down to what it was before and gave it another try. thx again and (hopefully) sorry Beni. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Anthony can you ( or anyone else alao ). Please elaborate on what makes centos 5.3 o much better for Freeswitch. Is there some specific library vesiion on centos that makes a massive difference ? Reason I ask ... I personally only have a preference for debian, but others may have policy mandated Os's For their companies, and it would be great to have some info about this. I'd imagine this actually boils down to a requirement for libfoo version X ... And if we ran those library versions on other OS's we'd be fine ? Jay I recommend you try your application on 64 bit CentOS which is the platform all of our paid customers use. hmm. our operations-people wont be happy with that. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Mostly based on that is what we developed it with and continue to use and have had the best luck and least problems with. I am sure you are right about it being certain version of the core toolchain I will guess it's the combo of libc and kernel and the fact that CentOS has a purist attitude towards patching such code. Some people use stable debian with no issues, other have reported problems resolved by switching to CentOS so presumably you could roll your own debian that was similar but we don't have the resources to try to find out what makes what happen etc. If you can configure debian to work, it's fine, if we get some inexplicable problem on debian we will start to wonder though. it's happened more than once with people doing nothing other than changing OS =D On Thu, Aug 27, 2009 at 4:55 PM, Jay Binks jaybi...@gmail.com wrote: Anthony can you ( or anyone else alao ). Please elaborate on what makes centos 5.3 o much better for Freeswitch. Is there some specific library vesiion on centos that makes a massive difference ? Reason I ask ... I personally only have a preference for debian, but others may have policy mandated Os's For their companies, and it would be great to have some info about this. I'd imagine this actually boils down to a requirement for libfoo version X ... And if we ran those library versions on other OS's we'd be fine ? Jay I recommend you try your application on 64 bit CentOS which is the platform all of our paid customers use. hmm. our operations-people wont be happy with that. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_miness...@hotmail.com msn%3aanthony_miness...@hotmail.com GTALK/JABBER/PAYPAL:anthony.miness...@gmail.compaypal%3aanthony.miness...@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:8...@conference.freeswitch.org sip%3a...@conference.freeswitch.org iax:gu...@conference.freeswitch.org/888 googletalk:conf+...@conference.freeswitch.orggoogletalk%3aconf%2b...@conference.freeswitch.org pstn:213-799-1400 ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak
Jay Binks jaybi...@gmail.com wrote: Reason I ask ... I personally only have a preference for debian, but others may have policy mandated Os's For their companies, and it would be great to have some info about this. The only problem I've had with FreeSWITCH under Debian Squeeze and Sid involves TLS-related segmentation faults that appear to be related to something in the version of libssl supplied with Debian. The same problem can't be reproduced on Fedora, for example, but it does occur under Debian Lenny as well as Debian Squeeze and Sid (i.e., testing and unstable, respectively). besides this, all of the issues that I have encountered turned out to be (usually short-lived) bugs in FreeSWITCH or one of the libraries included in the source tree - they're mostly FreeSWITCH issues. I should point out that the FreeSWITCH developers are very good at avoiding the introduction of bugs into their code and that known bugs get fixed. It appears to be an unchanging fact about programming that with a large and complex project, even given highly knowledgeable, experienced, committed and talented developers (as we have with FreeSWITCH), sometimes, bugs do slip through. ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
[Freeswitch-users] memory leak with switch_core_hash_insert
Hi, I am getting some strange vg malloc error message in switch_core_hash_insert. Does anyone know what is wrong with these few lines? Am I missing something? switch_core_hash_init(hash,pool); param_name =switch_core_sprintf(pool,%s, key); param_value =switch_core_sprintf(pool,%s, value); switch_core_hash_insert(hash,param_name,param_value); ==5459== ==5459== 8,973 (2,344 direct, 6,629 indirect) bytes in 10 blocks are definitely lost in loss record 62 of 89 ==5459==at 0x4A05809: malloc (vg_replace_malloc.c:149) ==5459==by 0x4CEBB68: sqlite3MallocX (sqliteInt.h:278) ==5459==by 0x4CE1C7E: rehash (hash.c:227) ==5459==by 0x4CE206F: sqlite3HashInsert (hash.c:386) ==5459==by 0x4C70EA4: switch_core_hash_insert (switch_core_hash.c:67) ==5459==by 0x7CEAD07: ??? (mod_specialivr.c:343) ==5459==by 0x7CEBF87: ??? (mod_specialivr.c:784) ==5459==by 0x7CEDBF0: ??? (mod_specialivr.c:1778) ==5459==by 0x4C790AC: switch_core_session_run (switch_core_state_machine.c:109) ==5459==by 0x4C751FF: switch_core_session_thread (switch_core_session.c:1066) ==5459==by 0x4FBC306: start_thread (in /lib64/libpthread-2.5.so) ==5459==by 0x35018D1DEC: clone (in /lib64/libc-2.5.so) ==5459== ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak with switch_core_hash_insert
Hi, You're missing: switch_core_hash_destroy(hash) Mathieu Rene Avant-Garde Solutions Inc Office: + 1 (514) 664-1044 x100 Cell: +1 (514) 664-1044 x200 mr...@avgs.ca On 18-Aug-09, at 11:04 AM, Juan Backson wrote: Hi, I am getting some strange vg malloc error message in switch_core_hash_insert. Does anyone know what is wrong with these few lines? Am I missing something? switch_core_hash_init(hash,pool); param_name =switch_core_sprintf(pool,%s, key); param_value =switch_core_sprintf(pool,%s, value); switch_core_hash_insert(hash,param_name,param_value); ==5459== ==5459== 8,973 (2,344 direct, 6,629 indirect) bytes in 10 blocks are definitely lost in loss record 62 of 89 ==5459==at 0x4A05809: malloc (vg_replace_malloc.c:149) ==5459==by 0x4CEBB68: sqlite3MallocX (sqliteInt.h:278) ==5459==by 0x4CE1C7E: rehash (hash.c:227) ==5459==by 0x4CE206F: sqlite3HashInsert (hash.c:386) ==5459==by 0x4C70EA4: switch_core_hash_insert (switch_core_hash.c:67) ==5459==by 0x7CEAD07: ??? (mod_specialivr.c:343) ==5459==by 0x7CEBF87: ??? (mod_specialivr.c:784) ==5459==by 0x7CEDBF0: ??? (mod_specialivr.c:1778) ==5459==by 0x4C790AC: switch_core_session_run (switch_core_state_machine.c:109) ==5459==by 0x4C751FF: switch_core_session_thread (switch_core_session.c:1066) ==5459==by 0x4FBC306: start_thread (in /lib64/libpthread-2.5.so) ==5459==by 0x35018D1DEC: clone (in /lib64/libc-2.5.so) ==5459== ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak with switch_core_hash_insert
try running freeswitch with -vg command line arg. Mike On Aug 18, 2009, at 11:04 AM, Juan Backson wrote: Hi, I am getting some strange vg malloc error message in switch_core_hash_insert. Does anyone know what is wrong with these few lines? Am I missing something? switch_core_hash_init(hash,pool); param_name =switch_core_sprintf(pool,%s, key); param_value =switch_core_sprintf(pool,%s, value); switch_core_hash_insert(hash,param_name,param_value); ==5459== ==5459== 8,973 (2,344 direct, 6,629 indirect) bytes in 10 blocks are definitely lost in loss record 62 of 89 ==5459==at 0x4A05809: malloc (vg_replace_malloc.c:149) ==5459==by 0x4CEBB68: sqlite3MallocX (sqliteInt.h:278) ==5459==by 0x4CE1C7E: rehash (hash.c:227) ==5459==by 0x4CE206F: sqlite3HashInsert (hash.c:386) ==5459==by 0x4C70EA4: switch_core_hash_insert (switch_core_hash.c:67) ==5459==by 0x7CEAD07: ??? (mod_specialivr.c:343) ==5459==by 0x7CEBF87: ??? (mod_specialivr.c:784) ==5459==by 0x7CEDBF0: ??? (mod_specialivr.c:1778) ==5459==by 0x4C790AC: switch_core_session_run (switch_core_state_machine.c:109) ==5459==by 0x4C751FF: switch_core_session_thread (switch_core_session.c:1066) ==5459==by 0x4FBC306: start_thread (in /lib64/libpthread-2.5.so) ==5459==by 0x35018D1DEC: clone (in /lib64/libc-2.5.so) ==5459== ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Re: [Freeswitch-users] memory leak with switch_core_hash_insert
Just a heads up: this is a rather technical subject. I highly recommend that you use freeswitch-dev list for this kind of thing. -MC On Tue, Aug 18, 2009 at 8:04 AM, Juan Backson juanback...@gmail.com wrote: Hi, I am getting some strange vg malloc error message in switch_core_hash_insert. Does anyone know what is wrong with these few lines? Am I missing something? switch_core_hash_init(hash,pool); param_name =switch_core_sprintf(pool,%s, key); param_value =switch_core_sprintf(pool,%s, value); switch_core_hash_insert(hash,param_name,param_value); ==5459== ==5459== 8,973 (2,344 direct, 6,629 indirect) bytes in 10 blocks are definitely lost in loss record 62 of 89 ==5459==at 0x4A05809: malloc (vg_replace_malloc.c:149) ==5459==by 0x4CEBB68: sqlite3MallocX (sqliteInt.h:278) ==5459==by 0x4CE1C7E: rehash (hash.c:227) ==5459==by 0x4CE206F: sqlite3HashInsert (hash.c:386) ==5459==by 0x4C70EA4: switch_core_hash_insert (switch_core_hash.c:67) ==5459==by 0x7CEAD07: ??? (mod_specialivr.c:343) ==5459==by 0x7CEBF87: ??? (mod_specialivr.c:784) ==5459==by 0x7CEDBF0: ??? (mod_specialivr.c:1778) ==5459==by 0x4C790AC: switch_core_session_run (switch_core_state_machine.c:109) ==5459==by 0x4C751FF: switch_core_session_thread (switch_core_session.c:1066) ==5459==by 0x4FBC306: start_thread (in /lib64/libpthread-2.5.so) ==5459==by 0x35018D1DEC: clone (in /lib64/libc-2.5.so) ==5459== ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org ___ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
