Bogdan, Are these memlogs useful? Please let me know if I can provide any other info.
Thanks again. On Sat, Nov 13, 2010 at 4:20 PM, thrillerbee <thriller...@gmail.com> wrote: > Bogdan, > > Since I made those changes, it doesn't core dump anymore - it just runs out > of memory and stops processing packets. It happened on both of my proxies - > the mem dump can be downloaded here: > http://www.starviewconnect.com/tmp/core1_mem_dump_20101112.gz > http://www.starviewconnect.com/tmp/core2_mem_dump_20101112.gz > > Thanks. > > On Thu, Nov 11, 2010 at 12:43 PM, thrillerbee <thriller...@gmail.com>wrote: > >> Bogdan, >> >> I had already increased the PKG_MEM_POOL_SIZE by *10. I'm making the >> changes suggested in the link you provided to try to narrow down the memory >> issue. I'll send over as soon as I have another crash with mem info. >> >> Thanks. >> >> >> On Thu, Nov 11, 2010 at 5:10 AM, Bogdan-Andrei Iancu < >> bog...@voice-system.ro> wrote: >> >>> This last crash had the same bt as the one from previous email ? the prev >>> fix I made reports this: >>> >>> ERROR:db_flatstore:flat_db_insert: uninitialized connection >>> and does not crash, so this time the crash may be in a different place. >>> >>> Now, it seams after all that the root of your problem is the mem >>> exhaustion. To see what causes this (if a mem leak or simply not enough >>> mem), there is a doc - http://www.opensips.org/Resources/DocsTsMem (it >>> is for shm, but the same for pkg). >>> >>> Maybe, first you should simply try to increase the pkg mem ( in config.h >>> you have PKG_MEM_POOL_SIZE that you can increase) - you NEED to recompile >>> and reinstall after that. >>> >>> If more pkg mem does not solve the problem, I can help you with using the >>> memory debugger to see what is going on there. >>> >>> Regards, >>> Bogdan >>> >>> >>> thrillerbee wrote: >>> >>>> My other proxy crashed as well with these ERRORs in the syslog: >>>> >>>> Nov 10 22:01:02 core2 /usr/local/sbin/opensips[22959]: >>>> ERROR:db_flatstore:get_name: pkg memory allocation failure >>>> Nov 10 22:01:02 core2 /usr/local/sbin/opensips[22959]: >>>> ERROR:db_flatstore:flat_reopen_connection: failed to get_name >>>> Nov 10 22:01:02 core2 /usr/local/sbin/opensips[22959]: >>>> ERROR:db_flatstore:flat_db_insert: uninitialized connection >>>> Nov 10 22:01:02 core2 /usr/local/sbin/opensips[22959]: >>>> ERROR:db_flatstore:flat_db_insert: uninitialized connection >>>> ... >>>> Nov 10 22:01:21 core2 /usr/local/sbin/opensips[22959]: >>>> ERROR:db_flatstore:flat_db_insert: uninitialized connection >>>> Nov 10 22:01:22 core2 /usr/local/sbin/opensips[22959]: >>>> ERROR:db_flatstore:flat_db_insert: uninitialized connection >>>> Nov 10 22:01:22 core2 /usr/local/sbin/opensips[22959]: >>>> ERROR:db_flatstore:new_flat_id: no pkg memory left >>>> Nov 10 22:01:22 core2 kernel: [4297088.404734] opensips[22959]: segfault >>>> at 10 ip 7f3db577e21f sp 7fffa260d640 error 4 in >>>> db_flatstore.so[7f3db577b000+5000] >>>> >>>> On Wed, Nov 10, 2010 at 10:19 AM, thrillerbee >>>> <thriller...@gmail.com<mailto: >>>> thriller...@gmail.com>> wrote: >>>> >>>> Bogdan, >>>> >>>> Well, I spoke too soon - it's not just an issue with the >>>> opensipsctl fifo calls - looks more like a memory leak. It >>>> crashed again today, but I did get some errors in the syslog this >>>> time right before the crash: >>>> Nov 10 15:42:32 core1 /usr/local/sbin/opensips[27044]: >>>> ERROR:db_flatstore:new_flat_id: no pkg memory left >>>> Nov 10 15:42:32 core1 kernel: [5508366.582447] opensips[27044]: >>>> segfault at 10 ip 7fa7ff74c21f sp 7fffdc101700 error 4 in >>>> db_flatstore.so[7fa7ff749000+5000] >>>> To be thorough, I've attached the backtrace & output from print >>>> commands (although they're the same as before). >>>> >>>> To answer your question, yes - I do use the flat_rotate MI command. >>>> >>>> Thanks again. >>>> >>>> On Wed, Nov 10, 2010 at 4:04 AM, Bogdan-Andrei Iancu >>>> <bog...@voice-system.ro <mailto:bog...@voice-system.ro>> wrote: >>>> >>>> Hi, >>>> >>>> opensipsctl takes care that each command takes a separate fifo >>>> reply, so here it should be no problem. But the problem may be >>>> when comes with sending multiple commands (via FIFO) in the >>>> same time - this translates into parallel writes to the same >>>> file and depends on the atomicity of the write op. >>>> >>>> But in the worst case, a mixture at the FIFO level may lead to >>>> bogus command and not in any kind of crash....Do you use the >>>> "flat_rotate" MI command ? >>>> >>>> Regards, >>>> Bogdan >>>> >>>> thrillerbee wrote: >>>> >>>> Bogdan, >>>> >>>> It seems the issue is with 'opensipsctl fifo' - it's very >>>> sensitive to simultaneous calls. Basically, I've combined >>>> all my scripts to prevent 'opensipsctl fifo' from being >>>> called too frequently and that seems (so far) to have >>>> mitigated the issue. Is there anything one should know >>>> about how (not) to use /opensipsctl/? >>>> >>>> Thanks. >>>> >>>> On Mon, Nov 8, 2010 at 6:07 AM, Bogdan-Andrei Iancu >>>> <bog...@voice-system.ro <mailto:bog...@voice-system.ro> >>>> <mailto:bog...@voice-system.ro >>>> <mailto:bog...@voice-system.ro>>> wrote: >>>> >>>> Hi, >>>> >>>> strange if you do not have any errors :(.... >>>> >>>> I just made a fix on both trunk and 1.6 to extend some >>>> checks in >>>> flatstore and prevent crashing (even if the DB op will >>>> not be >>>> executed). >>>> >>>> Could you update from SVN and see if stops crashing ? >>>> >>>> Regards, >>>> Bogdan >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> _______________________________________________ >>>> Users mailing list >>>> Users@lists.opensips.org >>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users >>>> >>>> >>> >>> >>> -- >>> Bogdan-Andrei Iancu >>> OpenSIPS Bootcamp >>> 15 - 19 November 2010, Edison, New Jersey, USA >>> www.voice-system.ro >>> >>> >>> _______________________________________________ >>> Users mailing list >>> Users@lists.opensips.org >>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users >>> >> >> >
_______________________________________________ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users
