Re: [OpenSIPS-Users] memory errors, too many transactions
Hi Anca, I had the same issue last night. My script is as follow: * if (subst_uri('/^sip:([0-9]+)@(.*)$/$json(cs/fwd_domainport)/i')){ if ($var(debug_level) 3) { xlog([leg2]LOG: We got: BillingID=$avp(s:billing_id) , Provider RateID=$avp(s:prov_raterec_id) , SrcipID=$avp(s:srcip_id) , Termination GW ID=$avp(s:prov_termgw_id)\n); } } b2b_init_request(top hiding); xlog(Saving to $avp(i:99): $json(cs)\n);* and my log says: *Jul 6 01:00:03 db01 opensips-leg2: [leg2]LOG: We got: BillingID=null , Provider RateID=null , SrcipID=null , Termination GW ID=null Jul 6 01:00:03 db01 opensips-leg2: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation Jul 6 01:00:03 db01 opensips-leg2: ERROR:core:parse_headers: pkg memory allocation failed Jul 6 01:00:03 db01 opensips-leg2: ERROR:b2b_logic:init_request: failed to parse message Jul 6 01:00:03 db01 opensips-leg2: Saving to null: { orig_callid: 2524605-3518920775-311...@domain.net, fwd_domainport: sip:16414656...@nnn.nnn.nnn.nnn:5060, billing_id: 1, prov_raterec_id: 2, srcip_id: 1, prov_termgw_id: 11, ani: , display_name: }* I had those errors for every single call attempt. It seems this happens on: *b2b_init_request(top hiding); *I had to restart the B2B to get rid of the issue. After the restart, here is how the same log looks like: *Jul 6 07:51:27 db01 opensips-leg2: [leg2]LOG: We got: BillingID=null , Provider RateID=null , SrcipID=null , Termination GW ID=null Jul 6 07:51:27 db01 opensips-leg2: [leg2][xxx.xxx.xxx.xxx]=LOCAL_ROUTE (INVITE - null) - CallID=B2B.131.4459132 , Term=nnn.nnn.nnn.nnn = Jul 6 07:51:27 db01 opensips-leg2: Saving to 131.0: { orig_callid: 2529570-3518945466-704...@domain.net, fwd_domainport: sip:16414656...@nnn.nnn.nnn.nnn:5060, billing_id: 1, prov_raterec_id: 2, srcip_id: 1, prov_termgw_id: 11, ani: , display_name: }* Here is my local route: *local_route { $var(debug_level) = 10; if (is_present_hf(LEG1CID)) xlog(LEG1CID AFTER TOP HIDING\n); if ($var(debug_level) 2) { xlog([leg2][$si]=LOCAL_ROUTE ($rm - $rr) - CallID=$ci , Term=$rd =\n); } if (is_method(ACK) $ci=~B2B\.[0-9]+\.[0-9]+) { route(19); } else if (is_method(BYE) $ci=~B2B\.[0-9]+\.[0-9]+ src_ip== **xxx.xxx.xxx.xxx**) { route(91); } }* Any idea?* Thanks -- Kamen* On 4 May 2011 11:45, Anca Vamanu a...@opensips.org wrote: ** Hi Jeff, Is this with B2BUA? I have a report with a complex b2b scenario and transactions hanged in memory. Regards, Anca On 29/04/11 16:28, Jeff Pyle wrote: Hello, Everything is running along just fine for weeks on 1.6.4, and then *bam*, this shows up in the logs: /usr/sbin/opensips[12290]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12290]: ERROR:tm:insert_tmcb: no more shared memory /usr/sbin/opensips[12290]: ERROR:uac:replace_uri: failed to install TM callback /usr/sbin/opensips[12284]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12284]: ERROR:core:new_avp: no more shm mem /usr/sbin/opensips[12284]: ERROR:core:add_avp: Failed to create new avp structure /usr/sbin/opensips[12306]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12306]: ERROR:tm:new_t: out of mem /usr/sbin/opensips[12302]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12302]: ERROR:tm:relay_reply: no more share memory /usr/sbin/opensips[12284]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12294]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12284]: ERROR:core:new_avp: no more shm mem /usr/sbin/opensips[12284]: ERROR:core:add_avp: Failed to create new avp structure /usr/sbin/opensips[12306]: ERROR:tm:t_newtran: new_t failed /usr/sbin/opensips[12294]: ERROR:tm:relay_reply: no more share memory Hundreds of lines like it. This is just a sample. But, it did not crash. I have shared memory set to 32M. Normally I run less than 8M used. After this event the max_used_size was at 32M, but the real_used_size was around 24M. And it wasn't changing. Strange. My first thought was a DoS attack but a pcap I took right after looked normal. Normally this proxy runs anywhere from 70-150 transactions in progress at any given moment. After this event, it was stuck at right around 1300. Things seemed to be processing; no symptoms on outbound calls. Very strange. I restarted Opensips and everything seems normal. I'm not sure where to begin on this one. I'm not sure if the shared memory problem was a symptom or a cause. In the few years I've been working with Opensips I've never seen anything like this. Any direction from the list
Re: [OpenSIPS-Users] memory errors, too many transactions
Hi Jeff, Is this with B2BUA? I have a report with a complex b2b scenario and transactions hanged in memory. Regards, Anca On 29/04/11 16:28, Jeff Pyle wrote: Hello, Everything is running along just fine for weeks on 1.6.4, and then /bam/, this shows up in the logs: /usr/sbin/opensips[12290]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12290]: ERROR:tm:insert_tmcb: no more shared memory /usr/sbin/opensips[12290]: ERROR:uac:replace_uri: failed to install TM callback /usr/sbin/opensips[12284]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12284]: ERROR:core:new_avp: no more shm mem /usr/sbin/opensips[12284]: ERROR:core:add_avp: Failed to create new avp structure /usr/sbin/opensips[12306]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12306]: ERROR:tm:new_t: out of mem /usr/sbin/opensips[12302]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12302]: ERROR:tm:relay_reply: no more share memory /usr/sbin/opensips[12284]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12294]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12284]: ERROR:core:new_avp: no more shm mem /usr/sbin/opensips[12284]: ERROR:core:add_avp: Failed to create new avp structure /usr/sbin/opensips[12306]: ERROR:tm:t_newtran: new_t failed /usr/sbin/opensips[12294]: ERROR:tm:relay_reply: no more share memory Hundreds of lines like it. This is just a sample. But, it did not crash. I have shared memory set to 32M. Normally I run less than 8M used. After this event the max_used_size was at 32M, but the real_used_size was around 24M. And it wasn't changing. Strange. My first thought was a DoS attack but a pcap I took right after looked normal. Normally this proxy runs anywhere from 70-150 transactions in progress at any given moment. After this event, it was stuck at right around 1300. Things seemed to be processing; no symptoms on outbound calls. Very strange. I restarted Opensips and everything seems normal. I'm not sure where to begin on this one. I'm not sure if the shared memory problem was a symptom or a cause. In the few years I've been working with Opensips I've never seen anything like this. Any direction from the list would be great. - Jeff ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users -- Anca Vamanu OpenSIPS Developer ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users
[OpenSIPS-Users] memory errors, too many transactions
Hello, Everything is running along just fine for weeks on 1.6.4, and then bam, this shows up in the logs: /usr/sbin/opensips[12290]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12290]: ERROR:tm:insert_tmcb: no more shared memory /usr/sbin/opensips[12290]: ERROR:uac:replace_uri: failed to install TM callback /usr/sbin/opensips[12284]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12284]: ERROR:core:new_avp: no more shm mem /usr/sbin/opensips[12284]: ERROR:core:add_avp: Failed to create new avp structure /usr/sbin/opensips[12306]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12306]: ERROR:tm:new_t: out of mem /usr/sbin/opensips[12302]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12302]: ERROR:tm:relay_reply: no more share memory /usr/sbin/opensips[12284]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12294]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation /usr/sbin/opensips[12284]: ERROR:core:new_avp: no more shm mem /usr/sbin/opensips[12284]: ERROR:core:add_avp: Failed to create new avp structure /usr/sbin/opensips[12306]: ERROR:tm:t_newtran: new_t failed /usr/sbin/opensips[12294]: ERROR:tm:relay_reply: no more share memory Hundreds of lines like it. This is just a sample. But, it did not crash. I have shared memory set to 32M. Normally I run less than 8M used. After this event the max_used_size was at 32M, but the real_used_size was around 24M. And it wasn't changing. Strange. My first thought was a DoS attack but a pcap I took right after looked normal. Normally this proxy runs anywhere from 70-150 transactions in progress at any given moment. After this event, it was stuck at right around 1300. Things seemed to be processing; no symptoms on outbound calls. Very strange. I restarted Opensips and everything seems normal. I'm not sure where to begin on this one. I'm not sure if the shared memory problem was a symptom or a cause. In the few years I've been working with Opensips I've never seen anything like this. Any direction from the list would be great. - Jeff ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users