Hi Bogdan: Yes, we are using autobridge. Can I email you/link you the script privately?
Thanks for the help, BobbyS Sent from my iPhone On Oct 31, 2012, at 8:37 AM, Bogdan-Andrei Iancu <bog...@opensips.org> wrote: > Hi Bobby, > > I would say the leaking is related to RTPproxy stuff: > > 32106 msg_callbacks.c: msg_callback_add(51) > 16053 rtpproxy_callbacks.c: rtpproxy_pre_fwd(51) > 16053 rtpproxy.c: force_rtp_proxy(2848) > 16053 rtpproxy.c: pkg_strdup(2301) > > Have you enabled the "rtpproxy_autobridge" option in rtpproxy module ? and > what rtpp functions are you using ? > > > Regards, > Bogdan-Andrei Iancu > OpenSIPS Founder and Developer > http://www.opensips-solutions.com > > On 10/26/2012 09:57 PM, Bobby Smith wrote: >> >> Here's some information from the memory dump summarized (total number of >> alloc's printed when doing a kill -SIGUSR1 after running for a few hours, >> during off peak time) >> >> This is a count of remaining fragments: >> >> 31 auth_mod.c: generate_random_secret(192) >> 31 avpops.c: avpops_init(220) >> 8866 cfg.lex: addstr(858) >> 155 cfg.y: yyparse(1164) >> 806 cfg.y: yyparse(1577) >> 32 data_lump.c: anchor_lump(398) >> 2 data_lump.c: del_lump(342) >> 24 data_lump.c: insert_cond_lump_after(225) >> 8 data_lump.c: insert_cond_lump_before(252) >> 32 data_lump.c: insert_new_lump_after(120) >> 46 data_lump.c: insert_new_lump_before(146) >> 16 data_lump.c: insert_subst_lump_after(172) >> 2 dbt_base.c: dbt_init(89) >> 11 dlg_vals.c: fetch_dlg_value(156) >> 1 fifo_fnc.c: mi_init_fifo_server(117) >> 1 fifo_fnc.c: mi_init_fifo_server(118) >> 155 ip_addr.c: mk_net(56) >> 7 lb_data.c: do_load_balance(403) >> 7 lb_data.c: do_load_balance(450) >> 31 lb_parser.c: parse_resources_list(115) >> 31 load_balancer.c: fixup_resources(163) >> 31 load_balancer.c: fixup_resources(193) >> 31 memcache.c: register_memcache(91) >> 1 mi/fmt.c: mi_fmt_init(45) >> 2 mi/mi_core.c: init_mi_uptime(62) >> 31 mi/mi.c: register_mi_cmd(145) >> 1 mi_parser.c: mi_parser_init(48) >> 29 mi_writer.c: mi_writer_init(51) >> 155 mod_fix.c: fixup_pvar(437) >> 155 mod_fix.c: fixup_regexp(303) >> 93 mod_fix.c: fixup_str(70) >> 32106 msg_callbacks.c: msg_callback_add(51) >> 93 name_alias.c: add_alias(59) >> 93 name_alias.c: add_alias(61) >> 5 parser/contact/contact.c: parse_contacts(192) >> 5 parser/contact/parse_contact.c: parse_contact(81) >> 16 parser/msg_parser.c: get_hdr_field(112) >> 10 parser/msg_parser.c: get_hdr_field(134) >> 10 parser/msg_parser.c: get_hdr_field(158) >> 120 parser/msg_parser.c: parse_headers(314) >> 3 parser/msg_parser.c: set_dst_uri(758) >> 10 parser/parse_content.c: parse_content_type_hdr(432) >> 8 parser/parse_from.c: parse_from_header(63) >> 10 parser/parse_multipart.c: get_all_bodies(212) >> 10 parser/parse_multipart.c: new_part(38) >> 3 parser/parse_param.c: parse_params(365) >> 5 parser/parse_rr.c: do_parse_rr_body(65) >> 13 parser/parse_to.c: parse_to_param(280) >> 24 parser/parse_via.c: parse_via(1774) >> 96 proxy.c: hostent_cpy(119) >> 96 proxy.c: hostent_cpy(130) >> 96 proxy.c: hostent_cpy(153) >> 96 proxy.c: hostent_cpy(163) >> 96 proxy.c: mk_proxy(238) >> 62 pvar.c: new_pv_context(4409) >> 341 pvar.c: pv_add_extra(4334) >> 31 pvar.c: pv_init_extra_list(4270) >> 4309 pvar.c: pv_parse_format(3782) >> 11 receive.c: receive_msg(92) >> 16 record.c: build_rr(158) >> 16 record.c: build_rr(159) >> 16 record.c: build_rr(160) >> 16 record.c: build_rr(161) >> 14 record.c: insert_rr_param_lump(126) >> 93 route.c: fix_actions(614) >> 155 route.c: fix_expr(191) >> 6386 route_struct.c: mk_action(106) >> 1984 route_struct.c: mk_elem(72) >> 2294 route_struct.c: mk_exp(56) >> 31 rr_cb.c: register_rrcb(59) >> 16053 rtpproxy_callbacks.c: rtpproxy_pre_fwd(51) >> 30 rtpproxy.c: append_opts(2742) >> 30 rtpproxy.c: connect_rtpproxies(1281) >> 16053 rtpproxy.c: force_rtp_proxy(2848) >> 16053 rtpproxy.c: pkg_strdup(2301) >> 124 script_cb.c: add_callback(60) >> 62 script_var.c: add_var(54) >> 62 script_var.c: add_var(61) >> 11 script_var.c: set_var_value(114) >> 124 socket_info.c: fix_socket_list(569) >> 124 socket_info.c: fix_socket_list(620) >> 124 socket_info.c: fix_socket_list(699) >> 1 socket_info.c: get_socket_list_from_proto(952) >> 124 socket_info.c: new_sock_info(112) >> 124 socket_info.c: new_sock_info(117) >> 589 sr_module.c: register_module(140) >> 279 textops.c: fixup_method(1315) >> 62 textops.c: hname_fixup(1244) >> 217 timer.c: new_sr_timer(150) >> 31 timer.c: new_timer_process_list(69) >> 186 transformations.c: parse_transformation(1626) >> 2 t_reply.c: fake_req(541) >> 27 udp_server.c: udp_rcv_loop(342) >> 465 usr_avp.c: new_avp_alias(451) >> 17 xlog.c: buf_init(50) >> 1 xr_writer.c: xr_writer_init(43) >> >> >> On Fri, Oct 26, 2012 at 11:06 AM, Bobby Smith <bobby.sm...@gmail.com> wrote: >>> Greetings list, >>> >>> We're trying to track down a memory leak with 1.7 R9225+ (It's actually >>> been happening since we've upgraded to 1.7 in general, but we've updated >>> svn a few times and recompiled to see if the problem goes away). >>> >>> Here's a diagram of memory usage over a period of time: >>> >>> http://i45.tinypic.com/ngpqwp.png >>> >>> Basically we're seeing a package memory leak where package memory is never >>> freed under negligible load. The memory used graph looks like a step >>> graph, where flat correlates to a period where we're off peak with traffic. >>> This will continue for 2-3 days before we see opensips processes start to >>> hang trying to free package memory. At this point we force an HA restart >>> and the memory dips. >>> >>> ########################## >>> >>> 2012-10-12T18:05:35.569400+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27800]: ERROR:core:parse_headers: pkg >>> memory allocation failed >>> 2012-10-12T18:05:35.569439+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27800]: ERROR:core:receive_msg: parse_msg >>> failed >>> 2012-10-12T18:05:35.569871+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27795]: ERROR:core:receive_msg: no pkg >>> mem left for sip_msg >>> 2012-10-12T18:05:35.587197+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27805]: [REQUEST >>> d9c7633662d475f58263856ca5b91efe@10.2.1.173] :: Beginning processing >>> OPTIONS for d9c7633662d475f58263856ca5b91efe@10.2.1.173 from 10.2.1.173 >>> 2012-10-12T18:05:35.590072+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27798]: ERROR:core:receive_msg: no pkg >>> mem left for sip_msg >>> 2012-10-12T18:05:35.590094+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27798]: ERROR:core:receive_msg: no pkg >>> mem left for sip_msg >>> 2012-10-12T18:05:35.590579+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27796]: ERROR:core:get_hdr_field: out of >>> pkg memory >>> 2012-10-12T18:05:35.590600+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27796]: INFO:core:parse_headers: bad >>> header field >>> 2012-10-12T18:05:35.590618+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27796]: ERROR:core:pv_get_callid: cannot >>> parse Call-Id header >>> >>> >>> 2012-10-12T18:05:35.791241+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27794]: >>> ERROR:core:build_res_buf_from_sip_req: parse_headers failed >>> 2012-10-12T18:05:35.791258+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27796]: ERROR:maxfwd:is_maxfwd_present: >>> parsing MAX_FORWARD header failed! >>> 2012-10-12T18:05:35.791280+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27794]: ERROR:sl:sl_send_reply_helper: >>> response building failed >>> 2012-10-12T18:05:35.791295+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27799]: ERROR:core:get_hdr_field: bad via >>> 2012-10-12T18:05:35.791309+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27796]: ERROR:core:get_hdr_field: out of >>> pkg memory >>> 2012-10-12T18:05:35.791324+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27799]: INFO:core:parse_headers: bad >>> header field >>> 2012-10-12T18:05:35.791338+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27796]: INFO:core:parse_headers: bad >>> header field >>> 2012-10-12T18:05:35.791354+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27796]: ERROR:tm:t_newtran: parse_headers >>> failed >>> 2012-10-12T18:05:35.791366+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27799]: ERROR:core:parse_via: no pkg >>> memory left >>> 2012-10-12T18:05:35.791376+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27796]: ERROR:core:get_hdr_field: out of >>> pkg memory >>> 2012-10-12T18:05:35.791384+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27796]: INFO:core:parse_headers: bad >>> header field >>> 2012-10-12T18:05:35.791392+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27799]: ERROR:core:get_hdr_field: bad via >>> 2012-10-12T18:05:35.791401+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27796]: ERROR:core:get_hdr_field: out of >>> pkg memory >>> 2012-10-12T18:05:35.791409+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27799]: INFO:core:parse_headers: bad >>> header field >>> 2012-10-12T18:05:35.791418+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27796]: INFO:core:parse_headers: bad >>> header field >>> 2012-10-12T18:05:35.791426+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27799]: >>> ERROR:core:build_res_buf_from_sip_req: parse_headers failed >>> 2012-10-12T18:05:35.791436+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27796]: >>> ERROR:core:build_res_buf_from_sip_req: parse_headers failed >>> 2012-10-12T18:05:35.791443+00:00 atlsvpctrl09 >>> /usr/local/opensips/sbin/opensips[27799]: ERROR:sl:sl_send_reply_helper: >>> response building > _______________________________________________ > Users mailing list > Users@lists.opensips.org > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
_______________________________________________ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users