[OpenSIPS-Users] Core dump in svn head
Over the past couple weeks I have been getting occasional segfaults just prior to (or perhaps in the process of) a t_relay() in my failure_route. Still haven't gotten to the bottom of the root cause in my config, but I was able to find and fix the symptomatic code in the pre_print_uac_request() function in t_fwd.c (based on Bogdan's previous fix for a similar problem). Backtrace: #0 0xb763ba55 in memcpy () from /lib/tls/i686/cmov/libc.so.6 #1 0xb72ede1d in pre_print_uac_request (t=0x972ddf78, branch=1, request=0x972cec64) at t_fwd.c:177 #2 0xb72eeac0 in add_uac (t=0x972ddf78, request=0x7fff, uri=0xbf8a250c, next_hop=0xbf8a2514, path=0xb7315100, proxy=0x0) at t_fwd.c:401 #3 0xb72efbc0 in t_forward_nonack (t=0x972ddf78, p_msg=0xb7314de0, proxy=0x0) at t_fwd.c:660 #4 0xb72fbdbf in w_t_relay (p_msg=0xb7314de0, proxy=0x0, flags=) at tm.c:1116 #5 0x08057810 in do_action (a=0x819f924, msg=0xb7314de0) at action.c:1155 #6 0x080557ff in run_action_list (a=0x819e7cc, msg=0xb7314de0) at action.c:140 #7 0x0805981e in do_action (a=0x819fed4, msg=0xb7314de0) at action.c:821 #8 0x080557ff in run_action_list (a=0x819e584, msg=0xb7314de0) at action.c:140 #9 0x0805981e in do_action (a=0x81a0390, msg=0xb7314de0) at action.c:821 #10 0x080557ff in run_action_list (a=0x819d588, msg=0xb7314de0) at action.c:140 #11 0x0805aef1 in run_top_route (a=0x819d588, msg=0xb7314de0) at action.c:120 #12 0xb7304c2c in t_should_relay_response (Trans=0x972ddf78, new_code=408, branch=0, should_store=0xbf8a3094, should_relay=0xbf8a3098, cancel_bitmap=0xbf8a30e8, reply=0x) at t_reply.c:621 #13 0xb730683d in relay_reply (t=0x972ddf78, p_msg=0x, branch=0, msg_status=408, cancel_bitmap=0xbf8a30e8) at t_reply.c:1133 #14 0xb72f21e5 in fake_reply (t=0x972ddf78, branch=0, code=408) at timer.c:259 #15 0xb72f3dd7 in timer_routine (ticks=3285, attr=0x0) at timer.c:365 #16 0x080c12c0 in start_timer_processes () at timer.c:325 #17 0x0806e37e in main (argc=Cannot access memory at address 0x7fff ) at main.c:872 Here is the patch to fix: Index: modules/tm/t_fwd.c === --- modules/tm/t_fwd.c (revision 7259) +++ modules/tm/t_fwd.c (working copy) @@ -166,7 +166,7 @@ -request->REQ_METHOD); /* copy dst_uri into branch (after branch route possible updated it) */ - if (request->dst_uri.len) { + if ( request->dst_uri.s && request->dst_uri.len>0 ) { t->uac[branch].duri.s = shm_resize(t->uac[branch].duri.s, request->dst_uri.len); if (t->uac[branch].duri.s==NULL) { After the code change and a recompile, the following lines appear in my logs, whereas previously at this spot OpenSIPS crashed: Oct 12 16:04:13 sipdev1 /usr/sbin/opensips[31245]: ERROR:core:parse_uri: uri too short: <> (-1) Oct 12 16:04:13 sipdev1 /usr/sbin/opensips[31245]: ERROR:tm:uri2proxy: bad_uri: (null) Oct 12 16:04:13 sipdev1 /usr/sbin/opensips[31245]: ERROR:tm:t_forward_nonack: failure to add branches Oct 12 16:04:13 sipdev1 /usr/sbin/opensips[31245]: ERROR:tm:w_t_relay: t_forward_nonack failed Dave. ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Re: [OpenSIPS-Users] Core dump in svn head
Hi Dave, In failure route, how do you add the new destination/branch ? Also, do you have a branch route set ? Regards, Bogdan Dave May wrote: > Over the past couple weeks I have been getting occasional segfaults just > prior to (or perhaps in the process of) a t_relay() in my failure_route. > Still haven't gotten to the bottom of the root cause in my config, but I > was able to find and fix the symptomatic code in the > pre_print_uac_request() function in t_fwd.c (based on Bogdan's previous > fix for a similar problem). > > Backtrace: > #0 0xb763ba55 in memcpy () from /lib/tls/i686/cmov/libc.so.6 > #1 0xb72ede1d in pre_print_uac_request (t=0x972ddf78, branch=1, > request=0x972cec64) at t_fwd.c:177 > #2 0xb72eeac0 in add_uac (t=0x972ddf78, request=0x7fff, > uri=0xbf8a250c, next_hop=0xbf8a2514, path=0xb7315100, proxy=0x0) at > t_fwd.c:401 > #3 0xb72efbc0 in t_forward_nonack (t=0x972ddf78, p_msg=0xb7314de0, > proxy=0x0) at t_fwd.c:660 > #4 0xb72fbdbf in w_t_relay (p_msg=0xb7314de0, proxy=0x0, flags= optimized out>) at tm.c:1116 > #5 0x08057810 in do_action (a=0x819f924, msg=0xb7314de0) at > action.c:1155 > #6 0x080557ff in run_action_list (a=0x819e7cc, msg=0xb7314de0) at > action.c:140 > #7 0x0805981e in do_action (a=0x819fed4, msg=0xb7314de0) at > action.c:821 > #8 0x080557ff in run_action_list (a=0x819e584, msg=0xb7314de0) at > action.c:140 > #9 0x0805981e in do_action (a=0x81a0390, msg=0xb7314de0) at > action.c:821 > #10 0x080557ff in run_action_list (a=0x819d588, msg=0xb7314de0) at > action.c:140 > #11 0x0805aef1 in run_top_route (a=0x819d588, msg=0xb7314de0) at > action.c:120 > #12 0xb7304c2c in t_should_relay_response (Trans=0x972ddf78, > new_code=408, branch=0, should_store=0xbf8a3094, > should_relay=0xbf8a3098, cancel_bitmap=0xbf8a30e8, reply=0x) > at t_reply.c:621 > #13 0xb730683d in relay_reply (t=0x972ddf78, p_msg=0x, branch=0, > msg_status=408, cancel_bitmap=0xbf8a30e8) at t_reply.c:1133 > #14 0xb72f21e5 in fake_reply (t=0x972ddf78, branch=0, code=408) at > timer.c:259 > #15 0xb72f3dd7 in timer_routine (ticks=3285, attr=0x0) at timer.c:365 > #16 0x080c12c0 in start_timer_processes () at timer.c:325 > #17 0x0806e37e in main (argc=Cannot access memory at address 0x7fff > ) at main.c:872 > > Here is the patch to fix: > > Index: modules/tm/t_fwd.c > === > --- modules/tm/t_fwd.c (revision 7259) > +++ modules/tm/t_fwd.c (working copy) > @@ -166,7 +166,7 @@ > -request->REQ_METHOD); > > /* copy dst_uri into branch (after branch route possible updated > it) */ > - if (request->dst_uri.len) { > + if ( request->dst_uri.s && request->dst_uri.len>0 ) { > t->uac[branch].duri.s = > shm_resize(t->uac[branch].duri.s, > request->dst_uri.len); > if (t->uac[branch].duri.s==NULL) { > > > After the code change and a recompile, the following lines appear in my > logs, whereas previously at this spot OpenSIPS crashed: > > Oct 12 16:04:13 sipdev1 /usr/sbin/opensips[31245]: ERROR:core:parse_uri: > uri too short: <> (-1) > Oct 12 16:04:13 sipdev1 /usr/sbin/opensips[31245]: ERROR:tm:uri2proxy: > bad_uri: (null) > Oct 12 16:04:13 sipdev1 /usr/sbin/opensips[31245]: > ERROR:tm:t_forward_nonack: failure to add branches > Oct 12 16:04:13 sipdev1 /usr/sbin/opensips[31245]: ERROR:tm:w_t_relay: > t_forward_nonack failed > > Dave. > > ___ > Users mailing list > Users@lists.opensips.org > http://lists.opensips.org/cgi-bin/mailman/listinfo/users > > -- Bogdan-Andrei Iancu OpenSIPS Bootcamp 15 - 19 November 2010, Edison, New Jersey, USA www.voice-system.ro ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Re: [OpenSIPS-Users] Core dump in svn head
Yes, I do set a branch_route to strip RPID header and strip/set the P-Asserted-Identity header. I'm using drouting, and that portion of the failure_route is based on the code from Flavio's book (with the addition of e164 mangling, t_on_branch to handle the header stuff mentioned above, and t_on_failure to handle multiple failures): if(isflagset(10)){ if (!t_check_status("408|[56][0-9][0-9]")) { # this is not a GW failure exit; } if (use_next_gw()) { xlog("$ci\t$C(rc)$si $rm$C(xx) $C(xr)failure_route[1]>$C(xx) $C(xr)use_next_gw()=$C(rg)$ru$C(xx)\n"); ### Need to duplicate e164 check from original do_routing() call ### Want to do this in branch_route, but not allowed to use is_uri_user_e164() there ### # # Does the gateway entry request e164 format? if ($avp(s:dr_gw_attrs) != "") && ($(avp(s:dr_gw_attrs){param.value,format})=="e164") { xlog("$ci\t$C(rc)$si $rm$C(xx) failure_route[1]> Gateway requires e164 format\n"); # Check and fix RURI if (is_uri_user_e164("$ru")) { xlog("$ci\t$C(rc)$si $rm$C(xx) failure_route[1]> $ru is in e164 format\n"); } else { xlog("$ci\t$C(rc)$si $rm$C(xx) failure_route[1]> $ru is not in e164 format\n"); dp_translate("3"); } } # Need to ensure that failures on the failure are also handled t_on_branch("1"); t_on_failure("1"); xlog("$ci\t$C(rc)$si $rm$C(xx) failure_route[1]> t_relay() next rU = $rU fU=$fU tU=$tU du=$du\n"); setdebug(4); t_relay(); setdebug(); xlog("$ci\t$C(rc)$si $rm$C(xx) failure_route[1]> t_relay() complete\n"); exit; } else { xlog("$ci\t$C(rc)$si $rm$C(xx) $C(xr)failure_route[1]->$C(xx) isflagset(10) $C(sr)--> use_next_gw() FAILED!!!$C(xx)\n"); t_reply("503", "Service not available, no more gateways"); exit; } Dave. -Original Message- From: users-boun...@lists.opensips.org [mailto:users-boun...@lists.opensips.org] On Behalf Of Bogdan-Andrei Iancu Sent: Wednesday, October 13, 2010 4:43 PM To: OpenSIPS users mailling list Subject: Re: [OpenSIPS-Users] Core dump in svn head Hi Dave, In failure route, how do you add the new destination/branch ? Also, do you have a branch route set ? Regards, Bogdan Dave May wrote: > Over the past couple weeks I have been getting occasional segfaults just > prior to (or perhaps in the process of) a t_relay() in my failure_route. > Still haven't gotten to the bottom of the root cause in my config, but I > was able to find and fix the symptomatic code in the > pre_print_uac_request() function in t_fwd.c (based on Bogdan's previous > fix for a similar problem). > > Backtrace: > #0 0xb763ba55 in memcpy () from /lib/tls/i686/cmov/libc.so.6 > #1 0xb72ede1d in pre_print_uac_request (t=0x972ddf78, branch=1, > request=0x972cec64) at t_fwd.c:177 > #2 0xb72eeac0 in add_uac (t=0x972ddf78, request=0x7fff, > uri=0xbf8a250c, next_hop=0xbf8a2514, path=0xb7315100, proxy=0x0) at > t_fwd.c:401 > #3 0xb72efbc0 in t_forward_nonack (t=0x972ddf78, p_msg=0xb7314de0, > proxy=0x0) at t_fwd.c:660 > #4 0xb72fbdbf in w_t_relay (p_msg=0xb7314de0, proxy=0x0, flags= optimized out>) at tm.c:1116 > #5 0x08057810 in do_action (a=0x819f924, msg=0xb7314de0) at > action.c:1155 > #6 0x080557ff in run_action_list (a=0x819e7cc, msg=0xb7314de0) at > action.c:140 > #7 0x0805981e in do_action (a=0x819fed4, msg=0xb7314de0) at > action.c:821 > #8 0x080557ff in run_action_list (a=0x819e584, msg=0xb7314de0) at > action.c:140 > #9 0x0805981e in do_action (a=0x81a0390, msg=0xb7314de0) at > action.c:821 > #10 0x080557ff in run_action_list (a=0x819d588, msg=0xb7314de0) at > action.c:140 > #11 0x0805aef1 in run_top_route (a=0x819d588,
Re: [OpenSIPS-Users] Core dump in svn head
Hi Dave, the crash refers to an invalid dst_uri and I see in your failure route you are not using any $du related op Also, I see you add a single new branch (via RURI) (no parallel forking) Also, in branch_route, you do not do any dst_uri or ruri ops - you work only on headers ? Could you print from gdb: frame 1 : request->dst_uri frame 2: *next_hop frame 3: p_msg->dst_uri Best regards, Bogdan Dave May wrote: > Yes, I do set a branch_route to strip RPID header and strip/set the > P-Asserted-Identity header. > > I'm using drouting, and that portion of the failure_route is based on > the code from Flavio's book (with the addition of e164 mangling, > t_on_branch to handle the header stuff mentioned above, and t_on_failure > to handle multiple failures): > > > > if(isflagset(10)){ > if (!t_check_status("408|[56][0-9][0-9]")) { > # this is not a GW failure > exit; > } > > if (use_next_gw()) { > xlog("$ci\t$C(rc)$si $rm$C(xx) > $C(xr)failure_route[1]>$C(xx) > $C(xr)use_next_gw()=$C(rg)$ru$C(xx)\n"); > > ### Need to duplicate e164 check from original > do_routing() call > ### Want to do this in branch_route, but not > allowed to use is_uri_user_e164() there > ### > # > # Does the gateway entry request e164 format? > if ($avp(s:dr_gw_attrs) != "") && > ($(avp(s:dr_gw_attrs){param.value,format})=="e164") { > > xlog("$ci\t$C(rc)$si $rm$C(xx) > failure_route[1]> Gateway requires e164 format\n"); > > # Check and fix RURI > if (is_uri_user_e164("$ru")) { > xlog("$ci\t$C(rc)$si $rm$C(xx) > failure_route[1]> $ru is in e164 format\n"); > } else { > xlog("$ci\t$C(rc)$si $rm$C(xx) > failure_route[1]> $ru is not in e164 format\n"); > dp_translate("3"); > } > } > > # Need to ensure that failures on the failure > are also handled > t_on_branch("1"); > t_on_failure("1"); > > xlog("$ci\t$C(rc)$si $rm$C(xx) > failure_route[1]> t_relay() next rU = $rU fU=$fU > tU=$tU du=$du\n"); > setdebug(4); > t_relay(); > setdebug(); > xlog("$ci\t$C(rc)$si $rm$C(xx) > failure_route[1]> t_relay() complete\n"); > exit; > } else { > xlog("$ci\t$C(rc)$si $rm$C(xx) > $C(xr)failure_route[1]->$C(xx) isflagset(10) $C(sr)--> use_next_gw() > FAILED!!!$C(xx)\n"); > > t_reply("503", "Service not available, no more > gateways"); > exit; > } > > Dave. > > -Original Message- > From: users-boun...@lists.opensips.org > [mailto:users-boun...@lists.opensips.org] On Behalf Of Bogdan-Andrei > Iancu > Sent: Wednesday, October 13, 2010 4:43 PM > To: OpenSIPS users mailling list > Subject: Re: [OpenSIPS-Users] Core dump in svn head > > Hi Dave, > > In failure route, how do you add the new destination/branch ? > > Also, do you have a branch route set ? > > Regards, > Bogdan > > Dave May wrote: > >> Over the past couple weeks I have been getting occasional segfaults >> > just > >> prior to (or perhaps in the process of) a t_relay() in my >> > failure_route. > >> Still haven't gotten to the bottom of the root cause in my config, but >> > I > >> was able to find and fix the symptomatic code in the >> pre_print_uac_request() function in t_fwd.c (based on Bogdan's >> > previous > >> fix for a similar problem). >> >> Backtrace: >> #0 0xb763ba55 in memcpy () from /lib/tls/i686/cmov/libc.so.6 >> #1 0xb72ede1d in pre_print_uac_request (t=0x972ddf78, branch=1, >> request=0x972cec64) at