subject:"\[OpenSIPS\-Users\] Segfault during load test"

Re: [OpenSIPS-Users] Segfault during load test

2010-04-14 Thread Bogdan-Andrei Iancu

Thanks to Andrei, the bug is fixed on trunk and 1.6 version

Regards,
Bogdan

Bogdan-Andrei Iancu wrote:
> Hi Takeshi,
>
> It seams there is a missing error handling in case of "no more mem".
>
> This will be fixed asap.
>
> Thanks for the report and help,
> Bogdan
>
> mayamatakeshi wrote:
>   
>> Hi Bogdan,
>>
>> On Mon, Apr 12, 2010 at 10:27 PM, Bogdan-Andrei Iancu 
>> mailto:bog...@voice-system.ro>> wrote:
>>
>> Hi Takeshi,
>>
>> Do you still have the core file ?  If yes, could you print  "dest" and
>> "*dest" ?
>>
>>
>>
>> (gdb) print dest
>> $1 = (void **) 0x0
>>  
>>
>>
>> Do you get any previous errors (from the same process) ? like mem
>> errors?
>>
>>
>> Now that you mentioned, I went to the log file and saw lots of this:
>>
>> Apr  3 05:25:19 centos53_02005 /usr/local/sbin/opensips[29732]: 
>> ERROR:tm:relay_reply: no more share memory
>> Apr  3 05:25:19 centos53_02005 /usr/local/sbin/opensips[29732]: 
>> ERROR:tm:_reply_light: failed to allocate shmem buffer
>>
>> (sorry, I should have looked into this as first thing)
>>  
>>
>>
>> Also, can you reproduce this ?
>>
>>
>> I was load testing r.  6605 with 800 Registrations per second (32000 
>> users) and didn't saw this ever happen before (I ran this same load 
>> session several times: one of them reached 1 billion registration 
>> requests). And actually, in previous tests I was using opensips with 
>> just 256 MB of shared memory and not with "-m 1024" as above.
>> With r. 6758 so far nothing wrong happened.
>> I will update to 1.6 head and will report if problem happens again, 
>> but maybe in the above case, my SIPp instance went crazy in the other 
>> end and generated too much traffic.
>>
>> Thanks and regards,
>> takeshi
>>
>>
>> mayamatakeshi wrote:
>> > Hello,
>> > I was load testing opensips-1.6 r. 6605
>> > and I got a seg fault.
>> > I will try updating to most recent revision of 1.6, but I thought
>> > someone would like to take a look at this.
>> >
>> > regards,
>> > takeshi
>> >
>> >
>> > Core was generated by `/usr/local/sbin/opensips -m 1024'.
>> > Program terminated with signal 11, Segmentation fault.
>> > [New process 29728]
>> > #0  slot_add (_s=0x2b361ce6ac10, _r=0x2b363326ca20) at hslot.c:151
>> > 151 *dest = _r;
>> > (gdb) bt
>> > #0  slot_add (_s=0x2b361ce6ac10, _r=0x2b363326ca20) at hslot.c:151
>> > #1  0x2b3619b9ac65 in mem_insert_urecord (_d=0x2b361ce69560,
>> > _aor=, _r=0x7fff919cb740)
>> > at udomain.c:648
>> > #2  0x2b3619b9ad42 in insert_urecord (_d=0x2b361ce69560,
>> > _aor=0x7fff919cb958, _r=0x7fff919cb740) at udomain.c:781
>> > #3  0x2b3619fb5769 in insert_contacts (_m=0x79b600, _c=0x79c468,
>> > _d=0x2b361ce69560, _a=0x7fff919cb958,
>> > _sctx=0x7fff919cb950) at save.c:398
>> > #4  0x2b3619fb6592 in save (_m=0x79b600, _d=0x2b361ce69560 "",
>> > _f=, _s=0x0) at save.c:680
>> > #5  0x0040e9e9 in do_action (a=0x78def8, msg=0x79b600) at
>> > action.c:967
>> > #6  0x004114bc in run_action_list (a=,
>> > msg=0x79b600) at action.c:139
>> > #7  0x00466b24 in eval_expr (e=0x78dfd0, msg=0x79b600,
>> > val=0x0) at route.c:1240
>> > #8  0x0046659d in eval_expr (e=0x78e020, msg=0x79b600,
>> > val=0x0) at route.c:1553
>> > #9  0x00466545 in eval_expr (e=0x78e070, msg=0x79b600,
>> > val=0x0) at route.c:1558
>> > #10 0x0040d9cd in do_action (a=0x78e2c8, msg=0x79b600) at
>> > action.c:689
>> > #11 0x004114bc in run_action_list (a=,
>> > msg=0x79b600) at action.c:139
>> > #12 0x0040fcc3 in do_action (a=0x78e888, msg=0x79b600) at
>> > action.c:706
>> > #13 0x004114bc in run_action_list (a=,
>> > msg=0x79b600) at action.c:139
>> > #14 0x00411819 in run_top_route (a=0x78ad18,
>> msg=0x79b600) at
>> > action.c:119
>> > #15 0x00455ff5 in receive_msg (
>> > buf=0x754f20 "REGISTER sip:domain13564.com
>> 
>> >  SIP/2.0\r\nVia: SIP/2.0/UDP
>> > 192.168.2.6:;rport;branch=z9hG4bK-4540-1443964-5\r\nFrom:
>> > \"13564\" > 
>> > > >>;tag=4540SIPpTag001443964\r\nTo:
>> > \"13564\" > > rcv_info=0x7fff919ccc10) at receive.c:162
>> > #16 0x0049a304 in udp_rcv_loop () at udp_server.c:492
>> > #17 0x00429bbd in main (argc=3, argv=> out>) at
>> > main.c:818
>> > (gdb)
>> >
>> >
>> 
>> >
>> > ___
>> > Users mailing list
>> > Users@lists.opensips.org 
>> > http://lists.opensips.org/cgi-bin

Re: [OpenSIPS-Users] Segfault during load test

2010-04-13 Thread Bogdan-Andrei Iancu

Hi Takeshi,

It seams there is a missing error handling in case of "no more mem".

This will be fixed asap.

Thanks for the report and help,
Bogdan

mayamatakeshi wrote:
> Hi Bogdan,
>
> On Mon, Apr 12, 2010 at 10:27 PM, Bogdan-Andrei Iancu 
> mailto:bog...@voice-system.ro>> wrote:
>
> Hi Takeshi,
>
> Do you still have the core file ?  If yes, could you print  "dest" and
> "*dest" ?
>
>
>
> (gdb) print dest
> $1 = (void **) 0x0
>  
>
>
> Do you get any previous errors (from the same process) ? like mem
> errors?
>
>
> Now that you mentioned, I went to the log file and saw lots of this:
>
> Apr  3 05:25:19 centos53_02005 /usr/local/sbin/opensips[29732]: 
> ERROR:tm:relay_reply: no more share memory
> Apr  3 05:25:19 centos53_02005 /usr/local/sbin/opensips[29732]: 
> ERROR:tm:_reply_light: failed to allocate shmem buffer
>
> (sorry, I should have looked into this as first thing)
>  
>
>
> Also, can you reproduce this ?
>
>
> I was load testing r.  6605 with 800 Registrations per second (32000 
> users) and didn't saw this ever happen before (I ran this same load 
> session several times: one of them reached 1 billion registration 
> requests). And actually, in previous tests I was using opensips with 
> just 256 MB of shared memory and not with "-m 1024" as above.
> With r. 6758 so far nothing wrong happened.
> I will update to 1.6 head and will report if problem happens again, 
> but maybe in the above case, my SIPp instance went crazy in the other 
> end and generated too much traffic.
>
> Thanks and regards,
> takeshi
>
>
> mayamatakeshi wrote:
> > Hello,
> > I was load testing opensips-1.6 r. 6605
> > and I got a seg fault.
> > I will try updating to most recent revision of 1.6, but I thought
> > someone would like to take a look at this.
> >
> > regards,
> > takeshi
> >
> >
> > Core was generated by `/usr/local/sbin/opensips -m 1024'.
> > Program terminated with signal 11, Segmentation fault.
> > [New process 29728]
> > #0  slot_add (_s=0x2b361ce6ac10, _r=0x2b363326ca20) at hslot.c:151
> > 151 *dest = _r;
> > (gdb) bt
> > #0  slot_add (_s=0x2b361ce6ac10, _r=0x2b363326ca20) at hslot.c:151
> > #1  0x2b3619b9ac65 in mem_insert_urecord (_d=0x2b361ce69560,
> > _aor=, _r=0x7fff919cb740)
> > at udomain.c:648
> > #2  0x2b3619b9ad42 in insert_urecord (_d=0x2b361ce69560,
> > _aor=0x7fff919cb958, _r=0x7fff919cb740) at udomain.c:781
> > #3  0x2b3619fb5769 in insert_contacts (_m=0x79b600, _c=0x79c468,
> > _d=0x2b361ce69560, _a=0x7fff919cb958,
> > _sctx=0x7fff919cb950) at save.c:398
> > #4  0x2b3619fb6592 in save (_m=0x79b600, _d=0x2b361ce69560 "",
> > _f=, _s=0x0) at save.c:680
> > #5  0x0040e9e9 in do_action (a=0x78def8, msg=0x79b600) at
> > action.c:967
> > #6  0x004114bc in run_action_list (a=,
> > msg=0x79b600) at action.c:139
> > #7  0x00466b24 in eval_expr (e=0x78dfd0, msg=0x79b600,
> > val=0x0) at route.c:1240
> > #8  0x0046659d in eval_expr (e=0x78e020, msg=0x79b600,
> > val=0x0) at route.c:1553
> > #9  0x00466545 in eval_expr (e=0x78e070, msg=0x79b600,
> > val=0x0) at route.c:1558
> > #10 0x0040d9cd in do_action (a=0x78e2c8, msg=0x79b600) at
> > action.c:689
> > #11 0x004114bc in run_action_list (a=,
> > msg=0x79b600) at action.c:139
> > #12 0x0040fcc3 in do_action (a=0x78e888, msg=0x79b600) at
> > action.c:706
> > #13 0x004114bc in run_action_list (a=,
> > msg=0x79b600) at action.c:139
> > #14 0x00411819 in run_top_route (a=0x78ad18,
> msg=0x79b600) at
> > action.c:119
> > #15 0x00455ff5 in receive_msg (
> > buf=0x754f20 "REGISTER sip:domain13564.com
> 
> >  SIP/2.0\r\nVia: SIP/2.0/UDP
> > 192.168.2.6:;rport;branch=z9hG4bK-4540-1443964-5\r\nFrom:
> > \"13564\"  
> >  >>;tag=4540SIPpTag001443964\r\nTo:
> > \"13564\"  > rcv_info=0x7fff919ccc10) at receive.c:162
> > #16 0x0049a304 in udp_rcv_loop () at udp_server.c:492
> > #17 0x00429bbd in main (argc=3, argv= out>) at
> > main.c:818
> > (gdb)
> >
> >
> 
> >
> > ___
> > Users mailing list
> > Users@lists.opensips.org 
> > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> >
>
>
> --
> Bogdan-Andrei Iancu
> www.voice-system.ro 
>
>
> ___
> Users mailing list
> Users@lists.opensips.org

Re: [OpenSIPS-Users] Segfault during load test

2010-04-13 Thread mayamatakeshi

Hi Bogdan,

On Mon, Apr 12, 2010 at 10:27 PM, Bogdan-Andrei Iancu <
bog...@voice-system.ro> wrote:

> Hi Takeshi,
>
> Do you still have the core file ?  If yes, could you print  "dest" and
> "*dest" ?
>


(gdb) print dest
$1 = (void **) 0x0


>
> Do you get any previous errors (from the same process) ? like mem errors?
>

Now that you mentioned, I went to the log file and saw lots of this:

Apr  3 05:25:19 centos53_02005 /usr/local/sbin/opensips[29732]:
ERROR:tm:relay_reply: no more share memory
Apr  3 05:25:19 centos53_02005 /usr/local/sbin/opensips[29732]:
ERROR:tm:_reply_light: failed to allocate shmem buffer

(sorry, I should have looked into this as first thing)


>
> Also, can you reproduce this ?
>

I was load testing r.  6605 with 800 Registrations per second (32000 users)
and didn't saw this ever happen before (I ran this same load session several
times: one of them reached 1 billion registration requests). And actually,
in previous tests I was using opensips with just 256 MB of shared memory and
not with "-m 1024" as above.
With r. 6758 so far nothing wrong happened.
I will update to 1.6 head and will report if problem happens again, but
maybe in the above case, my SIPp instance went crazy in the other end and
generated too much traffic.

Thanks and regards,
takeshi


> mayamatakeshi wrote:
> > Hello,
> > I was load testing opensips-1.6 r. 6605
> > and I got a seg fault.
> > I will try updating to most recent revision of 1.6, but I thought
> > someone would like to take a look at this.
> >
> > regards,
> > takeshi
> >
> >
> > Core was generated by `/usr/local/sbin/opensips -m 1024'.
> > Program terminated with signal 11, Segmentation fault.
> > [New process 29728]
> > #0  slot_add (_s=0x2b361ce6ac10, _r=0x2b363326ca20) at hslot.c:151
> > 151 *dest = _r;
> > (gdb) bt
> > #0  slot_add (_s=0x2b361ce6ac10, _r=0x2b363326ca20) at hslot.c:151
> > #1  0x2b3619b9ac65 in mem_insert_urecord (_d=0x2b361ce69560,
> > _aor=, _r=0x7fff919cb740)
> > at udomain.c:648
> > #2  0x2b3619b9ad42 in insert_urecord (_d=0x2b361ce69560,
> > _aor=0x7fff919cb958, _r=0x7fff919cb740) at udomain.c:781
> > #3  0x2b3619fb5769 in insert_contacts (_m=0x79b600, _c=0x79c468,
> > _d=0x2b361ce69560, _a=0x7fff919cb958,
> > _sctx=0x7fff919cb950) at save.c:398
> > #4  0x2b3619fb6592 in save (_m=0x79b600, _d=0x2b361ce69560 "",
> > _f=, _s=0x0) at save.c:680
> > #5  0x0040e9e9 in do_action (a=0x78def8, msg=0x79b600) at
> > action.c:967
> > #6  0x004114bc in run_action_list (a=,
> > msg=0x79b600) at action.c:139
> > #7  0x00466b24 in eval_expr (e=0x78dfd0, msg=0x79b600,
> > val=0x0) at route.c:1240
> > #8  0x0046659d in eval_expr (e=0x78e020, msg=0x79b600,
> > val=0x0) at route.c:1553
> > #9  0x00466545 in eval_expr (e=0x78e070, msg=0x79b600,
> > val=0x0) at route.c:1558
> > #10 0x0040d9cd in do_action (a=0x78e2c8, msg=0x79b600) at
> > action.c:689
> > #11 0x004114bc in run_action_list (a=,
> > msg=0x79b600) at action.c:139
> > #12 0x0040fcc3 in do_action (a=0x78e888, msg=0x79b600) at
> > action.c:706
> > #13 0x004114bc in run_action_list (a=,
> > msg=0x79b600) at action.c:139
> > #14 0x00411819 in run_top_route (a=0x78ad18, msg=0x79b600) at
> > action.c:119
> > #15 0x00455ff5 in receive_msg (
> > buf=0x754f20 "REGISTER sip:domain13564.com
> >  SIP/2.0\r\nVia: SIP/2.0/UDP
> > 192.168.2.6:;rport;branch=z9hG4bK-4540-1443964-5\r\nFrom:
> > \"13564\" 
> >  >>;tag=4540SIPpTag001443964\r\nTo:
> > \"13564\"  > rcv_info=0x7fff919ccc10) at receive.c:162
> > #16 0x0049a304 in udp_rcv_loop () at udp_server.c:492
> > #17 0x00429bbd in main (argc=3, argv=) at
> > main.c:818
> > (gdb)
> >
> > 
> >
> > ___
> > Users mailing list
> > Users@lists.opensips.org
> > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> >
>
>
> --
> Bogdan-Andrei Iancu
> www.voice-system.ro
>
>
> ___
> Users mailing list
> Users@lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
___
Users mailing list
Users@lists.opensips.org
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

Re: [OpenSIPS-Users] Segfault during load test

2010-04-12 Thread Bogdan-Andrei Iancu

Hi Takeshi,

Do you still have the core file ?  If yes, could you print  "dest" and 
"*dest" ?

Do you get any previous errors (from the same process) ? like mem errors?

Also, can you reproduce this ?

Regards,
Bogdan

mayamatakeshi wrote:
> Hello,
> I was load testing opensips-1.6 r. 6605
> and I got a seg fault.
> I will try updating to most recent revision of 1.6, but I thought 
> someone would like to take a look at this.
>
> regards,
> takeshi
>
>
> Core was generated by `/usr/local/sbin/opensips -m 1024'.
> Program terminated with signal 11, Segmentation fault.
> [New process 29728]
> #0  slot_add (_s=0x2b361ce6ac10, _r=0x2b363326ca20) at hslot.c:151
> 151 *dest = _r;
> (gdb) bt
> #0  slot_add (_s=0x2b361ce6ac10, _r=0x2b363326ca20) at hslot.c:151
> #1  0x2b3619b9ac65 in mem_insert_urecord (_d=0x2b361ce69560, 
> _aor=, _r=0x7fff919cb740)
> at udomain.c:648
> #2  0x2b3619b9ad42 in insert_urecord (_d=0x2b361ce69560, 
> _aor=0x7fff919cb958, _r=0x7fff919cb740) at udomain.c:781
> #3  0x2b3619fb5769 in insert_contacts (_m=0x79b600, _c=0x79c468, 
> _d=0x2b361ce69560, _a=0x7fff919cb958,
> _sctx=0x7fff919cb950) at save.c:398
> #4  0x2b3619fb6592 in save (_m=0x79b600, _d=0x2b361ce69560 "", 
> _f=, _s=0x0) at save.c:680
> #5  0x0040e9e9 in do_action (a=0x78def8, msg=0x79b600) at 
> action.c:967
> #6  0x004114bc in run_action_list (a=, 
> msg=0x79b600) at action.c:139
> #7  0x00466b24 in eval_expr (e=0x78dfd0, msg=0x79b600, 
> val=0x0) at route.c:1240
> #8  0x0046659d in eval_expr (e=0x78e020, msg=0x79b600, 
> val=0x0) at route.c:1553
> #9  0x00466545 in eval_expr (e=0x78e070, msg=0x79b600, 
> val=0x0) at route.c:1558
> #10 0x0040d9cd in do_action (a=0x78e2c8, msg=0x79b600) at 
> action.c:689
> #11 0x004114bc in run_action_list (a=, 
> msg=0x79b600) at action.c:139
> #12 0x0040fcc3 in do_action (a=0x78e888, msg=0x79b600) at 
> action.c:706
> #13 0x004114bc in run_action_list (a=, 
> msg=0x79b600) at action.c:139
> #14 0x00411819 in run_top_route (a=0x78ad18, msg=0x79b600) at 
> action.c:119
> #15 0x00455ff5 in receive_msg (
> buf=0x754f20 "REGISTER sip:domain13564.com 
>  SIP/2.0\r\nVia: SIP/2.0/UDP 
> 192.168.2.6:;rport;branch=z9hG4bK-4540-1443964-5\r\nFrom: 
> \"13564\"  >;tag=4540SIPpTag001443964\r\nTo: 
> \"13564\"  rcv_info=0x7fff919ccc10) at receive.c:162
> #16 0x0049a304 in udp_rcv_loop () at udp_server.c:492
> #17 0x00429bbd in main (argc=3, argv=) at 
> main.c:818
> (gdb)
>
> 
>
> ___
> Users mailing list
> Users@lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>   


-- 
Bogdan-Andrei Iancu
www.voice-system.ro


___
Users mailing list
Users@lists.opensips.org
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

[OpenSIPS-Users] Segfault during load test

2010-04-02 Thread mayamatakeshi

Hello,
I was load testing opensips-1.6 r. 6605
and I got a seg fault.
I will try updating to most recent revision of 1.6, but I thought someone
would like to take a look at this.

regards,
takeshi


Core was generated by `/usr/local/sbin/opensips -m 1024'.
Program terminated with signal 11, Segmentation fault.
[New process 29728]
#0  slot_add (_s=0x2b361ce6ac10, _r=0x2b363326ca20) at hslot.c:151
151 *dest = _r;
(gdb) bt
#0  slot_add (_s=0x2b361ce6ac10, _r=0x2b363326ca20) at hslot.c:151
#1  0x2b3619b9ac65 in mem_insert_urecord (_d=0x2b361ce69560, _aor=, _r=0x7fff919cb740)
at udomain.c:648
#2  0x2b3619b9ad42 in insert_urecord (_d=0x2b361ce69560,
_aor=0x7fff919cb958, _r=0x7fff919cb740) at udomain.c:781
#3  0x2b3619fb5769 in insert_contacts (_m=0x79b600, _c=0x79c468,
_d=0x2b361ce69560, _a=0x7fff919cb958,
_sctx=0x7fff919cb950) at save.c:398
#4  0x2b3619fb6592 in save (_m=0x79b600, _d=0x2b361ce69560 "", _f=, _s=0x0) at save.c:680
#5  0x0040e9e9 in do_action (a=0x78def8, msg=0x79b600) at
action.c:967
#6  0x004114bc in run_action_list (a=,
msg=0x79b600) at action.c:139
#7  0x00466b24 in eval_expr (e=0x78dfd0, msg=0x79b600, val=0x0) at
route.c:1240
#8  0x0046659d in eval_expr (e=0x78e020, msg=0x79b600, val=0x0) at
route.c:1553
#9  0x00466545 in eval_expr (e=0x78e070, msg=0x79b600, val=0x0) at
route.c:1558
#10 0x0040d9cd in do_action (a=0x78e2c8, msg=0x79b600) at
action.c:689
#11 0x004114bc in run_action_list (a=,
msg=0x79b600) at action.c:139
#12 0x0040fcc3 in do_action (a=0x78e888, msg=0x79b600) at
action.c:706
#13 0x004114bc in run_action_list (a=,
msg=0x79b600) at action.c:139
#14 0x00411819 in run_top_route (a=0x78ad18, msg=0x79b600) at
action.c:119
#15 0x00455ff5 in receive_msg (
buf=0x754f20 "REGISTER sip:domain13564.com SIP/2.0\r\nVia: SIP/2.0/UDP
192.168.2.6:;rport;branch=z9hG4bK-4540-1443964-5\r\nFrom: \"13564\" <
sip:13...@domain13564.com
>;tag=4540SIPpTag001443964\r\nTo:
\"13564\" ) at
main.c:818
(gdb)
___
Users mailing list
Users@lists.opensips.org
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

Re: [OpenSIPS-Users] Segfault during load test

Re: [OpenSIPS-Users] Segfault during load test

Re: [OpenSIPS-Users] Segfault during load test

Re: [OpenSIPS-Users] Segfault during load test

[OpenSIPS-Users] Segfault during load test

5 matches

Site Navigation

Mail list logo

Footer information