Hi Bogdan, Thanks for your answer, I've checked my configs and yes, for some reason I had tcp_async off!!! I will definitely switch it on for now and then give it a try!!! Can't believe I missed that one!!!
Best regards, Yury. On Wed, Sep 14, 2022 at 10:58 PM Bogdan-Andrei Iancu <bog...@opensips.org> wrote: > Hi Yury, > > You need to check the TCP setting and to be sure your OpenSIPS will (1) > not try to perform TCP connect against destination known not to be able to > accept (like TCP/WS end points behind NAT) - see the tcp_no_new_conn_bflag > [1] - or (2) not block for long time while attempting a connect - see the > tcp_connect_timeout [2] or consider enabling async [3]. > > [1] > https://www.opensips.org/Documentation/Script-CoreParameters-3-2#tcp_no_new_conn_bflag > [2] > https://www.opensips.org/Documentation/Script-CoreParameters-3-2#tcp_connect_timeout > [3] https://opensips.org/html/docs/modules/3.2.x/proto_tcp.html#idp168992 > > Regards, > > Bogdan-Andrei Iancu > > OpenSIPS Founder and Developer > https://www.opensips-solutions.com > OpenSIPS Summit 27-30 Sept 2022, Athens > https://www.opensips.org/events/Summit-2022Athens/ > > On 9/13/22 12:01 PM, Yury Kirsanov wrote: > > Hi Bogdan, > Thanks for this update, but it looks like I can't check autoscaler because > of this first issue with blocking TCP connect. Is there a way to resolve > it? Am I doing something wrong? Or is that something to do with OpenSIPS > code? As yes, you're right, as soon as I restart OpenSIPS having a lot of > SIP devices trying to connect to it - it goes crazy, starts to consume > memory and stops to forward packets sitting there at 100% load until it > runs out of memory and segfaults. Sometimes I can't even restart it to come > to normal state to make it work, it just loops into same crash whatever I > try to do. > > I've compiled OpenSIPS 3.3.1 with your patch and was able to start it but > not sure, maybe I was just lucky this time. > > What should I do? Thanks! > > Best regards, > Yury. > > On Tue, 13 Sept 2022, 18:56 Bogdan-Andrei Iancu, <bog...@opensips.org> > wrote: > >> Hi Yury, >> >> it looks like you some multiple issues, overlapping here. The traps you >> sent here have nothing to do with the auto-scaling, but with a blocking TCP >> connect for SIP - most of the procs get blocked into a sync TCP connect. >> >> Regards, >> >> Bogdan-Andrei Iancu >> >> OpenSIPS Founder and Developer >> https://www.opensips-solutions.com >> OpenSIPS Summit 27-30 Sept 2022, Athens >> https://www.opensips.org/events/Summit-2022Athens/ >> >> On 9/12/22 4:39 PM, Yury Kirsanov wrote: >> >> Hi Bogdan, >> I've applied the patch (had to find where to apply it manually for 3.2.8 >> downloaded from Web page, line 1568 instead of 1652) and restarted the >> server with only about 300-350 SIP devices and immediately got into same >> issue. I'm attaching two GDB dumps made within several minutes from each >> other. Autoscale was now OFF, please see my previous message as currently >> for some reason I'm experiencing lockups even when it's off :( >> >> >> Best regards, >> Yury. >> >> On Mon, Sep 12, 2022 at 7:48 PM Bogdan-Andrei Iancu <bog...@opensips.org> >> wrote: >> >>> Hi Yuri, >>> >>> Could you give this patch a try? it should fix the blocking you >>> experience (it should apply on 3.2 too). >>> >>> Best regards, >>> >>> Bogdan-Andrei Iancu >>> >>> OpenSIPS Founder and Developer >>> https://www.opensips-solutions.com >>> OpenSIPS Summit 27-30 Sept 2022, Athens >>> https://www.opensips.org/events/Summit-2022Athens/ >>> >>> On 9/7/22 2:54 PM, Bogdan-Andrei Iancu wrote: >>> >>> Hi Yury, >>> >>> Thanks for the details info here - let me do a review of some code and >>> run some tests, as at this point I have a good idea on the direction to dig >>> into. >>> >>> I will update here. >>> >>> Best regards, >>> >>> Bogdan-Andrei Iancu >>> >>> OpenSIPS Founder and Developer >>> https://www.opensips-solutions.com >>> OpenSIPS Summit 27-30 Sept 2022, Athens >>> https://www.opensips.org/events/Summit-2022Athens/ >>> >>> On 9/6/22 11:24 AM, Yury Kirsanov wrote: >>> >>> Hi Bogdan, >>> Yes, I'm listening on all types of sockets including UDP, TCP and TLS on >>> the outside public interface and then forward traffic into internal LAN via >>> UDP only. >>> >>> Previously it was getting stuck quite easily, now I had to wait for a >>> while before this actually happened. I've routed part of my customers to >>> this server to obtain this result so I will have to do that again. >>> >>> As soon as I see one of the processes stuck I'll dot the trap command >>> and send you all the details including processes load, ps output and so on. >>> >>> For now I had to switch autoscaling off and just create many listeners. >>> Do I understand correctly that I need to restart OpenSIPS in order to apply >>> autoscaling profiles and reload-routes is not sufficient? >>> >>> Also, do I need separate UDP profiles for public and private interfaces? >>> And do I need to apply autoscaling profile just to a socket or I need to >>> specify udp or tcp_workers with autoscaler too? >>> >>> Thanks and best regards, >>> Yury. >>> >>> On Tue, 6 Sept 2022, 18:18 Bogdan-Andrei Iancu, <bog...@opensips.org> >>> wrote: >>> >>>> Hi Yury, >>>> >>>> Thanks for the info. I see that the stuck process (24) is an >>>> auto-scalled one (based on its id). Do you have SIP traffic from UDP to TCP >>>> or doing some HEP capturing for SIP ? I saw a recent similar report where a >>>> UDP auto-scalled worked got stuck when trying to do some communication with >>>> the TCP main/manager process (in order to handle a TCP operation). >>>> >>>> BTW, any chance to do a "opensips-cli -x trap" when you have that stuck >>>> process, just to see where is it stuck? and is it hard to reproduce? as I >>>> may ask you to extract some information from the running process.... >>>> >>>> Regards, >>>> >>>> Bogdan-Andrei Iancu >>>> >>>> OpenSIPS Founder and Developer >>>> https://www.opensips-solutions.com >>>> OpenSIPS Summit 27-30 Sept 2022, Athens >>>> https://www.opensips.org/events/Summit-2022Athens/ >>>> >>>> On 9/3/22 6:54 PM, Yury Kirsanov wrote: >>>> >>> >>> >>> _______________________________________________ >>> Users mailing >>> listUsers@lists.opensips.orghttp://lists.opensips.org/cgi-bin/mailman/listinfo/users >>> >>> >>> >> >
_______________________________________________ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users
