Re: [OpenSIPS-Users] opensips 2.2.5 stop responding/processing requests. - Centos 6.5
Hi Guys, Thanks for the responses,appreciate it! In the end it was an overloaded DNS server, which caused a delay in another SBC passing requests to the opensips server, so overloading it. Thanks for the advice everyone. It wasnt OpenSIPS! Jon From: Bogdan-Andrei Iancu <bog...@opensips.org> Sent: 31 October 2017 12:04 To: OpenSIPS users mailling list; Jonathan Hunter Subject: Re: [OpenSIPS-Users] opensips 2.2.5 stop responding/processing requests. - Centos 6.5 Hi Jonathan, It looks like you have kind on task inside OpenSIPS that is using a lot of time/process resources - shortly, this translate into "there are no available worker processes to handle new jobs". * do you have any threshold set to catch any DB/DNS delays ? * have you checked the CPU usage ? * have you monitor the "load:" statistics group ? Regards, Bogdan-Andrei Iancu OpenSIPS Founder and Developer http://www.opensips-solutions.com Home — OpenSIPS Solutions<http://www.opensips-solutions.com/> www.opensips-solutions.com OpenSIPS is a mature Open Source implementation of a SIP server. OpenSIPS is more than a SIP proxy/router as it includes application-level functionalities. On 10/30/2017 07:33 PM, Jonathan Hunter wrote: Hi Guys, Has anyone had the following behaviour with this release of opensips? It has been running fine for a number of months, and then suddenly today the server which was running as the primary suddenly stopped responding to SIP requests, or stopped processing them, or there was a long pause in a sequence completing. So in some cases, INVITES or REGISTER messages were ignored and the initial requests werent always logged in the opensips log file (even though we do this for all messages), however the SIP requests were hitting the server as I was tracing them. In the logs I can see the following warnings we havent seen before; Oct 30 13:31:25 sgw6 VU-SIP-Proxy[11159]: WARNING:core:utimer_ticker: utimer task already scheduled for 9639891650 ms (now 9639891850 ms), it may overlap. Oct 30 13:52:48 sgw6 VU-SIP-Proxy[11159]: WARNING:core:timer_ticker: timer task already scheduled for 9640994180 ms (now 9641175280 ms), it may overlap.. Oct 30 13:52:48 sgw6 VU-SIP-Proxy[11159]: WARNING:core:timer_ticker: timer task already scheduled for 9640980270 ms (now 9641175280 ms), it may overlap.. A restart of the opensips application didnt help, and service was only restored fully when I failed over to the secondary server. Has anyone had these issues before? There are no obvious server level issues, and as I mentioned the logs dont contain anything too unusual. Any help appreciated. Many thanks Jon ___ Users mailing list Users@lists.opensips.org<mailto:Users@lists.opensips.org> http://lists.opensips.org/cgi-bin/mailman/listinfo/users ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Re: [OpenSIPS-Users] opensips 2.2.5 stop responding/processing requests. - Centos 6.5
Hi Jonathan, It looks like you have kind on task inside OpenSIPS that is using a lot of time/process resources - shortly, this translate into "there are no available worker processes to handle new jobs". * do you have any threshold set to catch any DB/DNS delays ? * have you checked the CPU usage ? * have you monitor the "load:" statistics group ? Regards, Bogdan-Andrei Iancu OpenSIPS Founder and Developer http://www.opensips-solutions.com On 10/30/2017 07:33 PM, Jonathan Hunter wrote: Hi Guys, Has anyone had the following behaviour with this release of opensips? It has been running fine for a number of months, and then suddenly today the server which was running as the primary suddenly stopped responding to SIP requests, or stopped processing them, or there was a long pause in a sequence completing. So in some cases, INVITES or REGISTER messages were ignored and the initial requests werent always logged in the opensips log file (even though we do this for all messages), however the SIP requests were hitting the server as I was tracing them. In the logs I can see the following warnings we havent seen before; Oct 30 13:31:25 sgw6 VU-SIP-Proxy[11159]: WARNING:core:utimer_ticker: utimer task already scheduled for 9639891650 ms (now 9639891850 ms), it may overlap. Oct 30 13:52:48 sgw6 VU-SIP-Proxy[11159]: WARNING:core:timer_ticker: timer task already scheduled for 9640994180 ms (now 9641175280 ms), it may overlap.. Oct 30 13:52:48 sgw6 VU-SIP-Proxy[11159]: WARNING:core:timer_ticker: timer task already scheduled for 9640980270 ms (now 9641175280 ms), it may overlap.. A restart of the opensips application didnt help, and service was only restored fully when I failed over to the secondary server. Has anyone had these issues before? There are no obvious server level issues, and as I mentioned the logs dont contain anything too unusual. Any help appreciated. Many thanks Jon ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Re: [OpenSIPS-Users] opensips 2.2.5 stop responding/processing requests. - Centos 6.5
This might help with debugging it. https://blog.opensips.org/2017/09/20/troubleshooting-opensips-script/ [https://blogopensips.files.wordpress.com/2017/09/troubleshooting.jpg]<https://blog.opensips.org/2017/09/20/troubleshooting-opensips-script/> Troubleshooting OpenSIPS script – Drops of wisdom ...<https://blog.opensips.org/2017/09/20/troubleshooting-opensips-script/> blog.opensips.org What makes OpenSIPS such an attractive and powerful SIP solutions is its high level of programmability, thanks to its C-like configuration script. From: Users <users-boun...@lists.opensips.org> on behalf of Jonathan Hunter <hunter...@hotmail.com> Sent: Monday, October 30, 2017 2:25 PM To: OpenSIPS users mailling list Subject: Re: [OpenSIPS-Users] opensips 2.2.5 stop responding/processing requests. - Centos 6.5 Hi Johan, Thanks for the response I appreciate it. We do use radius for accounting and fire all the information to a central server, but it appears all that was fine and connectivity/configuration was good at the time of the issue. Anywhere in particular it could break things? I have seen it when opensips cant reach radius it starts to be none responsive, is that a scenario you have seen? Many thanks Jon From: Users <users-boun...@lists.opensips.org> on behalf of Johan De Clercq <jo...@democon.be> Sent: 30 October 2017 17:53 To: OpenSIPS users mailling list Subject: Re: [OpenSIPS-Users] opensips 2.2.5 stop responding/processing requests. - Centos 6.5 If you use acc with radius, check radius config. Br On 30 Oct 2017 18:33, "Jonathan Hunter" <hunter...@hotmail.com<mailto:hunter...@hotmail.com>> wrote: Hi Guys, Has anyone had the following behaviour with this release of opensips? It has been running fine for a number of months, and then suddenly today the server which was running as the primary suddenly stopped responding to SIP requests, or stopped processing them, or there was a long pause in a sequence completing. So in some cases, INVITES or REGISTER messages were ignored and the initial requests werent always logged in the opensips log file (even though we do this for all messages), however the SIP requests were hitting the server as I was tracing them. In the logs I can see the following warnings we havent seen before; Oct 30 13:31:25 sgw6 VU-SIP-Proxy[11159]: WARNING:core:utimer_ticker: utimer task already scheduled for 9639891650 ms (now 9639891850 ms), it may overlap. Oct 30 13:52:48 sgw6 VU-SIP-Proxy[11159]: WARNING:core:timer_ticker: timer task already scheduled for 9640994180 ms (now 9641175280 ms), it may overlap.. Oct 30 13:52:48 sgw6 VU-SIP-Proxy[11159]: WARNING:core:timer_ticker: timer task already scheduled for 9640980270 ms (now 9641175280 ms), it may overlap.. A restart of the opensips application didnt help, and service was only restored fully when I failed over to the secondary server. Has anyone had these issues before? There are no obvious server level issues, and as I mentioned the logs dont contain anything too unusual. Any help appreciated. Many thanks Jon ___ Users mailing list Users@lists.opensips.org<mailto:Users@lists.opensips.org> http://lists.opensips.org/cgi-bin/mailman/listinfo/users<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.opensips.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fusers=02%7C01%7Crmundkowsky%40ets.org%7Cdd641319431d434b6ffa08d51fc3da41%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636449848404092587=ozDFP%2FN72W%2F0aWKRW1r7biEv6Y%2BT9fIzWZ3u06hSKHQ%3D=0> This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited. Thank you for your compliance. ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Re: [OpenSIPS-Users] opensips 2.2.5 stop responding/processing requests. - Centos 6.5
Hi Johan, Thanks for the response I appreciate it. We do use radius for accounting and fire all the information to a central server, but it appears all that was fine and connectivity/configuration was good at the time of the issue. Anywhere in particular it could break things? I have seen it when opensips cant reach radius it starts to be none responsive, is that a scenario you have seen? Many thanks Jon From: Users <users-boun...@lists.opensips.org> on behalf of Johan De Clercq <jo...@democon.be> Sent: 30 October 2017 17:53 To: OpenSIPS users mailling list Subject: Re: [OpenSIPS-Users] opensips 2.2.5 stop responding/processing requests. - Centos 6.5 If you use acc with radius, check radius config. Br On 30 Oct 2017 18:33, "Jonathan Hunter" <hunter...@hotmail.com<mailto:hunter...@hotmail.com>> wrote: Hi Guys, Has anyone had the following behaviour with this release of opensips? It has been running fine for a number of months, and then suddenly today the server which was running as the primary suddenly stopped responding to SIP requests, or stopped processing them, or there was a long pause in a sequence completing. So in some cases, INVITES or REGISTER messages were ignored and the initial requests werent always logged in the opensips log file (even though we do this for all messages), however the SIP requests were hitting the server as I was tracing them. In the logs I can see the following warnings we havent seen before; Oct 30 13:31:25 sgw6 VU-SIP-Proxy[11159]: WARNING:core:utimer_ticker: utimer task already scheduled for 9639891650 ms (now 9639891850 ms), it may overlap. Oct 30 13:52:48 sgw6 VU-SIP-Proxy[11159]: WARNING:core:timer_ticker: timer task already scheduled for 9640994180 ms (now 9641175280 ms), it may overlap.. Oct 30 13:52:48 sgw6 VU-SIP-Proxy[11159]: WARNING:core:timer_ticker: timer task already scheduled for 9640980270 ms (now 9641175280 ms), it may overlap.. A restart of the opensips application didnt help, and service was only restored fully when I failed over to the secondary server. Has anyone had these issues before? There are no obvious server level issues, and as I mentioned the logs dont contain anything too unusual. Any help appreciated. Many thanks Jon ___ Users mailing list Users@lists.opensips.org<mailto:Users@lists.opensips.org> http://lists.opensips.org/cgi-bin/mailman/listinfo/users ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users
Re: [OpenSIPS-Users] opensips 2.2.5 stop responding/processing requests. - Centos 6.5
If you use acc with radius, check radius config. Br On 30 Oct 2017 18:33, "Jonathan Hunter"wrote: > Hi Guys, > > Has anyone had the following behaviour with this release of opensips? > > It has been running fine for a number of months, and then suddenly today > the server which was running as the primary suddenly stopped responding to > SIP requests, or stopped processing them, or there was a long pause in a > sequence completing. > > So in some cases, INVITES or REGISTER messages were ignored and the > initial requests werent always logged in the opensips log file (even though > we do this for all messages), however the SIP requests were hitting the > server as I was tracing them. > > In the logs I can see the following warnings we havent seen before; > > Oct 30 13:31:25 sgw6 VU-SIP-Proxy[11159]: WARNING:core:utimer_ticker: > utimer task already scheduled for 9639891650 ms (now 9639891850 > ms), it may overlap. > Oct 30 13:52:48 sgw6 VU-SIP-Proxy[11159]: WARNING:core:timer_ticker: timer > task already scheduled for 9640994180 ms (now 9641175280 ms), it > may overlap.. > Oct 30 13:52:48 sgw6 VU-SIP-Proxy[11159]: WARNING:core:timer_ticker: timer > task already scheduled for 9640980270 ms (now 9641175280 ms), it > may overlap.. > > A restart of the opensips application didnt help, and service was only > restored fully when I failed over to the secondary server. > > Has anyone had these issues before? There are no obvious server level > issues, and as I mentioned the logs dont contain anything too unusual. > > Any help appreciated. > > Many thanks > > Jon > > > ___ > Users mailing list > Users@lists.opensips.org > http://lists.opensips.org/cgi-bin/mailman/listinfo/users > > ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users
[OpenSIPS-Users] opensips 2.2.5 stop responding/processing requests. - Centos 6.5
Hi Guys, Has anyone had the following behaviour with this release of opensips? It has been running fine for a number of months, and then suddenly today the server which was running as the primary suddenly stopped responding to SIP requests, or stopped processing them, or there was a long pause in a sequence completing. So in some cases, INVITES or REGISTER messages were ignored and the initial requests werent always logged in the opensips log file (even though we do this for all messages), however the SIP requests were hitting the server as I was tracing them. In the logs I can see the following warnings we havent seen before; Oct 30 13:31:25 sgw6 VU-SIP-Proxy[11159]: WARNING:core:utimer_ticker: utimer task already scheduled for 9639891650 ms (now 9639891850 ms), it may overlap. Oct 30 13:52:48 sgw6 VU-SIP-Proxy[11159]: WARNING:core:timer_ticker: timer task already scheduled for 9640994180 ms (now 9641175280 ms), it may overlap.. Oct 30 13:52:48 sgw6 VU-SIP-Proxy[11159]: WARNING:core:timer_ticker: timer task already scheduled for 9640980270 ms (now 9641175280 ms), it may overlap.. A restart of the opensips application didnt help, and service was only restored fully when I failed over to the secondary server. Has anyone had these issues before? There are no obvious server level issues, and as I mentioned the logs dont contain anything too unusual. Any help appreciated. Many thanks Jon ___ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users