I've looked at the logs that you sent. It is clearly a DOS attack.
The attacker is sending you several thousands of REGISTER request and
gradually depletes your system resource.
On Monday, 27 September, 2010 03:39 PM, Tran, Ly V. wrote:
Is it possible that sipxproxy is failing because we are being DOS
attacked via VOIP scanned? Looking at the sipxproxy log leading up to
it failing, I see alot of scans coming from a Chinese IP address and
*nUser-Agent: friendly-scanner.* There are a bunch of user names like
vlad, yvette, takashi etc. none of which are legit users in our
system. Are these logs telling us we are hacked and our system
compromised. CDR doesn't show any calls to the numbers mentioned in
this log, but then again we were being charged recently by our ITSP
for some un-authorized calls that didn't show up in the SipX CDRs.
"2010-09-26T06:59:45.407594Z":7925:OUTGOING:INFO:sipx.mydomain.com:SipRouter-11:B65BBB90:SipXProxy:"SipUserAgent::sendUdp
UDP SIP User Agent sent message:\n----Local Host:192.168.2.2---- Port:
5060----\n----Remote Host:24.100.100.100---- Port: 5060----\nREGISTER
sip:[email protected] SIP/2.0\r\nRecord-Route:
<sip:24.100.100.100:5060;lr;sipXecs-rs=%2Aauth%7E.%2Afrom%7ENzY2YzYxNjQwMTMxMzMzNzM1MzEzNTM2MzczNjMz%21b8fb0418f51986ff3263ed6be1351e37>\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-05bcuHDJHxHv1FOLcZUPZrReVA\r\nVia:
SIP/2.0/UDP
222.73.177.248:5140;branch=z9hG4bK-774575936;rport=5140\r\nContent-Length:
0\r\nFrom: \"vlad\"<sip:[email protected]>;
tag=766c61640131333735313536373633\r\nAccept:
application/sdp\r\nUser-Agent: friendly-scanner\r\nTo:
\"vlad\"<sip:[email protected]>\r\nContact:
<sip:[email protected];x-sipX-nonat>\r\nCseq: 1 REGISTER\r\nCall-Id:
1737450884\r\nMax-Forwards: 20\r\nDate: Sun, 26 Sep 2010 06:59:44
GMT\r\n\r\n--------------------END--------------------"
"2010-09-26T06:59:45.408250Z":7926:INCOMING:INFO:sipx.mydomain.com:SipClientUdp-8:B78FCB90:SipXProxy:"Read
SIP message:\n----Local Host:192.168.2.2---- Port:
5060----\n----Remote Host:192.168.2.254---- Port: 5060----\nREGISTER
sip:[email protected] SIP/2.0\r\nRecord-Route:
<sip:24.100.100.100:5060;lr;sipXecs-rs=%2Aauth%7E.%2Afrom%7ENzY2YzYxNjQwMTMxMzMzNzM1MzEzNTM2MzczNjMz%21b8fb0418f51986ff3263ed6be1351e37>\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-05bcuHDJHxHv1FOLcZUPZrReVA\r\nVia:
SIP/2.0/UDP
222.73.177.248:5140;branch=z9hG4bK-774575936;rport=5140\r\nContent-Length:
0\r\nFrom: \"vlad\"<sip:[email protected]>;
tag=766c61640131333735313536373633\r\nAccept:
application/sdp\r\nUser-Agent: friendly-scanner\r\nTo:
\"vlad\"<sip:[email protected]>\r\nContact:
<sip:[email protected];x-sipX-nonat>\r\nCseq: 1 REGISTER\r\nCall-Id:
1737450884\r\nMax-Forwards: 20\r\nDate: Sun, 26 Sep 2010 06:59:44
GMT\r\n\r\n====================END===================="
"2010-09-26T06:59:45.408352Z":7927:OUTGOING:INFO:sipx.mydomain.com:SipUserAgent-2:B66BCB90:SipXProxy:"SipUserAgent::sendUdp
resend 1 of UDP message\nUDP SIP User Agent sent message:\n----Local
Host:192.168.2.2---- Port: 5060----\n----Remote
Host:24.100.100.100---- Port: 5060----\nREGISTER
sip:[email protected] SIP/2.0\r\nRecord-Route:
<sip:24.100.100.100:5060;lr;sipXecs-rs=%2Aauth%7E.%2Afrom%7ENzQ2MTZiNjE3MzY4NjkwMTM0MzIzMDM1MzkzNDMyMzYzNTM1.900_ntap%2ACrT%7EMTkyLjE2OC4yLjI1NDo1MDYwO3RyYW5zcG9ydD11ZHA%60%21818c3dfdaab5833a8998bb50e63806a1>\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-0583FNk1cdV5C0srSd`_nPLpzw\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-034cqNBFoWDd98mdTI5YObOJqw;received=192.168.2.254;rport=5060\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-01f9R``DgdmJ4A626ty_t`Tnnw;received=192.168.2.254;rport=5060\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-0130rUP`gvrZOk4U4j58o53IKg;received=192.168.2.254;rport=5060\r\nVia:
SIP/2.0/UDP
222.73.177.248:5140;branch=z9hG4bK-1796203228;rport=5140\r\nContent-Length:
0\r\nFrom: \"takashi\"<sip:[email protected]>;
tag=74616b617368690134323035393432363535\r\nAccept:
application/sdp\r\nUser-Agent: friendly-scanner\r\nTo:
\"takashi\"<sip:[email protected]>\r\nContact:
<sip:[email protected]:5060;x-sipX-privcontact=24.100.100.100>\r\nCseq:
1 REGISTER\r\nCall-Id: 1910971822\r\nMax-Forwards: 17\r\nDate: Sun, 26
Sep 2010 06:59:43 GMT\r\nX-Sipx-Spiral:
true\r\n\r\n--------------------END--------------------"
"2010-09-26T06:59:45.409490Z":7928:INCOMING:INFO:sipx.mydomain.com:SipClientUdp-8:B78FCB90:SipXProxy:"Read
SIP message:\n----Local Host:192.168.2.2---- Port:
5060----\n----Remote Host:222.73.177.248---- Port: 5140----\nREGISTER
sip:[email protected] SIP/2.0\r\nVia: SIP/2.0/UDP
222.73.177.248:5140;branch=z9hG4bK-258420628;rport\r\nContent-Length:
0\r\nFrom: \"zachary\"<sip:[email protected]>;
tag=7a6163686172790131383135313630333435\r\nAccept:
application/sdp\r\nUser-Agent: friendly-scanner\r\nTo:
\"zachary\"<sip:[email protected]>\r\nContact:
sip:[email protected]\r\ncseq: 1 REGISTER\r\nCall-ID:
779215016\r\nMax-Forwards:
70\r\n\r\n====================END===================="
"2010-09-26T06:59:45.410316Z":7929:AUTH:INFO:sipx.mydomain.com:SipRouter-11:B65BBB90:SipXProxy:"EnforceAuthRules[400_authrules]::authorizeAndModify
no permission required for call 101914142"
"2010-09-26T06:59:45.410391Z":7930:AUTH:INFO:sipx.mydomain.com:SipRouter-11:B65BBB90:SipXProxy:"SipProxy::proxyMessage
authoritative authorization decision is ALLOW by 400_authrules for
101914142"
"2010-09-26T06:59:45.410464Z":7931:INCOMING:INFO:sipx.mydomain.com:SipClientUdp-8:B78FCB90:SipXProxy:"Read
SIP message:\n----Local Host:192.168.2.2---- Port:
5060----\n----Remote Host:192.168.2.254---- Port: 5060----\nREGISTER
sip:[email protected] SIP/2.0\r\nRecord-Route:
<sip:24.100.100.100:5060;lr;sipXecs-rs=%2Aauth%7E.%2Afrom%7ENzQ2MTZiNjE3MzY4NjkwMTM0MzIzMDM1MzkzNDMyMzYzNTM1.900_ntap%2ACrT%7EMTkyLjE2OC4yLjI1NDo1MDYwO3RyYW5zcG9ydD11ZHA%60%21818c3dfdaab5833a8998bb50e63806a1>\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-0583FNk1cdV5C0srSd`_nPLpzw\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-034cqNBFoWDd98mdTI5YObOJqw;received=192.168.2.254;rport=5060\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-01f9R``DgdmJ4A626ty_t`Tnnw;received=192.168.2.254;rport=5060\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-0130rUP`gvrZOk4U4j58o53IKg;received=192.168.2.254;rport=5060\r\nVia:
SIP/2.0/UDP
222.73.177.248:5140;branch=z9hG4bK-1796203228;rport=5140\r\nContent-Length:
0\r\nFrom: \"takashi\"<sip:[email protected]>;
tag=74616b617368690134323035393432363535\r\nAccept:
application/sdp\r\nUser-Agent: friendly-scanner\r\nTo:
\"takashi\"<sip:[email protected]>\r\nContact:
<sip:[email protected]:5060;x-sipX-privcontact=24.100.100.100>\r\nCseq:
1 REGISTER\r\nCall-Id: 1910971822\r\nMax-Forwards: 17\r\nDate: Sun, 26
Sep 2010 06:59:43 GMT\r\nX-Sipx-Spiral:
true\r\n\r\n====================END===================="
"2010-09-26T06:59:45.411609Z":7932:SIP:WARNING:sipx.mydomain.com:SipRouter-11:B65BBB90:SipXProxy:"SipUserAgent::send
REGISTER request matches existing transaction"
"2010-09-26T06:59:45.411693Z":7933:OUTGOING:INFO:sipx.mydomain.com:SipClientUdp-8:B78FCB90:SipXProxy:"SipUserAgent::sendUdp
UDP SIP User Agent sent message:\n----Local Host:192.168.2.2---- Port:
5060----\n----Remote Host:192.168.2.254---- Port: 5060----\nSIP/2.0
100 Trying\r\nFrom: \"takashi\"<sip:[email protected]>;
tag=74616b617368690134323035393432363535\r\nTo:
\"takashi\"<sip:[email protected]>\r\nCall-Id:
1910971822\r\nCseq: 1 REGISTER\r\nVia: SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-0583FNk1cdV5C0srSd`_nPLpzw;received=192.168.2.254;rport=5060\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-034cqNBFoWDd98mdTI5YObOJqw;received=192.168.2.254;rport=5060\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-01f9R``DgdmJ4A626ty_t`Tnnw;received=192.168.2.254;rport=5060\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-0130rUP`gvrZOk4U4j58o53IKg;received=192.168.2.254;rport=5060\r\nVia:
SIP/2.0/UDP
222.73.177.248:5140;branch=z9hG4bK-1796203228;rport=5140\r\nContent-Length:
0\r\n\r\n--------------------END--------------------"
"2010-09-26T06:59:45.412193Z":7934:INCOMING:INFO:sipx.mydomain.com:SipClientUdp-8:B78FCB90:SipXProxy:"Read
SIP message:\n----Local Host:192.168.2.2---- Port:
5060----\n----Remote Host:24.100.100.100---- Port: 5060----\nSIP/2.0
100 Trying\r\nFrom: \"takashi\"<sip:[email protected]>;
tag=74616b617368690134323035393432363535\r\nTo:
\"takashi\"<sip:[email protected]>\r\nCall-Id:
1910971822\r\nCseq: 1 REGISTER\r\nVia: SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-0583FNk1cdV5C0srSd`_nPLpzw;received=192.168.2.254;rport=5060\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-034cqNBFoWDd98mdTI5YObOJqw;received=192.168.2.254;rport=5060\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-01f9R``DgdmJ4A626ty_t`Tnnw;received=192.168.2.254;rport=5060\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-0130rUP`gvrZOk4U4j58o53IKg;received=192.168.2.254;rport=5060\r\nVia:
SIP/2.0/UDP
222.73.177.248:5140;branch=z9hG4bK-1796203228;rport=5140\r\nContent-Length:
0\r\n\r\n====================END===================="
"2010-09-26T06:59:45.412731Z":7935:OUTGOING:INFO:sipx.mydomain.com:SipUserAgent-2:B66BCB90:SipXProxy:"SipUserAgent::sendUdp
resend 1 of UDP message\nUDP SIP User Agent sent message:\n----Local
Host:192.168.2.2---- Port: 5060----\n----Remote
Host:24.100.100.100---- Port: 5060----\nREGISTER
sip:[email protected] SIP/2.0\r\nRecord-Route:
<sip:24.100.100.100:5060;lr;sipXecs-rs=%2Aauth%7E.%2Afrom%7ENzQ2ZjczNjg2OTc0NjU3MjAxMzEzNDMwMzUzNzM3MzUzNzM1Mzc%60%21d41d9ea6f70a9790ec9465517f52737a>\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-0586w7oxh5t_nkbb_M3tVpuIdg\r\nVia:
SIP/2.0/UDP
24.100.100.100:5060;branch=z9hG4bK-XX-034f_9QjVu8VLfz_t4RfIAXUAA;received=192.168.2.254;rport=5060\r\nVia:
SIP/2.0/UDP
222.73.177.248:5140;branch=z9hG4bK-2598216935;rport=5140\r\nContent-Length:
0\r\nFrom: \"toshiter\"<sip:[email protected]>;
tag=746f7368697465720131343035373735373537\r\nAccept:
application/sdp\r\nUser-Agent: friendly-scanner\r\nTo:
\"toshiter\"<sip:[email protected]>\r\nContact:
<sip:[email protected]:5060;x-sipX-privcontact=24.100.100.100>\r\nCseq:
1 REGISTER\r\nCall-Id: 421959325\r\nMax-Forwards: 19\r\nDate: Sun, 26
Sep 2010 06:59:43 GMT\r\nX-Sipx-Spiral:
true\r\n\r\n--------------------END--------------------"
"2010-09-26T06:59:45.412930Z":7936:SIP:ERR:sipx.mydomain.com:SipRouter-11:B65BBB90:SipXProxy:"SipUserAgent::send
outgoing call 1"
------------------------------------------------------------------------
*From:* Joegen Baclor
*Sent:* Sun 9/26/2010 6:50 PM
*To:* Discussion list for users of sipXecs software
*Cc:* Tran, Ly V.
*Subject:* Re: [sipx-users] Random SipxProxy Problem
Looks like some sort of deadlock on the message queue. Would you be
able to provide the log that shows the last successful transactions up
to the point when this error starts appearing?
Joegen
On Sunday, 26 September, 2010 10:22 PM, Tran, Ly V. wrote:
We have a SipX system with the most current build that was upgraded
from 4.04. It runs fine for a few weeks at a time, but ocassionally
all the phones stops working and shows up as Expired. The system has
8Gb of RAM and during normal operation, barely uses 1.2Gb, but when
all the phones goes into expiration.. checking RAM usage; it is above
2Gb. While we don't think this has anything to do with it.
Restarting the SipxProxy service, the phones starts working again and
RAM usage drops. This problem comes up every few weeks. Sometimes
twice a week. This morning, it happened again for the second time
since last Wednesday.
Something is causing SipxProxy to fail and can not figure it out.
Does anyone have an idea? This is what we are seeing. The alarms
shows the following:
SPX00002 "Process 'SIPXProxy' stopped unexpectedly. Attempting to
restart the process." WARNING 9/26/10 1:59 AM
I restart the SipxProxy from the GUI and RAM usage drops back down
and all the phones are registering and working again.
SipxProxy Log shows alot of these errors during the this time period:
"2010-09-26T09:15:24.956486Z":227913:OUTGOING:INFO:sipx.mydomain.com:SipClientUdp-8:B78FCB90:SipXProxy:"SipUserAgent::sendUdp
UDP SIP User Agent sent message:\n----Local Host:192.168.2.2----
Port: 5060----\n----Remote Host:192.168.2.248---- Port:
5060----\nSIP/2.0 100 Trying\r\nFrom:
\"Project\"sip:[email protected];tag=94bd83a0-c0a802f8-13c4-4d15a5-645c3fd0-4d15a5\r\nTo:
\"Project\"sip:[email protected]\r\ncall-id:
[email protected]\r\ncseq: 1
REGISTER\r\nVia: SIP/2.0/UDP
192.168.2.248:5060;branch=z9hG4bK-4d15a5-2d1c9008-435ef219\r\nContent-Length:
0\r\n\r\n--------------------END--------------------"
"2010-09-26T09:15:25.184203Z":227914:INCOMING:INFO:sipx.mydomain.com:SipClientUdp-8:B78FCB90:SipXProxy:"Read
SIP message:\n----Local Host:192.168.2.2---- Port:
5060----\n----Remote Host:173.70.144.20---- Port: 5060----\nREGISTER
sip:mydomain.com SIP/2.0\r\nVia: SIP/2.0/UDP
192.168.8.15:5060;branch=z9hG4bK2f4ea6a2\r\nFrom:
sip:[email protected];tag=001da21a482c099824f60b3b-750296f4\r\nTo:
sip:[email protected]\r\ncall-id:
[email protected]\r\nmax-forwards:
70\r\nCSeq: 636 REGISTER\r\nUser-Agent: Cisco-CP7960G/8.0\r\nContact:
sip:[email protected]:5060;transport=udp;+sip.instance=\"<urn:uuid:00000000-0000-0000-0000-001da21a482c>\";+u.sip!model.ccm.cisco.com=\"7\"\r\nContent-Length:
0\r\nExpires: 3600\r\n\r\n====================END===================="
"2010-09-26T09:15:25.184899Z":227915:KERNEL:ERR:sipx.mydomain.com:SipClientUdp-8:B78FCB90:SipXProxy:"OsMsgQShared::doSendCore
message send failed for queue 'SipRouter-11' - no room, ret = 9"
"2010-09-26T09:15:25.185174Z":227916:SIP:CRIT:sipx.mydomain.com:SipClientUdp-8:B78FCB90:SipXProxy:"SipUserAgent::queueMessageToInterestedObservers
send failed with status 12 (numMsgs = 2000, maxMsgs = 2000)"
"2010-09-26T09:15:25.185194Z":227917:SIP:CRIT:sipx.mydomain.com:SipClientUdp-8:B78FCB90:SipXProxy:"SipUserAgent::queueMessageToInterestedObservers
send failed to queue named 'SipRouter-11'"
"2010-09-26T09:15:25.185215Z":227918:SIP:CRIT:sipx.mydomain.com:SipClientUdp-8:B78FCB90:SipXProxy:"SipUserAgent::queueMessageToInterestedObservers
observerQueue 0x87fc8ac, observerData (nil), SIP method '',
wantsRequests 1, wantsResponses 0, wantsIncoming 1, wantsOutGoing 0,
eventName '', SipSession (nil)"
"2010-09-26T09:15:25.185268Z":227919:SIP:CRIT:sipx.mydomain.com:SipClientUdp-8:B78FCB90:SipXProxy:"SipUserAgent::queueMessageToInterestedObservers
failed message is: REGISTER sip:mydomain.com SIP/2.0\r\nVia:
SIP/2.0/UDP
192.168.8.15:5060;branch=z9hG4bK2f4ea6a2;received=173.70.144.20;rport=5060\r\nFrom:
sip:[email protected];tag=001da21a482c099824f60b3b-750296f4\r\nTo:
sip:[email protected]\r\ncall-id:
[email protected]\r\nmax-forwards:
70\r\nCseq: 636 REGISTER\r\nUser-Agent: Cisco-CP7960G/8.0\r\nContact:
sip:[email protected]:5060;transport=udp;+sip.instance=\"<urn:uuid:00000000-0000-0000-0000-001da21a482c>\";+u.sip!model.ccm.cisco.com=\"7\"\r\nContent-Length:
0\r\nExpires: 3600\r\nDate: Sun, 26 Sep 2010 09:15:25 GMT"
Thanks,
Ly Tran
_______________________________________________
sipx-users mailing list
[email protected]
List Archive:http://list.sipfoundry.org/archive/sipx-users/
_______________________________________________
sipx-users mailing list
[email protected]
List Archive: http://list.sipfoundry.org/archive/sipx-users/
