RE: anyone else seeing very long AS paths?
bgp maxas-limit has a default value of 75 if you don't include it explicitly in the config so in this case it wouldn't have made much of a difference. L. On Mon, 16 Feb 2009, Jon Lewis wrote: > On Mon, 16 Feb 2009, John van Oppen wrote: > > > Yep we saw the same, every customer with old IOS had their sessions die > > to us at the same time... That always makes for an interesting time > > when watching the NMS system... > > Is there a reason you don't use something like "bgp maxas-limit NN" on > your transit sessions? > > We saw this too, but it stopped at our transit routers. There was > actually another a few days ago. > > Feb 13 18:46:07: %BGP-6-ASPATH: Long AS path 4323 1299 12887 12741 39412 > 39625 39625 39625 39625 39625 39625 39625 39625 39625 39625 39625 39625 > 39625 39625 39625 39625 39625 39625 39625 39625 39625 39625 39625 39625 > 39625 39625 39625 39625... > > Feb 16 11:24:53: %BGP-6-ASPATH: Long AS path 4323 3257 29113 47868 47868 > 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 > 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 > 47868 47868 47868 47868... > > > -- > Jon Lewis | I route > Senior Network Engineer | therefore you are > Atlantic Net| > _ http://www.lewis.org/~jlewis/pgp for PGP public key_ >
RE: anyone else seeing very long AS paths?
http://www.cisco.com/en/US/docs/ios/12_2/iproute/command/reference/1rfbgp1.html#wp1254976 Defaults The default value in Cisco IOS software for the number argument is 75. On Mon, 16 Feb 2009, Jon Lewis wrote: > Why do you say that? I counted 78 instances of 47868 in this long > as-path, and our maxas-limit settings did trigger and reject these. > > On Mon, 16 Feb 2009, Leland E. Vandervort wrote: > > > > > bgp maxas-limit has a default value of 75 if you don't include it > > explicitly in the config so in this case it wouldn't have made much of a > > difference. > > > > L. > > > > > > On Mon, 16 Feb 2009, Jon Lewis wrote: > > > >> On Mon, 16 Feb 2009, John van Oppen wrote: > >> > >>> Yep we saw the same, every customer with old IOS had their sessions die > >>> to us at the same time... That always makes for an interesting time > >>> when watching the NMS system... > >> > >> Is there a reason you don't use something like "bgp maxas-limit NN" on > >> your transit sessions? > >> > >> We saw this too, but it stopped at our transit routers. There was > >> actually another a few days ago. > >> > >> Feb 13 18:46:07: %BGP-6-ASPATH: Long AS path 4323 1299 12887 12741 39412 > >> 39625 39625 39625 39625 39625 39625 39625 39625 39625 39625 39625 39625 > >> 39625 39625 39625 39625 39625 39625 39625 39625 39625 39625 39625 39625 > >> 39625 39625 39625 39625... > >> > >> Feb 16 11:24:53: %BGP-6-ASPATH: Long AS path 4323 3257 29113 47868 47868 > >> 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 > >> 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 > >> 47868 47868 47868 47868... > >> > >> > >> -- > >> Jon Lewis | I route > >> Senior Network Engineer | therefore you are > >> Atlantic Net| > >> _ http://www.lewis.org/~jlewis/pgp for PGP public key_ > >> > > > > -- > Jon Lewis | I route > Senior Network Engineer | therefore you are > Atlantic Net| > _ http://www.lewis.org/~jlewis/pgp for PGP public key_ >
Re: anyone else seeing very long AS paths?
On Tue, 17 Feb 2009, Mike Lewinski wrote: > German Martinez wrote: > bgp max-as will NOT protect you from this exploit (but if you are not > vulnerable it should prevent you from propogating it). > I can confirm this statement... (unfortunately) L.
SIP - perhaps botnet? anyone else seeing this?
Hi All, Over the past couple of days we have been seeing an exponential increase (about 200-fold) in the amount of UDP SIP Control traffic in our netflow data. The past 24 hours, for example, has shown a total of nearly 300 GB of this traffic incoming and over 400 GB outgoing -- this despite the fact that we do not host any SIP services ourselves, and currently to my knowledge, we have no hosting customers running any kind of SIP services. (Total RTP traffic for 24 hours is only in the region of 150 Kb -- so a vast inbalance between control and RTP) The local sources/destinations of the traffic are within our hosting space, but are spread across a wide range of hosts (i.e. nothing really related to a single or handful of hosts). Additionally over the past couple of days we have seen an increase of mails to our abuse desk for "brute force" attempts against a number of SIP services... possibly directly related to this traffic. Is anyone aware of a new variant or modus-operandi of botnets in circulation in the past couple of days which attempt to exploit SIP services? Has anyone else notice a significant increase in this kind of traffic? Thanks Leland
Re: SIP - perhaps botnet? anyone else seeing this?
Legally speaking, we can't "grab packets" in this sense without a specific validated complaint, court orders, and that kind of thing... So all we can do in the the absence of a specific complaint is in the context of our day to day traffic analysis from the netflow data to identify anomalies.. hence this one... (We have already taken action on a handful of known and identified cases of SIP brute-force attacks in recent days). Having said that, we have seen a vast increase in the amount of abuse complaints about SIP authentication brute force attacks in the past couple of days, which would tally with the traffic in general as being actual SIP-Control. The absence of associated RTP, however, leads me to believe that it's either scanning, exploits, or botnets, rather than legitimate SIP traffic. Based on what I've seen in the past couple of days, I am sure that it's as you mentioned, a SIP DDoS or brute-force attacks on SIP services... (circumstantial evidence that it's actually SIP related rather than something else on the same ports -- given the number of abuse complaints) I was simply wondering if this was an overall trend globally, or if it's simply a handful of bozos making life "fun" for the rest of us ;) Thanks Leland On Fri, 10 Apr 2009, Roland Dobbins wrote: > > On Apr 10, 2009, at 4:45 PM, Leland E. Vandervort wrote: > > > UDP SIP Control traffic in our netflow data. > > Have you grabbed some packets in order to ensure it's actually SIP, > vs. something else on the same ports? > > If it really is SIP-related, this could be caused by botted hosts > launching a SIP DDoS, or brute-forcing said SIP services in order to > steal service for resale, DoS someone else via the service at layer-7 > (i.e., call avallanche), sent VoIP spam, et. al. You may have botted > hosts in your hosting space, as well as hosts being scanned as > potential targets for exploitation. > > A quick search-engine query should reveal that this sort of thing has > been going on for quite some time; I believe there were some > convictions in NJ or somewhere else in the northeastern US within the > last year or so. > > --- > Roland Dobbins // +852.9133.2844 mobile > >Our dreams are still big; it's just the future that got small. > > -- Jason Scott > >
Re: SIP - perhaps botnet? anyone else seeing this?
On Fri, 10 Apr 2009, Roland Dobbins wrote: > > IANAL, but I suggest you check again with your legal department - I > doubt this is actually the case (your jurisdiction may vary, but in > most Western nations, you can grab packets for diagnostic/ > troubleshooting/forensics purposes). Already did check... we can't grab packets except in response to judicial order or specific abuse case with a valid ID of the end-user, or of course for general technical diagnostics -- if for diagnostics, we cannot use such collected data in the context of only a suspicion of abuse at all as it would constitute an infringement on the individual's privacy. So in short, we can do it REACTIVELY in response to a complaint.. but if we do it PROACTIVELY, then it cannot be used and is of "educational" value only (with caveats surrounding confidentiality, non-disclosure, and destruction,, etc.) > > So all we can do in the the absence of a specific complaint > > > But you said you *had* specific complaints, did you not? yes.. *specific* and action was taken on those *specific* cases... (didn't actually have to grab traffic though...) L.
Re: SIP - perhaps botnet? anyone else seeing this?
Managed to get to the bottom of it, and it was indeed a SIP User-Agent brute-force attempt. Interestingly, though, that your mail mentions specifically verizon... the majority of the remote addresses during this brute-force attempt were also behind verizon... coincidence? Hmm.. Regards, Leland On Wed, 15 Apr 2009, Dane wrote: > The timing of your email as well as a couple of seemingly unrelated > things that I have heard about make me think this might be related to > some large toll fraud scheme. > > Today I heard from someone who says Verizon is telling them they see > about 700 calls per hour to Cuba originating from their PRI. > > Obviously some type of toll fraud. Got me thinking about this persons > phone system and how there has always been the issue of toll fraud > where someone calls in and knows how to get an outbound call routed > through a poorly setup PBX. > > However the rate of 700 calls per hour and one PRI just don't make > sense or add up in a situation like the old toll fraud method > mentioned earlier since I believe that's more of a manual attack. > > That's when I recalled this post of yours. Made me wonder if there > was some way to exploit SIP to associate with a VoIP PBX or gateway or > something that was tied to PRI's and thus route your calls over > someones phone system. > > Sure enough found some discussions and posts regarding toll fraud to > Cuba (and others) in relation to SIP. > > For instance, Cisco's CallManager Express device which is a router as > well as voip pbx is often tied to PSTN or PRI's and by default allows > H323 TCP/1720 and SIP UDP/5060 ports open by default. > > It may seem obvious to others but new to me that these scans are > related to someone or some group looking to find devices with these > ports open in an effort to attach to them through SIP and hopefully > exploit if attached to PRI's or PSTN for toll fraud. > > I really do learn something new everyday, some smart deviant people out there. > > > On Fri, Apr 10, 2009 at 3:45 AM, Leland E. Vandervort > wrote: > > > > Hi All, > > > > Over the past couple of days we have been seeing an exponential increase > > (about 200-fold) > > in the amount of UDP SIP Control traffic in our netflow data. ?The past 24 > > hours, for example, has shown a total of nearly 300 GB of this traffic > > incoming and over 400 GB outgoing -- this despite the fact that we do not > > host any SIP services ourselves, and currently to my knowledge, we have no > > hosting customers running any kind of SIP services. ?(Total RTP traffic > > for 24 hours is only in the region of 150 Kb -- so a vast inbalance > > between control and RTP) > > > > The local sources/destinations of the traffic are within our hosting > > space, but are spread across a wide range of hosts (i.e. nothing really > > related to a single or handful of hosts). > > > > Additionally over the past couple of days we have seen an increase of > > mails to our abuse desk for "brute force" attempts against a number of SIP > > services... possibly directly related to this traffic. > > > > Is anyone aware of a new variant or modus-operandi of botnets in > > circulation in the past couple of days which attempt to exploit SIP > > services? ?Has anyone else notice a significant increase in this kind of > > traffic? > > > > Thanks > > > > Leland > > > > > > > > >
Re: delays to google
Works fine for me... all google services up and okay here (Paris). traceroute to googlemail.l.google.com (209.85.137.83), 30 hops max, 40 byte packets 1 fe0-0.gw1.discpro.org (217.70.182.177) 1.644 ms 1.518 ms 1.420 ms 2 gandi-discpro.gdist1-d.gandi.net (217.70.176.82) 0.887 ms 0.701 ms 0.606 ms 3 vl9.core3-d.gandi.net (217.70.176.101) 0.666 ms 0.632 ms 0.527 ms 4 vl9.core4-d.gandi.net (217.70.176.102) 0.592 ms 0.556 ms 0.526 ms 5 reserved.above.net.43.141.79.in-addr.arpa (79.141.43.5) 0.666 ms 0.827 ms 0.599 ms 6 xe-4-0-0.mpr1.lhr3.uk.above.net (64.125.31.249) 7.963 ms 7.907 ms 8.165 ms 7 so-1-0-0.mpr1.lhr2.uk.above.net (64.125.28.38) 7.878 ms 7.824 ms 8.161 ms 8 213-152-230-99.google.net (213.152.230.99) 8.905 ms 8.771 ms 8.751 ms 9 209.85.252.42 (209.85.252.42) 8.893 ms 209.85.252.40 (209.85.252.40) 8.228 ms 8.090 ms 10 72.14.233.63 (72.14.233.63) 12.992 ms 16.538 ms 209.85.248.80 (209.85.248.80) 21.237 ms 11 209.85.250.141 (209.85.250.141) 16.448 ms 19.357 ms 16.506 ms 12 72.14.238.128 (72.14.238.128) 23.823 ms 23.637 ms 209.85.248.248 (209.85.248.248) 22.558 ms 13 209.85.241.189 (209.85.241.189) 24.015 ms 23.872 ms 23.672 ms 14 216.239.43.26 (216.239.43.26) 26.986 ms 24.831 ms 216.239.43.98 (216.239.43.98) 24.033 ms 15 mg-in-f83.google.com (209.85.137.83) 24.097 ms 23.879 ms 23.818 ms traceroute to www.l.google.com (209.85.227.147), 30 hops max, 40 byte packets 1 fe0-0.gw1.discpro.org (217.70.182.177) 1.703 ms 2.808 ms 1.850 ms 2 gandi-discpro.gdist1-d.gandi.net (217.70.176.82) 2.301 ms 1.563 ms 2.236 ms 3 vl9.core3-d.gandi.net (217.70.176.101) 6.426 ms 2.403 ms 1.240 ms 4 vl9.core4-d.gandi.net (217.70.176.102) 1.253 ms 2.448 ms 1.267 ms 5 reserved.above.net.43.141.79.in-addr.arpa (79.141.43.5) 1.239 ms 1.806 ms 1.270 ms 6 xe-4-0-0.mpr1.lhr3.uk.above.net (64.125.31.249) 8.534 ms 9.148 ms 20.502 ms 7 so-1-0-0.mpr1.lhr2.uk.above.net (64.125.28.38) 10.123 ms 8.156 ms 101.081 ms 8 213-152-230-99.google.net (213.152.230.99) 10.094 ms 10.335 ms 8.779 ms 9 209.85.252.40 (209.85.252.40) 9.063 ms 8.662 ms 8.336 ms 10 66.249.95.170 (66.249.95.170) 11.511 ms 11.400 ms 72.14.232.134 (72.14.232.134) 10.318 ms 11 209.85.252.83 (209.85.252.83) 10.614 ms 10.413 ms 10.455 ms 12 209.85.243.93 (209.85.243.93) 23.418 ms 17.760 ms 17.950 ms 13 wy-in-f147.google.com (209.85.227.147) 11.635 ms 10.581 ms 10.587 ms Leland On Thu, 14 May 2009, Eduardo Silvestre wrote: > Hello, > > Same issue in portugal and spain. > > pt:~# traceroute www.google.com > traceroute: Warning: www.google.com has multiple addresses; using > 209.85.227.147 > traceroute to www.l.google.com (209.85.227.147), 30 hops max, 40 byte packets > 1 ge130-1000m.cr2.lisboa.nfsi.pt (81.92.200.126) 0.203 ms 0.085 ms 0.108 > ms > 2 xe12-10GE.xmr1.Lisbon.as25137.net (81.92.201.10) 7.358 ms 0.349 ms > 0.239 ms > 3 if-3-2.core1.PV9-Lisbon.as6453.net (195.219.186.9) 0.235 ms 0.347 ms > 0.239 ms > 4 if-12-0-0.mcore3.L78-London.as6453.net (195.219.144.5) 26.974 ms 27.200 > ms 26.847 ms > 5 if-5-0-0.mcore3.LDN-London.as6453.net (195.219.195.9) 32.343 ms 32.452 > ms 32.345 ms > 6 Vlan62.icore1.LDN-London.as6453.net (195.219.83.1) 39.464 ms 29.329 ms > 35.719 ms > 7 xe-10-2-0-edge3.london1.level3.net (4.68.63.105) 27.469 ms 27.456 ms > 27.474 ms > 8 ae-11-51.car1.London1.Level3.net (4.69.139.66) 27.596 ms > ae-21-52.car1.London1.Level3.net (4.69.139.98) 85.304 ms > ae-11-51.car1.London1.Level3.net (4.69.139.66) 36.955 ms > 9 * * * > 10 * * * > 11 * * * > 12 * 72.14.236.191 (72.14.236.191) 389.644 ms * > 13 * * 209.85.243.93 (209.85.243.93) 387.207 ms > 14 * * wy-in-f147.google.com (209.85.227.147) 392.378 ms > > es:~# traceroute www.google.com > traceroute to www.google.com (209.85.227.103), 30 hops max, 40 byte packets > 1 94.46.247.33 (94.46.247.33) 0.401 ms 0.448 ms 0.515 ms > 2 194.25.210.137 (194.25.210.137) 0.287 ms 0.297 ms 0.345 ms > 3 194.25.6.66 (194.25.6.66) 41.014 ms 33.308 ms 41.071 ms > 4 * * * > 5 * * 209.85.255.170 (209.85.255.170) 390.129 ms > 6 72.14.232.208 (72.14.232.208) 387.012 ms * * > 7 * 72.14.232.130 (72.14.232.130) 387.439 ms 386.662 ms > 8 * * 72.14.236.191 (72.14.236.191) 380.375 ms > 9 * * * > 10 * * * > 11 wy-in-f103.google.com (209.85.227.103) 382.547 ms 403.198 ms * > > Regards, > > --- > Eduardo Silvestre > nfsi telecom, lda. > > eduardo.silves...@nfsi.pt > Tel. (+351) 21 949 2300 - Fax (+351) 21 949 2301 > http://www.nfsi.pt/ > > - Original Message - > From: "Athanasios Douitsis" > To: "Steve Williams" > Cc: na...@merit.edu > Sent: Thursday, May 14, 2009 4:51:38 PM GMT +00:00 GMT Britain, Ireland, > Portugal > Subject: Re: delays to google > > On Thu, May 14, 2009 at 6:48 PM, Steve Williams > wrote: > > > am seeing significant delays in getting to google. anyone else seeing > > this? > > > > $ traceroute w
RE: NPE-G2 vs. Sup720-3BXL
We're running several six 65xx Sup720-3BXL with 3 full transit views and some 40-odd peers. We use two NPE-G1s for reflectors and some policy manipulation. Also running MPLS in the core to allow for traffic engineering and EoMPLS between certain services located in different locations. We're pushing up to between 800M and 1G at peak times (mostly outbound) with this setup and peak CPU on the 3BXLs is running maybe 30% -- average though of around 8 to 10%. Hope this helps side-note: I'm actually more worried about the distribution layer at the moment as it relies heavily on STP and HSRP/GLBP for the various vlans and access layer gunk. Currently these are 720-3B (non-XL), but looking eventually to build a business case to upgrade these to VSS1440 to simplify the architecture as well as providing better resilience and elimination of STP/HSRP/GLBP limitations between the dist and access layers. Problem is the budget side of that... blargh! Ideally I'd like to go more for the Nexus platform for this part of the network, given that we're doing a lot of virtualisation etc., but the downsides with that are primarily the COST of the bloody things, and secondly the fact that they don't currently support MPLS (planned for later this year, apparently). Leland On Fri, 15 May 2009, Paul Stewart wrote: > We've never pushed a NPE-G2 to 800Mb/s before but I would think they > would topple over... hopefully someone on here could confirm my > statement? > > Moving the BGP to the 12008's would be my choice with PRP-2 processors > if the budget fits we're faced with a similar upgrade next year > possibly moving BGP from a pair of 7606's (Sup720-3BXL) over to a pair > of GSR's running PRP2 I think - the BGP processing etc. is pushing the > CPU's too high on the 7600's > > Someone else might suggest the RSP720's but haven't had them in a > production environment yet... we had PRP2's running on 12012 for a > while and found them rock solid even with older line cards etc... > > Hope this helps a bit...;) > > Paul > > > -Original Message- > From: David Storandt [mailto:dstora...@teljet.com] > Sent: Friday, May 15, 2009 11:08 AM > To: NANOG list > Subject: NPE-G2 vs. Sup720-3BXL > > We're stuck in an engineering pickle, so some experience from this > crew would be useful in tie-breaking... > > We operate a business-grade FTTx ISP with ~75 customers and 800Mbps of > Internet traffic, currently using 6509/Sup2s for core routing and port > aggregation. The MSFC2s are under stress from 3x full route feeds, > pared down to 85% to fit the TCAM tables. One system has a FlexWAN > with an OC3 card and it's crushing the CPU on the MSFC2. System tuning > (stable IOS and esp. disabling SPD) helped a lot but still doesn't > have the power to pull through. Hardware upgrades are needed... > > We need true full routes and more CPU horsepower for crunching BGP > (+12 smaller peers + ISIS). OC3 interfaces are going to be mandatory, > one each at two locations. Oh yeah, we're still a larger startup > without endless pockets. Power, rack space, and SmartNet are not > concerns at any location (on-site cold spares). We may need an > upstream OC12 in the future but that's a ways out and not a concern > here. > > Our engineering team has settled on three $20k/node options: > - Sup720-3BXLs with PS and fan upgrades > - Sup2s as switches + ISIS + statics and no BGP, push BGP edge routing > off to NPE-G2s across a 2-3Gbps port-channel > - Sup2s as switches + ISIS + statics and no BGP, push BGP edge routing > off to a 12008 with E3 engines across a 2-3Gbps port-channel. > > Ideas and constructive opinions welcome, especially software and > stability-related. > > Many thanks, > -Dave > > > > > > > > "The information transmitted is intended only for the person or entity to > which it is addressed and contains confidential and/or privileged material. > If you received this in error, please contact the sender immediately and then > destroy this transmission, including all attachments, without copying, > distributing or disclosing same. Thank you." > >
