Re: Yahoo to MSN problems
On 05/19/04, Hank Nussbacher [EMAIL PROTECTED] wrote: Anyone know more? Things looked better today, but past experience shows that they may get awful again in a few days. While the problem may appear to be more on our (Hotmail's) end than Yahoo's, the volume of mail being shoved at us is in no way under our control. We will continue to tweak things until they stay good. (And for those who may be wondering, neither repeated phone calls nor demands for daily meetings with our executives will resolve the problem any faster.) -- J.D. Falk be crazy dumbsaint of the mind [EMAIL PROTECTED] -- Jack Kerouac
Re: Barracuda Networks Spam Firewall
Eric A. Hall wrote: What's most interesting about the half-dozen accusations of xenophobia I've received (off-list and on) is that they've almost all come from foreigners. I promise not to read anything into that. Really. Could it be perhaps because us foreigners are conditioned by repeated exposure to the xenephobic attitudes of USofA patriots ? Peter
Re: Barracuda Networks Spam Firewall
Folks, let's stop this thread. We're getting into 'spam is really bad' comments, which aren't particularly enlightening to the list.
Re: Barracuda Networks Spam Firewall
What's most interesting about the half-dozen accusations of xenophobia I've received (off-list and on) is that they've almost all come from foreigners. I promise not to read anything into that. Really. Could it be perhaps because us foreigners are conditioned by repeated exposure to the xenephobic attitudes of USofA patriots ? shut up or we'll bomb and torture you
Re: Barracuda Networks Spam Firewall
On 5/20/2004 8:25 AM, Randy Bush wrote: What's most interesting about the half-dozen accusations of xenophobia I've received (off-list and on) is that they've almost all come from foreigners. I promise not to read anything into that. Really. Could it be perhaps because us foreigners are conditioned by repeated exposure to the xenephobic attitudes of USofA patriots ? shut up or we'll bomb and torture you resist the cycle of violence and hate -- Eric A. Hallhttp://www.ehsco.com/ Internet Core Protocols http://www.oreilly.com/catalog/coreprot/
Re: Barracuda Networks Spam Firewall
On Wed, 19 May 2004 22:54:55 EDT, joe [EMAIL PROTECTED] said: either 1: SMTP/ESMTP is fixed so that spoofing cannot occur or 2: Another method/protocol of email/messaging is adopted 3: We change the economics of spamming in some other fashion. I've been advocating taking up a collection - every ISP that has an inbound spam problem kicks in just $100 - if there's 4,000 ISP's in the US (including all those mompop sites with E-bay routers), that's a pretty chunk of change. We then hire a few representatives from choose ethnic organized crime to explain our point of view to a few of the aforementioned 200 big offenders... Unfortunately, there's these concepts of legality and morality involved... :) pgpdjV5bJPBtY.pgp Description: PGP signature
OT: Avi Freeman at the WSOP
Avi Freeman is at the final two tables of the $5000 Pot-Limit Omaha event at the World Series of Poker: http://www.pokerpages.com/tournament/result8742.htm Pre-congratulations to Avi on making it that far in one of the toughest events against one of the toughest fields of the WSOP.
Road Runner contact needed
Does anyone have a Road Runner contact, or can a RR.com representative contact me offlist? Thanks. I've had no response (other than auto-ack) from [EMAIL PROTECTED] or similar channels for more than a month. Mark A Jones Systems Administrator netINS, Inc. http://netins.net (515) 830-0698 [EMAIL PROTECTED]
Charter: host problem
Charter, your abuse and security mailboxes are bouncing as unavailable. Can someone from Charter security or network please respond privately regarding a host issue at your customer TAIS in Asheville, NC? Thanks. -- Martin Hannigan (c) 617-388-2663 VeriSign(w) 703-948-7018 Network Enginer IV Operations Infrastructure [EMAIL PROTECTED]
Re: OT: Avi Freeman at the WSOP
On Thu, May 20, 2004 at 02:12:04PM -0400, [EMAIL PROTECTED] wrote: Avi Freeman is at the final two tables of the $5000 Pot-Limit Omaha event at the World Series of Poker: http://www.pokerpages.com/tournament/result8742.htm Pre-congratulations to Avi on making it that far in one of the toughest events against one of the toughest fields of the WSOP. Minor correction: Freedman OT notes: Many of your fellow network engineers play poker, and, having sat at a table with Avi and some other Akamai folks, I wouldn't want to meet them at WSOP with a lower chip count ;) PS - If you are interested in the Texas Reunion in SF, drop me a line. This is not for people from Texas. :-) Tim
handling ddos attacks
I've been trying to find out what the current BCP is for handling ddos attacks. Mostly what I find is material about how to be a good net.citizen (we already are), how to tune a kernel to better withstand a syn flood, router stuff you can do to protect hosts behind it, how to track the attack back to the source, how to determine the nature of the traffic, etc. But I don't care about most of that. I care that a gazillion pps are crushing our border routers (7206/npe-g1). Other than getting bigger routers, is it still the case that the best we can do is identify the target IP (with netflow, for example) and have upstreams blackhole it? Thanks, -mark
Re: handling ddos attacks
I too would be interested if someone could point a good white paper for cisco DDOS protection mechanisms and best practices in general. On Thu, May 20, 2004 at 11:52:01AM -0700, Mark Kent wrote: I've been trying to find out what the current BCP is for handling ddos attacks. Mostly what I find is material about how to be a good net.citizen (we already are), how to tune a kernel to better withstand a syn flood, router stuff you can do to protect hosts behind it, how to track the attack back to the source, how to determine the nature of the traffic, etc. But I don't care about most of that. I care that a gazillion pps are crushing our border routers (7206/npe-g1). Other than getting bigger routers, is it still the case that the best we can do is identify the target IP (with netflow, for example) and have upstreams blackhole it? Thanks, -mark --- Wayne Bouchard [EMAIL PROTECTED] Network Dude http://www.typo.org/~web/
Re: handling ddos attacks
The dearth of comprehensive BCP asserting the end-all-be-all for DDoS is likely and largely due to the lack of an end-all-be-all DDoS. The range of variants, strains, chewy fillings and flavors of fuxor out there beg different techniques for alleviation, so prescribing a single poultice for blanket application does not seem to be in wide practice outside marketing stratagem and other blustering. The resources requiring protection and receiving priority, as well as the trade-off in exacting reactive measures, also have a say in how things are managed. In general, however, yeah...identifying the source or target is a must. Or a source port or destination port or protocol type or packet size or point of ingress/egress...the list of signature-worthy candidates is significant and also determines how a DDoS is triaged. The only thing that can be said for certain is that *some* unifying factor must be discovered. :P Furthermore, how you do that and what you do with that is a fluid thing, and further refinement or definition of the type of DDoS you are seeking to relieve may be required before you will be able to root out an attack management template that is worth its salt. Blackhole servers, sinkhole routers, IDS, extrusion detection, heuristic baselining, and definitely bigger routers never hurt this effort either. ;) If you are able to elaborate on what you might be seeking to accomplish on- or off-list, I will try to proffer any appropriate resources I have available. Good luck. --ra -- Rachael Treu-Gomes, CISSP [EMAIL PROTECTED] ..quis costodiet ipsos custodes?.. On Thu, May 20, 2004 at 11:52:01AM -0700, Mark Kent said something to the effect of: I've been trying to find out what the current BCP is for handling ddos attacks. Mostly what I find is material about how to be a good net.citizen (we already are), how to tune a kernel to better withstand a syn flood, router stuff you can do to protect hosts behind it, how to track the attack back to the source, how to determine the nature of the traffic, etc. But I don't care about most of that. I care that a gazillion pps are crushing our border routers (7206/npe-g1). Other than getting bigger routers, is it still the case that the best we can do is identify the target IP (with netflow, for example) and have upstreams blackhole it? Thanks, -mark
Re: [NANOG-LIST] handling ddos attacks
Is there any quantification on what qualifies as a Large DDOS attack and perhaps a comparison of what type of routers can/can't handle such a load? Typical DDOS's that I've seen are 10-20X the normal incoming packet rate, upto and over 1Mpps. Having to switch that amount of additonal load has a tremendous impact on linecard CPU and any amount of additional features to try and protect your customer will sometimes result in a degradation to *everyone* not just the target. In my experience calling the upstream provider and having it blocked is still the only thing that can be done. When working on the backbone I've spent hours tracking the majority of flows back to one or more peering points and blocking it there where the attack isn't as concentrated and thus safer to filter. -Brent At 11:52 AM 5/20/2004, Mark Kent wrote: I've been trying to find out what the current BCP is for handling ddos attacks. Mostly what I find is material about how to be a good net.citizen (we already are), how to tune a kernel to better withstand a syn flood, router stuff you can do to protect hosts behind it, how to track the attack back to the source, how to determine the nature of the traffic, etc. But I don't care about most of that. I care that a gazillion pps are crushing our border routers (7206/npe-g1). Other than getting bigger routers, is it still the case that the best we can do is identify the target IP (with netflow, for example) and have upstreams blackhole it? Thanks, -mark
ntp config tech note
sorry to take you away from discussing spam with an actual tech note, but twice this morning i have hit incidents where much needed ntp clients were blown. so, as i was gonna have to write it up, i figured i would bore you all with it. --- ntp config hint 2004.05.20 ntpd will not work if your clock is off my a few minutes. it just sits there forever with its finger in its ear. so, at boot, before you start ntpd, use ntpdate to whack your system's time from a friendly low-numbered strat chimer. do not background ntpdate with -b, because, if it is slow to complete, ntpd can't get the port when you try to start it next in the boot sequence. if ntpdate takes a minute and thus adds to your boot time, then something is wrong anyway; fix it. in case your dns resolver is slow, servers are in trouble, etc. have an entry for your ntpdate chimer in /etc/hosts. yes, i too hate /etc/hosts; but i have been bitten without this hack; named is even more fragile than ntpd. once ntpdate has run, then and only then, start your ntpd. and read all the usual advice on configuration, selection and solicitation of chimers with which to peer, ... and then, if having accurate time on this host is critical, cron a script which runs `ntpq -c peers` and pipes it to a hack which looks to be sure that one of the chimers has a splat in front of it. run this script hourly, and scream bloody hell via email if it finds problems. --- now back to your regular spam discussion. /* yes, spam is an important issue. but, if your local organization, this mailing list, ... gets swamped with discussions of spam, then the spammers have won. you have to compartmentalize it, in your organization and in the general net culture. that's why there are separate mailing lists for spam, ddos, and other net crap with which we have to deal. that's why we have more than one mailing list in the world, to compartmentalize so we can focus. */ randy
Re: handling ddos attacks
A paper based on a presentation I did at the PAIX peering forum in December is here: http://www.stevegibbard.com/ddos-talk.htm I should probably update it a bit, but that may not happen any time soon. Slides from another presentation at the same conference are here: http://www.prostructure.com/content/research/presentations/ddos_intro/ -Steve On Thu, 20 May 2004, Mark Kent wrote: I've been trying to find out what the current BCP is for handling ddos attacks. Mostly what I find is material about how to be a good net.citizen (we already are), how to tune a kernel to better withstand a syn flood, router stuff you can do to protect hosts behind it, how to track the attack back to the source, how to determine the nature of the traffic, etc. But I don't care about most of that. I care that a gazillion pps are crushing our border routers (7206/npe-g1). Other than getting bigger routers, is it still the case that the best we can do is identify the target IP (with netflow, for example) and have upstreams blackhole it? Thanks, -mark
Re: Barracuda Networks Spam Firewall
On Wed, 19 May 2004, Eric A. Hall wrote: my last 10 survivors are at http://www.ehsco.com/misc/last-10-spams.eml the relevant data for them in order of occurrance is below. eight are CN, one is KR, one is Geocities, and one is dead Different people get different spam, from different sources. For years I was under the impression that spammers must be blasting everybody, so everybody would get similar spam. I was surprised to find out that this isn't the case... Rik -- Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. - Brian W. Kernighan
Re: OT: Avi Freeman at the WSOP
On May 20, 2004, at 2:46 PM, Timothy Brown wrote: On Thu, May 20, 2004 at 02:12:04PM -0400, [EMAIL PROTECTED] wrote: Avi Freeman is at the final two tables of the $5000 Pot-Limit Omaha event at the World Series of Poker: http://www.pokerpages.com/tournament/result8742.htm Pre-congratulations to Avi on making it that far in one of the toughest events against one of the toughest fields of the WSOP. Minor correction: Freedman He made the money, but has to go back at 2 PM (PST) for the final round. This is pretty impressive given that 1) He has had about 4 hours sleep (on an airplane) in the 48 before the tournament and B) Avi only paid $5K, no rebuys, no add ons. The top player had to buy more chips multiple times. Right now Avi is guaranteed a $10K profit no matter what. Some of the other people are not guaranteed a profit unless they make it to the top 5 or so. Funny stories about his tournament play: During the 1 hour dinner break, Avi wen to play a cash game instead of eating. :) And this morning, he spent a couple hours on his computer fixing his personal server instead of sleeping some more. (Worse, he had to do it over a modem!) OT notes: Many of your fellow network engineers play poker, and, having sat at a table with Avi and some other Akamai folks, I wouldn't want to meet them at WSOP with a lower chip count ;) PS - If you are interested in the Texas Reunion in SF, drop me a line. This is not for people from Texas. :-) I am s in. -- TTFN, patrick
Re: Barracuda Networks Spam Firewall
On 5/20/2004 2:30 PM, Rik van Riel wrote: Different people get different spam, from different sources. Yah, I've been advocating the use of a CIDR match-list from the beginning for this and other reasons. Actually what you'd want is per-entry weighting, so for me and my mailbox: CIDR 221.232.0.0/14 score = 3.0 CIDR 147.28.0.0/16 score = -3.0 The ASN matching has merit too, so maybe: ASN 4134 score = 3.0 CIDR holes punched = -3.0 etcetera -- Eric A. Hallhttp://www.ehsco.com/ Internet Core Protocols http://www.oreilly.com/catalog/coreprot/
Re: Yahoo to MSN problems
In article [EMAIL PROTECTED], Hank Nussbacher [EMAIL PROTECTED] writes We are sorry that you are experiencing delay in receiving messages at your hotmail.com or msn.com email address. Yahoo! has contacted MSN and has determined that the source of the problem resides on their end. They are aware of the issue, but do not yet have an estimate of when the problem will be fixed. My USA-based ISP has been reporting issues related to delivering email to Hotmail/MSN addresses, on and off for several months. I don't believe I have a single real correspondent (out of several thousand) who uses such an address, but as a long term anti-spam campaigner, who has received huge amounts of email with forged hotmail addresses, I'd be interested to hear more detail about what's really going on here. -- Roland Perry
Re: ntp config tech note
One minor (operational! -- gasp) addition: More modern copies of ntpd have a '-g' option that will allow the clock to jump once at boot time. Tony On May 20, 2004, at 12:27 PM, Randy Bush wrote: sorry to take you away from discussing spam with an actual tech note, but twice this morning i have hit incidents where much needed ntp clients were blown. so, as i was gonna have to write it up, i figured i would bore you all with it. --- ntp config hint 2004.05.20 ntpd will not work if your clock is off my a few minutes. it just sits there forever with its finger in its ear. so, at boot, before you start ntpd, use ntpdate to whack your system's time from a friendly low-numbered strat chimer. do not background ntpdate with -b, because, if it is slow to complete, ntpd can't get the port when you try to start it next in the boot sequence. if ntpdate takes a minute and thus adds to your boot time, then something is wrong anyway; fix it. in case your dns resolver is slow, servers are in trouble, etc. have an entry for your ntpdate chimer in /etc/hosts. yes, i too hate /etc/hosts; but i have been bitten without this hack; named is even more fragile than ntpd. once ntpdate has run, then and only then, start your ntpd. and read all the usual advice on configuration, selection and solicitation of chimers with which to peer, ... and then, if having accurate time on this host is critical, cron a script which runs `ntpq -c peers` and pipes it to a hack which looks to be sure that one of the chimers has a splat in front of it. run this script hourly, and scream bloody hell via email if it finds problems. --- now back to your regular spam discussion. /* yes, spam is an important issue. but, if your local organization, this mailing list, ... gets swamped with discussions of spam, then the spammers have won. you have to compartmentalize it, in your organization and in the general net culture. that's why there are separate mailing lists for spam, ddos, and other net crap with which we have to deal. that's why we have more than one mailing list in the world, to compartmentalize so we can focus. */ randy
Dell power connect switches.
Good afternoon, We are planning to deploy several Dell PowerConnect 3324, 3348 and 6024 switches on our network. We currently have between 200-300 users and servers that these switches will service. We are also planning to add about 300-400 more users in the next 2-3 mos. 85% of the users are for our call center where they all use Terminal clients and connect to W2K TS. The rest is regular staff and our application servers. Can anyone tell me any good/bad points about them? I originally proposed using RiverStone as L2 switches but price was a factor in our decision to go with Dell. That is my main concern at this point. The Dell switches are very cheap compared to other L2 switches out there. Will this be a case of you get what you pay for or are they really good performing units? I really have not been able to find any lists or other sources with comments on these units. I'd appreciate any info you guys might have. Regards, - Joel Perez| Network Engineer [EMAIL PROTECTED] | www.USPGI.com -
Re: ntp config tech note
More modern copies of ntpd have a '-g' option that will allow the clock to jump once at boot time. Saku Ytti [EMAIL PROTECTED] also told me this. have you tested. i remember a bad experience with it some years back, and, being a normally supersitious hacker, have avoided ever since. (yes, i still walk around that crack in the sidewalk where i tripped in the third grade:-). randy
Re: Barracuda Networks Spam Firewall
On May 20, 3:30pm, Rik van Riel [EMAIL PROTECTED] wrote: Different people get different spam, from different sources. For years I was under the impression that spammers must be blasting everybody, so everybody would get similar spam. I was surprised to find out that this isn't the case... This is very true. We're four people in the same company, and there is the odd overlapping spam, but generally not at all; not even over several days. There must be some undiscovered science in there. -- Per
Re: ntp config tech note
From: Randy Bush [EMAIL PROTECTED] Date: Thu, 20 May 2004 12:27:48 -0700 Sender: [EMAIL PROTECTED] ntp config hint 2004.05.20 ntpd will not work if your clock is off my a few minutes. it just sits there forever with its finger in its ear. so, at boot, before you start ntpd, use ntpdate to whack your system's time from a friendly low-numbered strat chimer. For the initial ntpdate, I recommend that you use fairly local, highly reliable hosts. Low numbered stratum is not very relevant. If your clock is off by 600 ms, ntpd will fix it just fine. do not background ntpdate with -b, because, if it is slow to complete, ntpd can't get the port when you try to start it next in the boot sequence. Huh? On every system I have worked on (Unix types), -b is the boot option and does exactly what you want to do at boot time. It sets the clock immediately by stepping and never slews the time. This is what you want at boot time as you want the time to be correct ASAP, not in a few minuted. if ntpdate takes a minute and thus adds to your boot time, then something is wrong anyway; fix it. If you use '-b' and have a list of reachable servers, it should take less than a second. in case your dns resolver is slow, servers are in trouble, etc. have an entry for your ntpdate chimer in /etc/hosts. yes, i too hate /etc/hosts; but i have been bitten without this hack; named is even more fragile than ntpd. Rather than put the servers in my hosts file (which would screw up everything should they move), I just five ntpdate a list of servers by IP address. This does everything putting a systems into hosts without the possibility of impacting other stuff. once ntpdate has run, then and only then, start your ntpd. and read all the usual advice on configuration, selection and solicitation of chimers with which to peer, ... and then, if having accurate time on this host is critical, cron a script which runs `ntpq -c peers` and pipes it to a hack which looks to be sure that one of the chimers has a splat in front of it. run this script hourly, and scream bloody hell via email if it finds problems. I use 'ntpq -p', but I'm just lazy enough to save a few keystrokes. Both commands produce identical output. Randy, what version of ntpdate are you running that ntpdate backgrounds on '-b'? -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: [EMAIL PROTECTED] Phone: +1 510 486-8634
Re: ntp config tech note
From: Tony Li [EMAIL PROTECTED] Date: Thu, 20 May 2004 13:06:37 -0700 Sender: [EMAIL PROTECTED] One minor (operational! -- gasp) addition: More modern copies of ntpd have a '-g' option that will allow the clock to jump once at boot time. OK. Am I in a alternate universe? I have run ntpdate for years on a variety of systems, almost all of the BSD family. (I count the VMS implementation in TGV software as BSD.) I have never seen '-g' and have always had '-b' as the boot option. I have confirmed the '-b' with the official sources at Deleware. -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: [EMAIL PROTECTED] Phone: +1 510 486-8634
Re: Dell power connect switches.
Can anyone tell me any good/bad points about them? I looked at the 3248 and 5224s about a year ago, and would strongly advise against deploying any of the gear in a production environment. They had a number of issues with LACP/dot1q. The most severe issue is that the management interface would occasionally crash hard -- no control possible, even by serial line. It was still passing packets, but had to be reloaded to change anything. They were also a number of exploits of the web interface (incorrect implementations of access control, improper bounds checking, etc.) You get what you pay for. I believe the switches are OEMed from Accton; you'll find other vendors (e.g. SMC) selling the same boxes. -Kevin --- Kevin C. Miller [EMAIL PROTECTED]
Re: ntp config tech note
On Thu, May 20, 2004 at 01:14:32PM -0700, Randy Bush wrote: More modern copies of ntpd have a '-g' option that will allow the clock to jump once at boot time. Saku Ytti [EMAIL PROTECTED] also told me this. have you tested. i remember a bad experience with it some years back, and, being a normally supersitious hacker, have avoided ever since. (yes, i still walk around that crack in the sidewalk where i tripped in the third grade:-). randy it works here. --bill
Re: ntp config tech note
One minor (operational! -- gasp) addition: More modern copies of ntpd have a '-g' option that will allow the clock to jump once at boot time. If you have not told the kernel to refuce to change the time when the system is in multiuser mode for security reasons. -Peter There is an easy workaround, just make sure your local clock in the computer is as close to UTC as you can
Re: Dell power connect switches.
On Thu, 20 May 2004, Joel Perez wrote: out there. Will this be a case of you get what you pay for or are they really good performing units? I really have not been able to find any lists or other sources with comments on these units. I'd appreciate any info you guys might have. In the low-end market it's mostly management and other software issues you pay for. I have an example of a 24 port 10/100 switch with dual 1000TX uplinks for $95 from an Taiwan manufacturer, where you get some kind of windows-only special management program (not telnet/snmp able). It's still very inexpensive and they claim wire-speed and I have no reason to doubt it, making a 20gigsbit/s unit is not very hard today. If you like the management interface of your Dells then they'll most likely perform what you need in the pure shuffle packets-area as long as you do IPv4 unicast. If you want to muck around with multicast, several vlans perhaps leaking multicast from one vlan to another, private vlan edge, QoS etc, (mostly metro ethernet stuff, for delivering triple play services to subscribers), then that's a whole other ballgame. -- Mikael Abrahamssonemail: [EMAIL PROTECTED]
Re: handling ddos attacks
[EMAIL PROTECTED] disait : On Thu, May 20, 2004 at 11:52:01AM -0700, Mark Kent wrote: I've been trying to find out what the current BCP is for handling ddos attacks. Mostly what I find is material about how to be a good net.citizen (we already are), how to tune a kernel to better withstand a syn flood, router stuff you can do to protect hosts behind it, how to track the attack back to the source, how to determine the nature of the traffic, etc. But I don't care about most of that. I care that a gazillion pps are crushing our border routers (7206/npe-g1). Other than getting bigger routers, is it still the case that the best we can do is identify the target IP (with netflow, for example) and have upstreams blackhole it? or acl it. some providers offer blackhole services where you can inject a route to them via bgp over the same session (with communities) or over a different session that just takes blackhole routes.. that can be used by you to cause them to null0/discard the traffic within their network automatically.. At last Ripe meeting, i made a presentation about the way France Telecom is handling DDOS attack : http://www.ripe.net/ripe/meetings/ripe-48/eof.html#nocexp Slides at http://www.ripe.net/ripe/meetings/ripe-48/presentations/ripe48-eof-gillet.pdf We presented our practice from a NOC perspective (ACL, blackhole, sinkhole, netflow, sample, ... etc) and our next steps. We proposed to give this presentation at coming Nanog, but we were not so succesfull. Next nanog meeting maybe ... Vincent.
Re: ntp config tech note
note that ntpdate is actually depreciated. and at some point you'll have to run ntpd to set the time (with the -q flag) then run it again. joelja On Thu, 20 May 2004, Randy Bush wrote: sorry to take you away from discussing spam with an actual tech note, but twice this morning i have hit incidents where much needed ntp clients were blown. so, as i was gonna have to write it up, i figured i would bore you all with it. --- ntp config hint 2004.05.20 ntpd will not work if your clock is off my a few minutes. it just sits there forever with its finger in its ear. so, at boot, before you start ntpd, use ntpdate to whack your system's time from a friendly low-numbered strat chimer. do not background ntpdate with -b, because, if it is slow to complete, ntpd can't get the port when you try to start it next in the boot sequence. if ntpdate takes a minute and thus adds to your boot time, then something is wrong anyway; fix it. in case your dns resolver is slow, servers are in trouble, etc. have an entry for your ntpdate chimer in /etc/hosts. yes, i too hate /etc/hosts; but i have been bitten without this hack; named is even more fragile than ntpd. once ntpdate has run, then and only then, start your ntpd. and read all the usual advice on configuration, selection and solicitation of chimers with which to peer, ... and then, if having accurate time on this host is critical, cron a script which runs `ntpq -c peers` and pipes it to a hack which looks to be sure that one of the chimers has a splat in front of it. run this script hourly, and scream bloody hell via email if it finds problems. --- now back to your regular spam discussion. /* yes, spam is an important issue. but, if your local organization, this mailing list, ... gets swamped with discussions of spam, then the spammers have won. you have to compartmentalize it, in your organization and in the general net culture. that's why there are separate mailing lists for spam, ddos, and other net crap with which we have to deal. that's why we have more than one mailing list in the world, to compartmentalize so we can focus. */ randy -- -- Joel Jaeggli Unix Consulting [EMAIL PROTECTED] GPG Key Fingerprint: 5C6E 0104 BAF0 40B0 5BD3 C38B F000 35AB B67F 56B2
Re: Dell power connect switches.
dell managed switches = accton and smc managed switches the cli is cisco style. early revs of their firmware had frequent managemnt interface crashes, that appears to be mostly fixed in more recent builds. joelja On Thu, 20 May 2004, Joel Perez wrote: Good afternoon, We are planning to deploy several Dell PowerConnect 3324, 3348 and 6024 switches on our network. We currently have between 200-300 users and servers that these switches will service. We are also planning to add about 300-400 more users in the next 2-3 mos. 85% of the users are for our call center where they all use Terminal clients and connect to W2K TS. The rest is regular staff and our application servers. Can anyone tell me any good/bad points about them? I originally proposed using RiverStone as L2 switches but price was a factor in our decision to go with Dell. That is my main concern at this point. The Dell switches are very cheap compared to other L2 switches out there. Will this be a case of you get what you pay for or are they really good performing units? I really have not been able to find any lists or other sources with comments on these units. I'd appreciate any info you guys might have. Regards, - Joel Perez| Network Engineer [EMAIL PROTECTED] | www.USPGI.com - -- -- Joel Jaeggli Unix Consulting [EMAIL PROTECTED] GPG Key Fingerprint: 5C6E 0104 BAF0 40B0 5BD3 C38B F000 35AB B67F 56B2
Re: ntp config tech note
Kevin Oberman [EMAIL PROTECTED] writes: OK. Am I in a alternate universe? I have run ntpdate for years on a variety of systems, almost all of the BSD family. (I count the VMS implementation in TGV software as BSD.) I have never seen '-g' and have always had '-b' as the boot option. I have confirmed the '-b' with the official sources at Deleware. According to the current man page for ntpd, the ntpdate is to be retired, hence the incorporation of the functionality of ntpdate(8) into the ntpd(8) program. It's not clear to me why Randy considered this newsworthy enough to post to NANOG, nor why he feels the need to write it up rather than just sending his internal customer an excerpt of the man page, where this behavior is clearly documented (and has been since at least xntp3 circa 1997). Is it possible he's decided to compete with the guys who discovered last week that CSMA networks are vulnerable to jabber? ---Rob
Re: ntp config tech note
I've found it useful on older machines (PCs with cheap clocks and oscilators) to cron ntpdate once an hour to prevent the clock from getting too far off by itself. I've found the daemon doesn't do good enough of a job to sync on it's own... I'm also wondering, how many people are using the ntp.mcast.net messages to sync their clocks? what about providing ntp to your customers via the ntp broadcast command on serial links, etc..? - jared -- Jared Mauch | pgp key available via finger from [EMAIL PROTECTED] clue++; | http://puck.nether.net/~jared/ My statements are only mine.
Re: ntp config tech note
On Thu May 20, 2004 at 05:12:31PM -0400, Robert E. Seastrom wrote: It's not clear to me why Randy considered this newsworthy enough to post to NANOG, nor why he feels the need to write it up rather than just sending his internal customer an excerpt of the man page, where this behavior is clearly documented (and has been since at least xntp3 circa 1997). Is it possible he's decided to compete with the guys who discovered last week that CSMA networks are vulnerable to jabber? Or, maybe, as he alluded to in his email, he's just trying to get us to talk about something other than spam ;-) Simon -- Simon Lockhart | Tel: +44 (0)1628 407720 (x(01)37720) | Si fractum Technology Manager | Fax: +44 (0)1628 407701 (x(01)37701) | non sit, noli BBC Internet Ops | Email: [EMAIL PROTECTED]| id reficere BBC Technology, Maiden House, Vanwall Road, Maidenhead. SL6 4UB. UK
RE: ntp config tech note
That's NTPv4 isn't it? I also prefer to use three peers vs. two. Always an odd number, greater than 1. Assumptions can't be made about the mathematics behind time, but in a reference model, odd numbers are better. [Not to be confused with network timing, although the same clocks are used to provide sources for time over different layer 1/2/3 protocols ] -M -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Tony Li Sent: Thursday, May 20, 2004 4:07 PM To: Randy Bush Cc: [EMAIL PROTECTED] Subject: Re: ntp config tech note One minor (operational! -- gasp) addition: More modern copies of ntpd have a '-g' option that will allow the clock to jump once at boot time. Tony On May 20, 2004, at 12:27 PM, Randy Bush wrote: sorry to take you away from discussing spam with an actual tech note, but twice this morning i have hit incidents where much needed ntp clients were blown. so, as i was gonna have to write it up, i figured i would bore you all with it. --- ntp config hint 2004.05.20 ntpd will not work if your clock is off my a few minutes. it just sits there forever with its finger in its ear. so, at boot, before you start ntpd, use ntpdate to whack your system's time from a friendly low-numbered strat chimer. do not background ntpdate with -b, because, if it is slow to complete, ntpd can't get the port when you try to start it next in the boot sequence. if ntpdate takes a minute and thus adds to your boot time, then something is wrong anyway; fix it. in case your dns resolver is slow, servers are in trouble, etc. have an entry for your ntpdate chimer in /etc/hosts. yes, i too hate /etc/hosts; but i have been bitten without this hack; named is even more fragile than ntpd. once ntpdate has run, then and only then, start your ntpd. and read all the usual advice on configuration, selection and solicitation of chimers with which to peer, ... and then, if having accurate time on this host is critical, cron a script which runs `ntpq -c peers` and pipes it to a hack which looks to be sure that one of the chimers has a splat in front of it. run this script hourly, and scream bloody hell via email if it finds problems. --- now back to your regular spam discussion. /* yes, spam is an important issue. but, if your local organization, this mailing list, ... gets swamped with discussions of spam, then the spammers have won. you have to compartmentalize it, in your organization and in the general net culture. that's why there are separate mailing lists for spam, ddos, and other net crap with which we have to deal. that's why we have more than one mailing list in the world, to compartmentalize so we can focus. */ randy
Re: ntp config tech note
On Thu, 2004-05-20 at 15:33, Jared Mauch wrote: I'm also wondering, how many people are using the ntp.mcast.net messages to sync their clocks? what about providing ntp to your customers via the ntp broadcast command on serial links, etc..? - jared I have used NTP mcast for some time, most of my gear sets it's time this way. I run mcast on the inside network, ie customer and internet edge interfaces don't run it. There is no customer interest in mcast, here. Plus it is allot more to consider if I let customers join my mcast network. Dr. Mills suggested looking at manycast so clients select the closest NTP server (I have 1 strat1 and 3 strat2). -- James H. Edwards Routing and Security Administrator At the Santa Fe Office: Internet at Cyber Mesa [EMAIL PROTECTED] [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: List of dynamic IP's
--- Bob Martin wrote: Does anyone know of a list of dynamic IP's by ISP? Two children of the Wirehub/Easynet Dynablock are available via rsync: http://www.njabl.org/dynablock.html http://www.dnsbl.us.sorbs.net/DUL-FAQ.html You can use grepcidr with BGP data if you need to split the lists by ASN. http://www.pc-tools.net/unix/grepcidr/ However for blocking incoming SMTP at the moment it may be more effective to simply use the Spamhaus XBL and basic HELO sanity checks. -- Alex Clark __ Do you Yahoo!? Yahoo! Domains Claim yours for only $14.70/year http://smallbusiness.promotions.yahoo.com/offer
Re: ntp config tech note
On Thu, 20 May 2004, Jared Mauch wrote: I've found it useful on older machines (PCs with cheap clocks and oscilators) to cron ntpdate once an hour to prevent the clock from getting too far off by itself. I've found the daemon doesn't do good enough of a job to sync on it's own... Isn't that a lot safer anyway than running a daemon (ntpd) as root ? I do this on my systems (run ntpdate from cron), even though the xntpd docs IIRC specifically advised against this hack. One less vulnerability waiting to be exploited ... is the way I see it.
Re: Load Balancing Multiple DS3s (outgoing) on a 7500
Drew, Something that was just released that you might be interested in if you haven't already found an alternate solution. http://www.cisco.com/en/US/products/sw/iosswrel/ps5207/products_feature_guide09186a0080221544.html It's a new feature in 12.3(8)T. Rodney On Fri, Mar 12, 2004 at 10:39:16PM -0500, Drew Weaver wrote: Does anyone know of an article, or documentation regarding load balancing the traffic on 3 or more FastEthernet interfaces on the outgoing direction? Right now we're running BGP internally, and the routes that are being chosen based upon the final BGP decision step or what I like to call the 'IP address tie breaker' which is not always optimal. We have a cisco 7500 that is connected to 4 other Cisco 7500s which each have 45Mbps ds3s to the Internet, we would like to load balance the outgoing traffic across all 4 of these 7500s, can anyone shine any advice my way? I noticed that there are instructions on Cisco's site regarding doing LB on 12000s. Anyways thanks in advance ;-) -Drew
southern utah outage
QWest lost an OC48 through southern Utah around 20:29 UTC. ---
Re: ntp config tech note
On Thu, May 20, 2004 at 06:37:23PM -0400, C. Jon Larsen wrote: On Thu, 20 May 2004, Jared Mauch wrote: I've found it useful on older machines (PCs with cheap clocks and oscilators) to cron ntpdate once an hour to prevent the clock from getting too far off by itself. I've found the daemon doesn't do good enough of a job to sync on it's own... Isn't that a lot safer anyway than running a daemon (ntpd) as root ? I do this on my systems (run ntpdate from cron), even though the xntpd docs IIRC specifically advised against this hack. One less vulnerability waiting to be exploited ... is the way I see it. well, it does help if your clock goes nicely (or poorly) askew. problem is any timestamps you may have on that host (radius, smtp, etc..) that you use to track down the (l)users on your network can cause a problem. all you have to be concerned with is am i doing ntpdate from something that can be poisoned. that's amongst many reasons to have the your clock is too far off, you must reset manually log messages. - jared -- Jared Mauch | pgp key available via finger from [EMAIL PROTECTED] clue++; | http://puck.nether.net/~jared/ My statements are only mine.
ntpd config tech note redux
with constructive criticism/input from a number of net folk, version .0002! randy --- an ntpd config hint 2004.05.20 executive summary o if you have a recent ntpd, use `ntpd -g`, and be sure to start it before you go multiuser if you have clock lock security in multiuser o if the above does not work for you, sorry, you need to read on; you may want to anyway many applications need to be run in host environments where an accurate clock is needed. this is why most hosts today chime with ntp. but, ntpd will not work if your clock is off by a few minutes. it quits or just sits there forever with its finger in its ear. so, newer versions of ntpd have the -g parameter, which allows for a big first step. this obviates the use of ntpdate in the next paragraphs. but, this will not work if you have told the kernel to refuse to change the time when the system is in multiuser mode for security reasons. at boot, before you start ntpd, you can use ntpdate to whack your system's time from a list of friendly nearby and very sure to be connected chimers. if ntpdate takes a minute and thus adds to your boot time, then something is wrong anyway; fix it. in case your dns resolver is slow, servers are in trouble, you're running ntpdate before dns resolution is up [0], etc. have an entry for your ntpdate chimer(s) in /etc/hosts. yes, i too hate /etc/hosts; but i have been bitten without this hack; named is even more fragile than ntpd. run ntpdate -b with a list of servers. this will help if one or more are unreachable. once ntpdate has run, then and only then, start your ntpd. and read all the usual advice on configuration, selection and solicitation of chimers with which to peer, ... the 'iburst' keyword for servers listed in ntp.conf will cause ntpd toperform an initial sync (defined as any synchronization after a transition out of an unsynchronized state) with a short burst of packets in a small interval. so, you get a faster clock update for a small tradeoff in accuracy. not considered polite to public servers, but if you have local boxes that keep pretty good time, it's may be worth the minute amount of extra traffic. and then, if having accurate time on this host is critical, cron a script which runs `ntpq -p` and pipes it to a hack which looks to be sure that one of the chimers has a splat in front of it. run this script hourly, and scream bloody hell via email if it finds problems. for more info, see http://twiki.ntp.org/. Thanks to Rob Foehl Brad Knowles Peter Lothberg Kevin Oberman Saku Ytti --- [0] - if dnssec is deployed, somewhat accurate time will be needed before name resolution will work. so, if you are an optimimst, expect to see ntpd up before named more and more in the future -30-
66.164.232.0/24 HI-JACKED, I need some help
Hello Folks, I have tried everything but keep running into walls, someone has hi-jacked a block 66.164.232.0/24and is routing it out of Ga. They come online, we catch heck, they go offline and so it goes. I havecalled everyone and can't seem to get anyone to take the route out of their session. Can anyone in Nanog help out here, please. Sincerely, Peter BGP routing table entry for , version 46510837Paths: (4 available, best #2, table Default-IP-Routing-Table) Not advertised to any peer 209 19262 6350 30174 -OK THIS IS US ;-) 165.117.201.58 from 165.117.201.58 (205.171.0.124) Origin IGP, metric 100, localpref 100, valid, external Community: 2548:666 701 6389 6197(HI-JACKEDBLOCK 66.164.232.0/24)--- BAD NEWS 204.255.169.61 (metric 342) from 165.117.162.200 (165.117.162.200) Origin IGP, metric 100, localpref 100, valid, internal, best Community: 2548:666 Originator: 165.117.162.10, Cluster list: 165.117.162.200 701 6389 6197(HI JACKEDOUR IPS) 204.255.169.61 (metric 342) from 165.117.162.201 (165.117.162.201) Origin IGP, metric 100, localpref 100, valid, internal Community: 2548:666 Originator: 165.117.162.10, Cluster list: 165.117.162.201 701 6389 6197 (HI JACKEDOUR IPS) 204.255.169.61 (metric 342) from 165.117.162.202 (165.117.162.202) Origin IGP, metric 100, localpref 100, valid, internal Community: 2548:666 Originator: 165.117.162.10, Cluster list: 165.117.162.202, 165.117.162.200 AS6197 -| \/OrgName: BellSouth Network Solutions, IncOrgID: BNS-14Address: 1100 Ashwood Parkway, Suite 200City:StateProv: GAPostalCode:Country: USASNumber: 6197ASName: BATI-ATLASHandle: AS6197Comment:RegDate:Updated: 1996-01-04TechHandle: WD14-ARINTechName: Dawson, WillardTechPhone: +1-770-814-5099TechEmail: [EMAIL PROTECTED] **There is a old timer that answers the phone and has no idea what is going on; he mops the floors in the switch gear room. AS6389.http://ws.arin.net/cgi-bin/whois.pl No match found for AS6389 !! However . . ASNumber: 6380 - 6389ASName: BELLSOUTH-NET-BLKASHandle: AS6380Comment:RegDate: 1996-03-28Updated: 2001-01-03TechHandle: DR791-ARINTechName: Ringen, DeronTechPhone: +1-678-441-7919TechEmail: [EMAIL PROTECTED]OrgAbuseHandle: ABUSE81-ARINOrgAbuseName: Abuse Group OrgAbusePhone: +1-404-499-5224OrgAbuseEmail: [EMAIL PROTECTED]: JG726-ARINOrgTechName: Geurin, JoeOrgTechPhone: +1-404-499-5240OrgTechEmail: [EMAIL PROTECTED]
RE: Charter: host problem
It wouldn't matter. All of the notices I have sent to Charter were just ignored. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Hannigan, Martin Sent: Thursday, May 20, 2004 11:21 AM To: '[EMAIL PROTECTED]' Subject: Charter: host problem Charter, your abuse and security mailboxes are bouncing as unavailable. Can someone from Charter security or network please respond privately regarding a host issue at your customer TAIS in Asheville, NC? Thanks. -- Martin Hannigan (c) 617-388-2663 VeriSign(w) 703-948-7018 Network Enginer IV Operations Infrastructure [EMAIL PROTECTED]
Fw: Symantec Mail Security detected that you sent a message containing prohibited content (SYM:32487773651441470267)
- Original Message - From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, May 20, 2004 8:30 PM Subject: Symantec Mail Security detected that you sent a message containing prohibited content (SYM:32487773651441470267) Subject of the message: 66.164.232.0/24 HI-JACKED, I need some help Recipient of the message: NANOG [EMAIL PROTECTED] And what Symantec Mail Security detect? Peter
Re: ntp config tech note
On Thu, 20 May 2004 17:33:22 -0400 Jared Mauch [EMAIL PROTECTED] wrote: I'm also wondering, how many people are using the ntp.mcast.net messages to sync their clocks? what about providing ntp We have had one user that I know of who was receiving time sync info via multicast announcements, but personally I don't care for doing NTP this way. In my experience systems/users don't bother to do any sort of authentication or filtering on NTP sources. Most server admins and some implementations do not support authentication either. I'm pretty sure I don't want to get time from just anyone who sends to 224.0.1.1 especially on networks connected to the multicast-enabled Internet. That group address I might note is one I tend to scope at admin boundaries for just that reason. John
OT: Telemerc Contact
Attempting to contact anyone formerly associated with this venture.
Re: 66.164.232.0/24 HI-JACKED, I need some help
On Thu, 20 May 2004, P.Schroebel wrote: Hello Folks, I have tried everything but keep running into walls, someone has hi-jacked a block 66.164.232.0/24 and is routing it out of Ga. They come online, we catch heck, they go offline and so it goes. I have called everyone and can't seem to get anyone to take the route out of their session. Can anyone in Nanog help out here, please. http://www.ris.ripe.net/cgi-bin/risprefix.cgi will give you an audit-trail of sorts to take to bellsouth, assuming you can find someone cooperative. Regards, J. -- Jess Kitchen ^ burstfire.net[works] _25492$ | www.burstfire.net.uk
Re: Barracuda Networks Spam Firewall
Different people get different spam, from different sources. ... This is very true. We're four people in the same company, and there is the odd overlapping spam, but generally not at all; not even over several days. There must be some undiscovered science in there. according to http://www.dcc-servers.net/dcc/graphs/, most people get most of the same spam, even if this doesn't appear in local measurements. (note that these graphs are subtle and complex and wonderful, and deserve several minutes of careful study before you try to draw any conclusions.) -- Paul Vixie
Re: handling ddos attacks
[EMAIL PROTECTED] (Mark Kent) writes: I've been trying to find out what the current BCP is for handling ddos attacks. Mostly what I find is material about ... But I don't care about most of that. I care that a gazillion pps are crushing our border routers (7206/npe-g1). Other than getting bigger routers, is it still the case that the best we can do is identify the target IP (with netflow, for example) and have upstreams blackhole it? that seems hardly worthwhile. ddos is astonishingly easier to launch than to defend against. if you stop a flow the attacker *might* get bored and decide to do something else, but they could also decide to attack you from a different direction, or wait two days and do it all over again, and every time they attack and you defend it's 10 minutes of their time and 10 hours of yours. far better to involve law enforcement and get some bad guys arrested, if you possibly can. this changes your costs from 10 hours to 15 hours but it actually puts some chips on the table and makes the game worthwhile. -- Paul Vixie
Re: ntpd config tech note redux
In message [EMAIL PROTECTED], Randy Bush writes: with constructive criticism/input from a number of net folk, version .0002! I'll add my .02 currency units: if you can, make one of your ntp peers XX.pool.ntp.org, where XX is your country code. Obviously, not all values of XX work -- among the surprising failures are il.pool.ntp.org, jp.pool.ntp.org, and kr.pool.ntp.org -- but it's worth looking for your country or a neighboring one. This will give you a selection among many different choices, if you aren't concerned with picking a specific one (say, for security reasons). --Steve Bellovin, http://www.research.att.com/~smb
Re: handling ddos attacks
- Original Message - From: Paul Vixie [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, May 20, 2004 9:48 PM Subject: Re: handling ddos attacks [EMAIL PROTECTED] (Mark Kent) writes: I've been trying to find out what the current BCP is for handling ddos attacks. Mostly what I find is material about ... But I don't care about most of that. I care that a gazillion pps are crushing our border routers (7206/npe-g1). Other than getting bigger routers, is it still the case that the best we can do is identify the target IP (with netflow, for example) and have upstreams blackhole it? that seems hardly worthwhile. ddos is astonishingly easier to launch than to defend against. if you stop a flow the attacker *might* get bored and decide to do something else, but they could also decide to attack you from a different direction, or wait two days and do it all over again, and every time they attack and you defend it's 10 minutes of their time and 10 hours of yours. far better to involve law enforcement and get some bad guys arrested, if you possibly can. this changes your costs from 10 hours to 15 hours but it actually puts some chips on the table and makes the game worthwhile. -- Paul Vixie Hey Paul ! Ok, I 'll buy that right now; we have a DDoS Attack on our core nameservers from 66.165.10.24. Where do we start, do I call the police in Bellingham or Washington State Police. We have blocked their ips but, we know they will come in another way. Peter OrgName:Western Washington University OrgID: WWU Address:Computer Center Address: 516 High Street City: Bellingham StateProv: WA PostalCode: 98225 Country:US NetRange: 66.165.0.0 - 66.165.31.255 CIDR: 66.165.0.0/19 NetName:WWU-RESIDENT-1 NetHandle: NET-66-165-0-0-2 Parent: NET-66-165-0-0-1 NetType:Reassigned NameServer: VIKING.WWU.EDU NameServer: HENSON.CC.WWU.EDU Comment: RegDate:2002-08-15 Updated:2002-08-15 TechHandle: JSW12-ARIN TechName: Williams, J. Scott TechPhone: +1-360-650-2868 TechEmail: [EMAIL PROTECTED]
Re: ntp config tech note
Hannigan, Martin [EMAIL PROTECTED] writes: That's NTPv4 isn't it? I also prefer to use three peers vs. two. Always an odd number, greater than 1. Assumptions can't be made about the mathematics behind time, but in a reference model, odd numbers are better. Actually, three is not enough; Mills says at least four. Diversity in manufacturer (and controlling organization if you can spare the cycles) is a big big plus. You may wish to read Dr. Mills' post to comp.protocols.time.ntp in the wake of the TrueTime bug of the 2001-2002 new year: http://groups.google.com/groups?hl=enselm=3C32924F.994E1D01%40udel.edu ---Rob
fiber cut 19 May/PM - 20 May/AM in Ashburn, VA (lawnmower?!)
Since none of the usual suspects have noted it, I'll give a cursory nod to an ILEC (Verizon) fiber cut that happened mid-afternoon yesterday in Ashburn, VA. About a thousand POTS customers were down (including several OOB dialups of which I am aware in the Equinix facility in Ashburn), as well as some T1 and faster loops to Equinix and elsewhere in the immediate area. Outage was likely off the radar because despite the big concentration of connectivity in the affected area, the natural cost disadvantage of the ILEC meant that few circuits of consequence were riding that fiber. Service was resumed after approximately 12 hours. RFO given was that the fiber cut was caused by a commercial lawnmower. Humorous comments left as an exercise to the reader. ---Rob
Re: ntp config tech note
On Thu, May 20, 2004, C. Jon Larsen wrote: On Thu, 20 May 2004, Jared Mauch wrote: I've found it useful on older machines (PCs with cheap clocks and oscilators) to cron ntpdate once an hour to prevent the clock from getting too far off by itself. I've found the daemon doesn't do good enough of a job to sync on it's own... Isn't that a lot safer anyway than running a daemon (ntpd) as root ? I do this on my systems (run ntpdate from cron), even though the xntpd docs IIRC specifically advised against this hack. One less vulnerability waiting to be exploited ... is the way I see it. Kind of. ntpdate just sets the time. ntpd will actually notice your clock running fast/slow and slowly step your kernel time to deal with your bad clock frequency. man ntpd. Its quite fascinating. RE the ntpd as root thing, is there a capability in some UNIXen which lets you fudge with the kernel time/timecounter frequency without being root? I think thats all it really needs root privilege for. Adrian -- Adrian ChaddI'm only a fanboy if [EMAIL PROTECTED] I emailed Wesley Crusher.
Re: fiber cut 19 May/PM - 20 May/AM in Ashburn, VA (lawnmower?!)
Since none of the usual suspects have noted it, I'll give a cursory nod to an ILEC (Verizon) fiber cut that happened mid-afternoon yesterday in Ashburn, VA. About a thousand POTS customers were down (including several OOB dialups of which I am aware in the Equinix facility in Ashburn), as well as some T1 and faster loops to Equinix and elsewhere in the immediate area. Outage was likely off the radar because despite the big concentration of connectivity in the affected area, the natural cost disadvantage of the ILEC meant that few circuits of consequence were riding that fiber. Service was resumed after approximately 12 hours. RFO given was that the fiber cut was caused by a commercial lawnmower. Humorous comments left as an exercise to the reader. ---Rob Either raise the Blade or Lower the Fiber ! The big issue I saw as contractor was that the fiber was laid without a tracer nor anchored as the IEEE and building codes don't address the installation and even then the landscapers come in and move things around. Pretty soon the fiber was right on top and all you have to do is crack it just a little and the game is over for 12 hours. Peter
Re: handling ddos attacks
On Thu, 20 May 2004, P.Schroebel wrote: Ok, I 'll buy that right now; we have a DDoS Attack on our core nameservers from 66.165.10.24. Where do we start, do I call the police in Bellingham or Washington State Police. We have blocked their ips but, we know they will come in another way. Call your local branch of the US Secret Service, if you're in the states, and ask for their electronic crimes division. If you're not in the states, contact your comprable local authority. They can work with you to coordinate with other jurisdictions, etc. You may have some luck directly with the local police at the point of origin, but it generally helps to have a broader agency involved to coordinate matters. -- Tim Wilde [EMAIL PROTECTED] Systems Administrator Dynamic Network Services, Inc. http://www.dyndns.org/
Re: Dell power connect switches.
On Thu, 20 May 2004, Joel Perez wrote: We are planning to deploy several Dell PowerConnect 3324, 3348 and 6024 switches on our network. I don't know how related they are (if at all), but we were suckered into buying several Dell PowerConnect 3248's some time ago. We have a serious issue with them in that the telnet CLI tends to cease properly accepting connections after a while...making them effectively dumb unmanaged L2 switches. If anyone's aware of a fix for this (other than serial consoles), I'd love to hear it. -- Jon Lewis [EMAIL PROTECTED]| I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_
Re: fiber cut 19 May/PM - 20 May/AM in Ashburn, VA (lawnmower?!)
On Thu, 20 May 2004, Robert E. Seastrom wrote: in the immediate area. Outage was likely off the radar because despite the big concentration of connectivity in the affected area, the natural cost disadvantage of the ILEC meant that few circuits of consequence were riding that fiber. It also affected 9-1-1 service in Ashburn and was reported through the normal channels. Unfortunately, the FCC no longer makes the outage reports available on its web site. Stuff happens, stuff has always happened, stuff will continue to happen.
Re: handling ddos attacks
On May 20, 2004, at 8:10 PM, Tim Wilde wrote: Call your local branch of the US Secret Service, if you're in the states, and ask for their electronic crimes division. If you're not in the states, contact your comprable local authority. They can work with you to coordinate with other jurisdictions, etc. You may have some luck directly with the local police at the point of origin, but it generally helps to have a broader agency involved to coordinate matters. I'd love to hear from anyone who has actually been successful prosecuting an attacker for launching a distributed DOS attack. I suspect it occurs very INfrequently (with the recent trend in extortion aside, as it often results in paper trails) -- unfortunately. Any pointers? -danny
Re: ntp config tech note
On Fri, 21 May 2004, Adrian Chadd wrote: Isn't that a lot safer anyway than running a daemon (ntpd) as root ? I do this on my systems (run ntpdate from cron), even though the xntpd docs IIRC specifically advised against this hack. One less vulnerability waiting to be exploited ... is the way I see it. Kind of. ntpdate just sets the time. ntpd will actually notice your clock running fast/slow and slowly step your kernel time to deal with your bad clock frequency. man ntpd. Its quite fascinating. I know what ntpd is supposed to do. Its what its *not* supposed to do that worries me - i.e. when someone finds that next flaw and exploits it. My personal feeling was that for most systems its better to not have the daemon running - i.e. the benefit of smaller more frequent clock adjustments does not outweigh the cost of another service running, especially as root or even as a jailed non-root user. I checked and the cron job usually adjusts the clock by about 0.2 to 0.3 sec every hour. Sure thats probably more than ntpd would adjust it in any one iteration were ntpd running ... according to: http://www.eecis.udel.edu/~mills/ntp/html/ntpdate.html its not too kooky or dangerous to use ntpdate + cron rather than ntpd; 0.5 sec is given as a cutoff for it being less disruptive when making clock adjustments. Its interesting to hear what other folks are doing. I had assumed folks normally don't run ntpd on each and every server and that ntpdate + cron was much preferred; maybe I am off-base.
Re: ntp config tech note
On Thu, May 20, 2004, C. Jon Larsen wrote: I checked and the cron job usually adjusts the clock by about 0.2 to 0.3 sec every hour. Sure thats probably more than ntpd would adjust it in any one iteration were ntpd running ... according to: http://www.eecis.udel.edu/~mills/ntp/html/ntpdate.html its not too kooky or dangerous to use ntpdate + cron rather than ntpd; 0.5 sec is given as a cutoff for it being less disruptive when making clock adjustments. Its interesting to hear what other folks are doing. I had assumed folks normally don't run ntpd on each and every server and that ntpdate + cron was much preferred; maybe I am off-base. ntpdate can set my clock backwards. ntpd, after you've first run it, won't. If you're using this to combine logs between machines you may not appreciate an hourly backwards step in time. :) adrian -- Adrian ChaddI'm only a fanboy if [EMAIL PROTECTED] I emailed Wesley Crusher.
Re: ntp config tech note
you ask do folk run ntpd on every server. i wonder if folk run ntpd on every router. i did and do. randy
Re: fiber cut 19 May/PM - 20 May/AM in Ashburn, VA (lawnmower?!)
Forgive me, but Isn't Sonet usually deployed in a ring? Why the heck would a fiber this important not be? Sean Donelan wrote: On Thu, 20 May 2004, Robert E. Seastrom wrote: in the immediate area. Outage was likely off the radar because despite the big concentration of connectivity in the affected area, the natural cost disadvantage of the ILEC meant that few circuits of consequence were riding that fiber. It also affected 9-1-1 service in Ashburn and was reported through the normal channels. Unfortunately, the FCC no longer makes the outage reports available on its web site. Stuff happens, stuff has always happened, stuff will continue to happen.
Re: ntp config tech note
Jared Mauch wrote: I've found it useful on older machines (PCs with cheap clocks and oscilators) to cron ntpdate once an hour to prevent the clock from getting too far off by itself. I've found the daemon doesn't do good enough of a job to sync on it's own... I'm also wondering, how many people are using the ntp.mcast.net messages to sync their clocks? what about providing ntp to your customers via the ntp broadcast command on serial links, etc..? I run two stratum-1 servers and a few stratum-2s and I provide time via multicast (224.0.0.1), but I don't use it for my servers, except for testing and verification. I am also providing anycast ntp, and, if the belt and suspenders weren't enough, I am experimenting with manycast. That's an NTPv4 feature where the *client* sends a multicast message to an administratively-scoped group soliciting servers and then the servers respond and set up associations. From a client-configuration standpoint, it's about as convenient as multicast or anycast, but it's more accurate than multicast (since the servers set up true associations with the client) and it allows you to do NTP authentication (which I think breaks with anycast). It seems to work pretty well--the client builds up several associations as if they were all configured manually. michael
Re: ntp config tech note
On Fri, 21 May 2004, Adrian Chadd wrote: RE the ntpd as root thing, is there a capability in some UNIXen which lets you fudge with the kernel time/timecounter frequency without being root? I think thats all it really needs root privilege for. Close enough? http://www.onlamp.com/pub/a/bsd/2003/02/13/chroot.html?page=1 I don't know if the other *BSDs have followed or not... Charles Adrian -- Adrian Chadd I'm only a fanboy if [EMAIL PROTECTED] I emailed Wesley Crusher.
Re: ntp config tech note
Robert E. Seastrom wrote: Hannigan, Martin [EMAIL PROTECTED] writes: That's NTPv4 isn't it? I also prefer to use three peers vs. two. Always an odd number, greater than 1. Assumptions can't be made about the mathematics behind time, but in a reference model, odd numbers are better. Actually, three is not enough; Mills says at least four. Diversity in manufacturer (and controlling organization if you can spare the cycles) is a big big plus. You may wish to read Dr. Mills' post to comp.protocols.time.ntp in the wake of the TrueTime bug of the 2001-2002 new year: http://groups.google.com/groups?hl=enselm=3C32924F.994E1D01%40udel.edu If you're really paranoid, diversity in reference sources should also be considered. You should have more than one stratum-1, and as a group they should get time from more than one of [GPS, WWV/WWVB/DCF77/CHU/JJY/ETC., USNO, ACTS, etc.] and your stratum1s should get time from multiple stratum-1s of similarly diverse references. Many NTP folk look down their nose at the radio sources, since GPS is more accurate. But if you already have a GPS stratum-1, then perhaps your next stratum-1 should be WWVB and friends, or you should have a backup assocation with someone who does. And remember that CDMA gets its time from GPS, so it doesn't count as a diverse source. Like I said, if you're really paranoid... michael
Re: handling ddos attacks
Ok, I 'll buy that right now; we have a DDoS Attack on our core nameservers from 66.165.10.24. Where do we start, do I call the police in Bellingham or Washington State Police. We have blocked their ips but, we know they will come in another way. the best thing is if you call the FBI, or NIPC. if you call your local FBI field office and say you're experiencing a cyberattack and could they give you the number for NIPC then it'll probably produce the results you want, even if NIPC has been renamed one or more times since i last talked to them, or if this old functionality within FBI is now handled by DHS, or both.
RE: Filtering network content (rev.)
Is content filtering something ISPs are looking at or already doing? I'm assuming this question would mostly apply to I did this for a customer back in 1996 or 1997, before transparent devices were around. The users dialed in, and their tacacs/radius profile restricted them to an ACL which blocked traffic should they accidentally have removed their browser proxy config. A Squid proxy was set up with a URL filter list, which was snarfed periodically (I think I automated this somehow) from a list the customer maintained. During black-out times, a time-based rule blocked everything. Worked great, though faded away from lack of interest. I haven't seen similar requests come up since. regards, Steve Steve Birnbaum SkyVision Global Networks Phone: +44 20 83871750 Email: [EMAIL PROTECTED] Experience is something you don't get until just after you need it.
OT: NANOG 31 and Kaboom
For those of you getting to SF a bit early (before Saturday night), there is a local SF radio station that sponsors a big block-party-type event on the waterfront on Saturday afternoon, and they have a huge fireworks show after dark (about 9pm PDT). If you're into pyrotechnics, the fireworks tend to be quite good, and you should be able to see them anywhere on the bayfront (The Embarcadero) south of the Bay Bridge, all the way down to the SBC Ballpark. It's called the Kaboom! and the actual location of the party is a Piers 30-32, just off the Embarcadero. If you don't like pyrotechnics, and/or the noise bothers you, you may want to stay away from the bayfront on Saturday night between 9-10 PM local time. michael
Spring time fiber cuts (was Re: fiber cut 19 May/PM - 20 May/AM)
On Thu, 20 May 2004, Dan Armstrong wrote: Forgive me, but Isn't Sonet usually deployed in a ring? Why the heck would a fiber this important not be? You are making assumptions. Large Part of Southern Utah Without 911 Service May 20 2004 http://tv.ksl.com/index.php?nid=5sid=95368 Verizon phone service, 911 interrupted May 20 2004 http://www.dfw.com/mld/dfw/news/8711763.htm Phone Outage Could Limit 911 Access May 19 2004 http://www.nbc4.com/news/3324749/detail.html Stuff happens, stuff has always happened, stuff will continue to happen. 9-1-1 is much more complex than a normal dialed telephone call, is it any surpise it has problems every once in a while. Its always a good idea to keep the normal 7 or 10 digit phone number for your local emergency services some place. You don't get the benefit of automatic location; but direct dialing has the advantage of working over any working connection to the PSTN including wireline, cellular, voip, satellite, ham radio patch, etc.
Re: handling ddos attacks
At 12:00 PM 20-05-04 -0700, Wayne E. Bouchard wrote: I too would be interested if someone could point a good white paper for cisco DDOS protection mechanisms and best practices in general. For Cisco specific ideas try: http://www.ripe.net/ripe/meetings/archive/ripe-41/tutorials/eof-ddos.pdf specifically slides 86-92 and 105-127. -Hank On Thu, May 20, 2004 at 11:52:01AM -0700, Mark Kent wrote: I've been trying to find out what the current BCP is for handling ddos attacks. Mostly what I find is material about how to be a good net.citizen (we already are), how to tune a kernel to better withstand a syn flood, router stuff you can do to protect hosts behind it, how to track the attack back to the source, how to determine the nature of the traffic, etc. But I don't care about most of that. I care that a gazillion pps are crushing our border routers (7206/npe-g1). Other than getting bigger routers, is it still the case that the best we can do is identify the target IP (with netflow, for example) and have upstreams blackhole it? Thanks, -mark --- Wayne Bouchard [EMAIL PROTECTED] Network Dude http://www.typo.org/~web/
Re: ntpd config tech note redux
Steven M. Bellovin wrote: I'll add my .02 currency units: if you can, make one of your ntp peers XX.pool.ntp.org, where XX is your country code. Obviously, not all values of XX work -- among the surprising failures are il.pool.ntp.org, jp.pool.ntp.org, and kr.pool.ntp.org -- but it's worth looking for your country or a neighboring one. This will give you a selection among many different choices, if you aren't concerned with picking a specific one (say, for security reasons). I seem to get always the same answer, even from the authorative servers of the zone; ;; ANSWER SECTION: ntp.pool.org. 2H IN CNAME sd3.mailbank.com. sd3.mailbank.com. 30M IN A64.15.175.6 Pete
Re: ntpd config tech note redux
Petri Helenius wrote: ;; ANSWER SECTION: ntp.pool.org. 2H IN CNAME sd3.mailbank.com. sd3.mailbank.com. 30M IN A64.15.175.6 ;; ANSWER SECTION: pool.ntp.org. 1h30m IN A 64.44.160.38 pool.ntp.org. 1h30m IN A 65.39.134.11 pool.ntp.org. 1h30m IN A 80.85.129.25 pool.ntp.org. 1h30m IN A 80.254.168.209 pool.ntp.org. 1h30m IN A 81.174.128.183 pool.ntp.org. 1h30m IN A 130.60.7.43 pool.ntp.org. 1h30m IN A 130.60.7.44 pool.ntp.org. 1h30m IN A 193.140.151.9 pool.ntp.org. 1h30m IN A 202.49.159.9 pool.ntp.org. 1h30m IN A 209.162.205.202 pool.ntp.org. 1h30m IN A 209.204.172.153 pool.ntp.org. 1h30m IN A 212.13.201.101 pool.ntp.org. 1h30m IN A 217.125.14.244 pool.ntp.org. 1h30m IN A 217.127.32.90 pool.ntp.org. 1h30m IN A 217.127.249.18 -- suresh ramasubramanian [EMAIL PROTECTED] gpg EDEDEFB9 manager, security and antispam operations, outblaze ltd
Re: ntpd config tech note redux
Suresh Ramasubramanian wrote: ;; ANSWER SECTION: pool.ntp.org. 1h30m IN A 64.44.160.38 pool.ntp.org. 1h30m IN A 65.39.134.11 pool.ntp.org. 1h30m IN A 80.85.129.25 pool.ntp.org. 1h30m IN A 80.254.168.209 pool.ntp.org. 1h30m IN A 81.174.128.183 pool.ntp.org. 1h30m IN A 130.60.7.43 Whoops, too early, my bad. Pete
