Re: More long AS-sets announced
Many of us in the operational community are required to conduct testing in lab environments, followed by well-announced maintenance windows. Thanks for this funny post. I needed a good laugh. It has been years since people have needed a reminder that as the biggest and most complex telecommunications network in the world, the Internet cannot be tested in a lab because it is not possible to construct a lab environment that matches the scale and complexity of the Internet. Why is this operational test supposed to be given freer reign on the 'net than our own operations? It's not. Your operations are also an EXPERIMENT as are the operations of every other ISP. People can pretend that this is not so and make claims to the contrary in their marketing literature, but the Internet by its very nature is and will always remain, AN EXPERIMENT. Your network probably poses more threat to the net than the long AS experiment. That is because you haven't tested all the possible configurations of fat-finger mistakes in your announcements and therefore your peers do not know if their routers can handle it when your network goes nuts. And no matter how much you test it and how tightly you manage it, you can never get that probability down to zero. At least the long-AS folks are warning in advance that their area of the net could be acting strange soon. How many other ISPs provide their peers with warnings about misconfigurations? If, after this advance warning, your network falls over because of the long AS tests then I suggest that your own lab testing has been inadequate. --Michael Dillon
Re: More long AS-sets announced
Hank wrote: Wrong again. You used both AS1221 and AS2121: I logged pretty much the same thing with max-as limit blocking/logging the announcements (of course, the paths, and time stamp seconds are different, YMMV) Perhaps the unforeseen technical difficulties have something to do with not sticking to the original plan? Announce something different to get around filters, and thus, detect who put filters in place? Or more innocent. maybe fat fingers, or copy/paste gone horribly wrong? Doesn't really matter what happened though, because a controlled announce is a lot better than a malicious one. Since a stupid person is the most dangerous type of person (fifth basic law of human stupidity), a malicious announcement is even safer than a clueless one. I'd much rather see this done by someone with clue that's going to announce and withdraw them, then check for damage, than by someone that might not know what the heck they're doing. If there is damage, we'll all hear about it, and figure out how to stop it in the future when someone else tries to be malicious. Or when someone else is just plain clueless. Apparently that's not the case since this whole experiment was so disruptive that it took 16 hours for someone to notice and point out on NANOG that it neither did nor did not go off as previously announced. (I'm guilty of looking it over in the logs, and not even noticing the difference between 2121 and 1221) The internet is our playground... can't we just all get along? If someone's going to load 50 kids on a merry-go-round, and get 50 more kids to push it I'll just stand over by the monkey bars and try to avoid the flying vomit. :-) -Jerry
Re: More long AS-sets announced
On Tue, Jun 21, 2005 at 08:13:08AM +0300, Hank Nussbacher wrote:
due to unforeseen technical difficulties, we have been forced to
postpone these experiments. We plan to make the announcements at the
same times on Monday 20 June.
The prefixes will be the same (84.205.73.0/24 and 84.205.89.0/24) and
will be originated by AS12654 as before, but the AS-set will consist of
AS2121 repeated n times, so the paths will look like 12654 {2121, 2121,
.., 2121}. AS2121 is the RIPE meeting AS, which is reserved for RIPE
meetings and does not currently appear in the global routing table.
Wrong again. You used both AS1221 and AS2121:
Yups, smells like a small but very dangerous typo.
From a netcitizen's perpesctive and as being involved in operating a part
of the internet, I'm starting to dislike this whole experiment more and
more. I understand, as people already commented, the internet in fact is
one big experiment, but this is getting out-of-hand.
Can the people responsible for this please reconsider and put things on
hold until certain conditions are met:
Publish a detailed workplan including AS-es used, windows and risk
analysis to the various mailinglists.
A reasonable time between annoucements and the experiment instead of 24
hours.
Some assurance that input files are checked for typos.
Take another look wether it is smart to use 'production' AS-es for it,
such as the RIS or meeting AS numbers, instead of a seperate set. I think
people are going to get real unhappy if somewhere in October they found
out the meeting network is blocked at various places.
Just my 2 cents,
MarcoH
Re: More long AS-sets announced
Can the people responsible for this please reconsider and put things on hold until certain conditions are met: Publish a detailed workplan including AS-es used, windows and risk analysis to the various mailinglists. given they are using their own prefixes, can you please tell us what risk there might be. Some assurance that input files are checked for typos. hopefully better than the mean of operators doing so :-)/2 randy
Re: More long AS-sets announced
On Tue, Jun 21, 2005 at 10:27:18AM +0100, Randy Bush wrote: Can the people responsible for this please reconsider and put things on hold until certain conditions are met: Publish a detailed workplan including AS-es used, windows and risk analysis to the various mailinglists. given they are using their own prefixes, can you please tell us what risk there might be. Not much, but I guess people in the audience might get happy from the small note that labtests showed IOS won't crash when it encounters these annoucements. MarcoH
Re: More long AS-sets announced
Can the people responsible for this please reconsider and put things on hold until certain conditions are met: Publish a detailed workplan including AS-es used, windows and risk analysis to the various mailinglists. given they are using their own prefixes, can you please tell us what risk there might be. Not much, but I guess people in the audience might get happy from the small note that labtests showed IOS won't crash when it encounters these annoucements. showing that ios won't crash is very difficult because the number of versions of ios, and the amazing dependencies of things on which blade is in which slot and what phase is the moon. but reading the roma gang's papers and the main email note leads me to feel they have done as good a job on this as we can reasonably expect. considering that we have fellow isps dumping horrifying garbage in the rib, it's amusing how we attack a seemingly well-run very small experiment. randy
Re: More long AS-sets announced
Randy Bush wrote: showing that ios won't crash is very difficult because the number of versions of ios, and the amazing dependencies of things on which blade is in which slot and what phase is the moon. Thank you. You've provided a clean, concise counter to Lorenzo's original claim that long AS sets won't trip on IOS bugs. This may be a well-run, very small experiment, but it's experimenting with a space that's rarely explored and therefore less likely to have encountered the same level of operational testing that horrifying garbage leaks have tested. As such, I'm frustrated that the testers consider requests to provide more advance notice to be so obtuse. Many of us in the operational community are required to conduct testing in lab environments, followed by well-announced maintenance windows. Why is this operational test supposed to be given freer reign on the 'net than our own operations? Alternatively, why can't the test be conducted in a lab, with interested operators providing router configurations and xOS versions to give the test bed the most realistic sample of the 'net, without using the production 'net? pt
Re: More long AS-sets announced
On Tue, 21 Jun 2005, Pete Templin wrote: Randy Bush wrote: showing that ios won't crash is very difficult because the number of versions of ios, and the amazing dependencies of things on which blade is in which slot and what phase is the moon. Thank you. You've provided a clean, concise counter to Lorenzo's original claim that long AS sets won't trip on IOS bugs. .. Alternatively, why can't the test be conducted in a lab, with interested operators providing router configurations and xOS versions to give the test bed the most realistic sample of the 'net, without using the production 'net? Has anyone considered that the project may have indeed done testing of available IOS/$ROUTER versions in a lab environment before even considering testing on the 'live' internet? Reading the cited material might be of benefit to the vocal complainers. I think Lorenzo would be the first to admit that the timing of operator notifications, and more importantly the wording, may be less than desired, however this does not detract from the caution, and professionalism exhibited thus far in this set of experiments. This may be a well-run, very small experiment, but it's experimenting with a space that's rarely explored and therefore less likely to have encountered the same level of operational testing that horrifying garbage leaks have tested. Part of the problem is that because it is, as you put it, a rarely explored problem space, the number of interested parties with sufficient and varied resources is extremely small, resulting in a less-than-complete testing environment. Another part of the problem is that you cannot put the concept back in the box. The black-hats do read operational lists, and with all the fuss being made over possible breakage caused by AS-sets, some of them will do will perform their own, possibly destructive experiments in order to find out what specific $ROUTER versions do under various inputs. So, which would you prefer.. Lorenzo at a known contact number with known working hours (+31 20 535 , 10am to 5pm GMT +1), and with the Internet's best interests at heart, or some malcontent with unknown contacts, unknown hours, and very definitely not your best interests at heart ? --==-- Bruce. Unfortunately, option 3, do nothing to see whether it is actually a problem, is no longer valid.
Re: More long AS-sets announced
RB Date: Tue, 21 Jun 2005 14:40:47 +0100 RB From: Randy Bush [ trimming CC list ] RB considering that we have fellow isps dumping horrifying garbage in RB the rib, it's amusing how we attack a seemingly well-run very small RB experiment. Bears would rather attack fish than wolverines. Considering Lorenzo's attitude, I'm sure he's taking into account the requests for more heads up. If he tickles an IOS bug, I'd rather have it happen in this scenario than when a less-clued individual or a miscreant tries announcing wacky routes. Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita DO NOT send mail to the following addresses: [EMAIL PROTECTED] -*- [EMAIL PROTECTED] -*- [EMAIL PROTECTED] Sending mail to spambait addresses is a great way to get blocked. Ditto for broken OOO autoresponders and foolish AV software backscatter.
Re: More long AS-sets announced
Thank you. You've provided a clean, concise counter to Lorenzo's original claim that long AS sets won't trip on IOS bugs. no problem. you're quite welcome. This may be a well-run, very small experiment, but it's experimenting with a space that's rarely explored and therefore less likely to have encountered the same level of operational testing that horrifying garbage leaks have tested. As such, I'm frustrated that the testers consider requests to provide more advance notice to be so obtuse. could you please give me the command to configure ios to not crash if given advance notice? Why is this operational test supposed to be given freer reign on the 'net than our own operations? Alternatively, why can't the test be conducted in a lab, with interested operators providing router configurations and xOS versions to give the test bed the most realistic sample of the 'net, without using the production 'net? the first announcement of this experiment was months ago. randy
Re: More long AS-sets announced
On Tue, 21 Jun 2005, Bruce Campbell wrote: Has anyone considered that the project may have indeed done testing of available IOS/$ROUTER versions in a lab environment before even considering testing on the 'live' internet? Reading the cited material might be of benefit to the vocal complainers. Not everyone runs IOS. Wasn't it something similar to this that crashed gated and possibly other BGP implementations a few years ago? Is this test intended to make sure everyone upgraded / nobody's deployed new gear with old affected code? And finally, we're doing it, we're not doing it, Surprise, we did it is a crappy way to notify the community that they're about to piss in the global pool. At least there was some level of notification, but why bother if you're not going to stick to what you publicize? -- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_
Re: More long AS-sets announced
Randy Bush wrote: could you please give me the command to configure ios to not crash if given advance notice? telnet your.mail.server 25 helo your.pc mail.from you mail.to you data Be sure to sit near a terminal with OOB access to your network at XYZ while an experiment is conducted with the Internet. Have vendor support contracts handy, and find a PC with a dialup connection in case IOS crashes. .
Re: More long AS-sets announced
Edward B. Dreger wrote: Considering Lorenzo's attitude, I'm sure he's taking into account the requests for more heads up. If he tickles an IOS bug, I'd rather have it happen in this scenario than when a less-clued individual or a miscreant tries announcing wacky routes. Bull. His attitude (at least to me) was he needed a consensus of the operational community before he would feel compelled to provide more notice and/or postpone the testing to provide said notice. pt
Re: More long AS-sets announced
And finally, we're doing it, we're not doing it, Surprise, we did it is a crappy way to notify the community that they're about to piss in the global pool. At least there was some level of notification, but why bother if you're not going to stick to what you publicize? One might suspect that the change of plans was due to the impact of operational reality. I would expect some small amount of sympathy from those on this list for those types of events. Tony
Re: More long AS-sets announced
On Tue, 21 Jun 2005, Tony Li wrote:
And finally, we're doing it, we're not doing it, Surprise, we did it
is a crappy way to notify the community that they're about to piss in
the global pool. At least there was some level of notification, but why
bother if you're not going to stick to what you publicize?
One might suspect that the change of plans was due to the impact of
operational reality. I would expect some small amount of sympathy from
those on this list for those types of events.
So send out another email.
Hey people, something came up and we couldn't get started on schedule.
We've rescheduled the test to begin at 16:00 UTC, June 20.
Incidentally, I got a small flurry of MAXAS-LIMIT messages from our
transit routers, but only saw the {2121,...} set.
--
Jon Lewis | I route
Senior Network Engineer | therefore you are
Atlantic Net|
_ http://www.lewis.org/~jlewis/pgp for PGP public key_
Re: More long AS-sets announced
On Mon, 2005-06-20 at 01:10 +0200, Lorenzo Colitti wrote:
Lorenzo Colitti wrote:
as part of our AS-set stuffing experiments (announced, including links
to in-depth information, in [1]), we will be announcing unusually large
AS-sets tomorrow, Thursday 16 June.
Hi,
due to unforeseen technical difficulties, we have been forced to
postpone these experiments. We plan to make the announcements at the
same times on Monday 20 June.
Yeah, add more monday morning trouble, people will love to get to work
then ;)
Btw, if you postponed the 'experiment', how come I did pick up this one:
84.205.73.0/24 12654 12654
{1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221}
Greets,
Jeroen
PS: Is the 'technical difficulty' your own router falling over? :)
signature.asc
Description: This is a digitally signed message part
Re: More long AS-sets announced
June 15th: Lorenzo gives us 24 hours notice that he is going to be using our (a very general our here, meaning all Internet operators) network for performing his experiments on. (oh, and points out that hes been doing the same with IPv6 since last year, just unannounced, but thats okay because noone noticed) June 15th: Those that check their email frequently enough to spot this ask Lorenzo for at least a week notice before doing this in future. June 20th: Lorenzo gives us same day notice that he will be using our network as his plaything again. Anyone sufficiently behind Lorenzo in the grand scheme of things (either they have better things to do than read their email all day, or perhaps they're on the west coast USA) won't even know about this until it is too late. If something strange happens I'm not sure everyone will suddenly think better check to see if Lorenzo is playing again. I see the use of the Internet for experiments like this to be somewhat frivilous, and the notice periods given as warning even more so. I'm sure Lorenzo would not appreciate if I were to give 20 minutes warning of some clandestine experiment, such as announcing more specifics of his institutions prefixes as particularly helpful! he is announcing his own bleedin' prefixes. get a life randy
Re: More long AS-sets announced
Jeroen Massar wrote:
Btw, if you postponed the 'experiment', how come I did pick up this one:
84.205.73.0/24 12654 12654
{1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221}
That path was announced during the window of notice we had given for the
announcements. However, you will notice that that was not the complete
set of announcements we intended to make, which included 25-, 50-, 75-.
and 100-element AS-sets.
Since we were not able to send all the announcements within the window
of notice we had provided, we postponed them to avoid sending
announcements when people were not expecting them.
PS: Is the 'technical difficulty' your own router falling over? :)
No. :)
Regards,
Lorenzo
--
[EMAIL PROTECTED] [EMAIL PROTECTED]
www.ripe.netwww.dia.uniroma3.it/~compunet
RIPE NCCRoma Tre Computer Networks research group
Re: More long AS-sets announced
On Mon, Jun 20, 2005 at 11:58:33AM +0200, Lorenzo Colitti wrote:
Jeroen Massar wrote:
Btw, if you postponed the 'experiment', how come I did pick up this one:
84.205.73.0/24 12654 12654
{1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221}
That path was announced during the window of notice we had given for the
announcements. However, you will notice that that was not the complete
set of announcements we intended to make, which included 25-, 50-, 75-.
and 100-element AS-sets.
By the way, accoording to your annoucement:
| The prefixes involved will be 84.205.73.0/24 and 84.205.89.0/24, both
| orignating in AS12654. The AS-sets will consist of AS12654 repeated n
| times, thus the paths will look like 12654 {12654, 12654, ..., 12654}.
| No other AS numbers will be used. The values of n we will use are 25,
| 50, 75 and 100.
What's the '1221' doing there ?
Grtx,
MarcoH
Re: More long AS-sets announced
On Mon, 2005-06-20 at 12:33 +0200, MarcoH wrote:
On Mon, Jun 20, 2005 at 11:58:33AM +0200, Lorenzo Colitti wrote:
Jeroen Massar wrote:
Btw, if you postponed the 'experiment', how come I did pick up this one:
84.205.73.0/24 12654 12654
{1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221}
That path was announced during the window of notice we had given for the
announcements. However, you will notice that that was not the complete
set of announcements we intended to make, which included 25-, 50-, 75-.
and 100-element AS-sets.
By the way, accoording to your annoucement:
| The prefixes involved will be 84.205.73.0/24 and 84.205.89.0/24, both
| orignating in AS12654. The AS-sets will consist of AS12654 repeated n
| times, thus the paths will look like 12654 {12654, 12654, ..., 12654}.
| No other AS numbers will be used. The values of n we will use are 25,
| 50, 75 and 100.
What's the '1221' doing there ?
That is the RIPE Meeting AS apparently, which is not used at the moment.
This was mentioned in todays mail (not in the original one).
According to whois though, I hope that you enlightened Geoff of taking
over his AS:
aut-num: AS1221
as-name: ASN-TELSTRA
descr:Telstra Pty Ltd
I also do hope that people are not going to filter out 1221 btw as that
would sever connectivity of that AS when it is needed.
Another thing to note is that neither of those two AS's have any
reference to these experiments in whois.
aut-num: AS12654
as-name: RIPE-NCC-RIS-AS
descr:RIPE NCC RIS Project.
descr:http://www.ripe.net/ris/
admin-c: HU266-RIPE
tech-c: RISM-RIPE
remarks: Different subsets of the routes in AS12654:RS-RIS are
announced
remarks: at each location.
remarks: Please send peering requests to [EMAIL PROTECTED]
mnt-by: RIPE-NCC-RIS-MNT
source: RIPE # Filtered
It would be nice to list it there too as mentioned before, not everybody
reads the various mailinglists and there is no mention on that site
either
Greets,
Jeroen
signature.asc
Description: This is a digitally signed message part
