San Jose UUT?

[Mark Kent]

Looking to fill the NANOG US Thanksgiving void
(as a Canadian in the USA, I have some spare time):

In San Jose, an upstream is charging us this 5% tax:

  http://www2.csjfinance.org/UUT.asp

I think it's bogus, because the tax applies to intrastate telephone
communications only and, consistent with the long-standing FCC 128
tariff, Internet circuits are regarded has being interstate and not
intrastate.

But the upstream says:

 Our tax advisors at Deloitte have confirmed that we are required to
 charge the San Jose User Utility Tax shown on your invoice. 
 The Internet Tax Freedom Act exemption is not applicable to our
 services as this exemption is for internet access only, not
 bandwidth usage.

I think the last sentence is not relevant, but even if it
is relevant, is it correct?   I note that wikipedia says:

 This law bars federal, state and local governments from taxing
 Internet access and from imposing discriminatory Internet-only
 taxes such as bit taxes, BANDWIDTH TAXES, and email taxes.

I know, IANAL, YANAL, Wiki is not a lawyer...  but I figure someone
else must have previously dealt with this (or something very close).

Thanks,
-mark

Re: icmp rpf

[Mark Kent]

I asked:
 Who among AS1239, AS701, AS3356, AS7018, AS209 does loose RPF
 (not just strict RPF on single-homed customers)?

and Patrick answered:
 I'm wondering why that is relevant.

It's relevant because it was suggested that loose RPF should be a 
best common practice so I was curious which of those ASes decided
that the benefits outweighed the negatives and actually do it.
Don't worry, those were randomly chosen AS.  I didn't intend to 
make any suggestion that the answer would be more important to me
for that set of ASes than any other. 

But, you were correct that I wasn't asking the question
I really wanted answered.   What I wanted to know was, among the
attentive nanog membership, which of you think and/or know that
any/all of those AS do loose RPF?

The motivation here is that, if asked last week, I would have guessed
that none of them run loose RPF.  But at least one of them does.  
The two answers, how many actually do plus whether everyone knew it,
will help me decide if I need to spend more time reading nanog email
and nanog proceedings (or actually go to a meeting), or not...

Thanks,
-mark

Re: icmp rpf

[Mark Kent]

Jared Mauch wrote:
 I would hope they're doing it for more than just ICMP packets.

yes, loose RPF, but I just care about ICMP.

 I would argue should be, or is a current best practice.

OK, so I must have missed the memo :-)

Who among AS1239, AS701, AS3356, AS7018, AS209 does loose RPF
(not just strict RPF on single-homed customers)?

 Did big.net just turn this on, or has it been on for
 months/years now?

I'm pretty sure it's months and not years.
I've noticed it for a while, but it just recently drove me to the
point where I'd complain about it.

Thanks,
-mark

Re: icmp rpf

[Mark Kent]

In response to this:
 Mark Smith wrote:
  The non-announcers, because they're also breaking PMTUD.
 
 Really?   How? 

Mark Smith replied with two paragraphs, but it's not 100% clear to me
that he got the reason why I asked.   I asked because his initial statement
boiled down to numbering on un-announced space breaks PMTUD...
but it doesn't, not by itself (which he later expanded).

It only does so in the presence of filtering.

I think this is an important point to make because of my interaction
with small.net.  When I pointed out the timeouts they said that it was
because they don't announce the router IP addresses, which is true but
not the whole story.  I mentioned that some providers in the past
numbered on rfc1918 space and traceroute still worked, so that alone
was not enough.

Then they said it's because of the asymmetric path, and that also is
true, but again not enough.  A large proportion of traffic is
asymmetric, but traceroute still works.

Then they gave me an explanation that rested on the fact that the
routers will not respond to pings because they are unannounced outside
of their world.  That too is true, but irrelevant and I told them how
Jacobson's traceroute works and told them that *someone* was
dropping/filtering the return packets and I'ld like to know who/why.

They somewhat implied that it was my fault, and this situation was
unique to my net, so I used the big.net looking glass to show how the
same things happens from space not associated with my network.
(Yes, I should have done this from the outset.)

With that they asked big.net, and big.net said they filtered, 
and that's where we are.  

My point here is that it took me ten (10) emails with small.net to get
this information partly because the small.net support staff had notions
in their head premised on too simplistic statements like numbering on
un-announced space breaks PMTUD.  

I wanted to clear this up because this list is likely read by support
people at various networks, and it's pretty clear that not all of them
are well versed even on something as thoroughly discussed over the ages
as traceroute.

Thanks,
-mark

Re: icmp rpf

[Mark Kent]

virendra rode wrote:
 This is yet another reason one shouldn't rely on pings  traceroutes to
 perform reachability analysis.

So, you're in the traceroute is not important camp?
(you'll note that in my email I did ask whether we think 
traceroute is important)

Mark Smith wrote:
 The non-announcers, because they're also breaking PMTUD.

Really?   How?   Remember, we're not talking about RFC1918 space,
where there is a BCP that says we should filter it at the edge.
We're talking about public IP space, that just doesn't happen to be
announced outside of a particular AS.

Thanks,
-mark

Re: Kremen VS Arin Antitrust Lawsuit - Anyone have feedback?

[Mark Kent]

Joe McGuckin typed:
 2) Why does ARIN believe that it can ignore a court order?

Maybe because ARIN wasn't a party to the original proceedings
that generated that order?

Let's say you're eating lunch one day, minding your own business,
and a sheriff comes up with an official looking document and
says You need to hand your car over to Fred... because,
unknown to you, Fred and Barney just finished court proceedings
where the judge ruled that Barney had to give Fred his car,
even though that car was owned by you and just loaned to Barney.

Not a great analogy, because of the whole pink slip thing,
but you get the point.

-mark

Re: Dampening considered harmful?

[Mark Kent]

Back in mid-December someone typed:

  One reason to be careful with dampening is that flaps can be 
  multiplied. (Connect to routeviews and see the different flap counts 
  under different peers for the same flap at your end to observe this.)

How about in this scenario:

  asA gets transit from asT
  asA gets backup transit (ASpath padding) from asB
asB gets transit from asT
asB gets transit from asJ
  asJ gets transit from asT
asT peers with whole world(*)

Now, as asA flaps to asT, we see bad things happen to their routes,
namely an unreasonable amount of flap at even nearest neighbors to asT.

Can this flap magnification be explained by the hierarchy I describe
above?  That is, asT treats all of these ASpaths as customer routes:

  asA
  asB_asA
  asJ_asB_asA

and so we might expect to see multiple flaps as different best
routes come into view inside the geographically diverse asT...  right?

Thanks,
-mark

(*)you know what I mean.

Re: T1 short-haul vs. long-haul

[Mark Kent]

  o SmartJack with demarcation point in the office (or same floor) instead
of the building entrance point

You are not likely going to be able to control that,
it depends on how the install tech's day is going.
Strictly speaking, I believe they are supposed to put
it at the MPOE.

 If I am unlucky the T1 gets delivered either directly or via repeaters as

You are worrying about this too much.

Order the T1 (ESF, B8ZS).  As you order the circuit specify where you
want it to end up (server room, 2nd floor) and that you want an extended
demarc (will cost extra).  

AT install time, make sure someone is around to cajole the installer
into getting the NIU as close as you can to where your router will be.
The installer should test from the extended demarc (make sure they do).  

Get a standard built-in T1 dsu like as been mentioned in previous posts.
Plug it into the extended demarc with an ethernet patch cable.

Now, even though I say to not worry, it is important to remember the
#1 lesson when dealing with telephone companies.  I figure it is the
same all over the world, but just in case it isn't, I'll repeat it here:

  The telco is not your friend.

-mark

Re: T1 short-haul vs. long-haul

[Mark Kent]

 I don't think standard ethernet pinouts are correct.  You want a cable
 with pins 12 on one twisted pair and 45 on another (78 for DDS 56K).

Correct has nothing to do with it.
Any straight-through cable will work just fine.  It's just from the
jack to the equipment...  and it's already been specified that the
extended demarc has been made to the place specified by the customer
(i.e., near the router).   

If you're only going 14 feet (up a wall, across a ladder, down a rack)
then you can use silver satin, or just about any straight-through cable
you find on the floor or under the tiles or in the trunk of your car.

-mark

Re: Looking for recommendations for Datacenter off CA Faultline

[Mark Kent]

 A company I work with (who's servers are located in the San Jose,
 CA) is looking to setup some backup servers at a datacenter whose
 connectivity and location is off any faultline, or away from other
 malady, that might effect its main servers datacenter or
 connectivity. Problem is, they also want them as physically close
 as possible.

Go to So. San Francisco (200 Paul; who runs that?) and choose an
alternate, significant (ATT/Sprint/MCI), provider.  If something
happens that is big enough to knock out that site *and* your San Jose
site then probably most people in the company are dead, together with
millions of people in the SF Bay area.  So the unavailability of
servers, belonging to a company not willing to put something in New
Jersey because it is too far away, becomes pretty insignificant at
that point.

-mark

Re: Can a Customer take their IP's with them? (Court says yes!)

[Mark Kent]

 If you read through
 http://www.e-gerbil.net/ras/nac-case/plantiff-affidavit1.pdf you'll
 see that NAC was blackmailing their client because they knew they
 could not quickly move out

I think that argument is close to being bogus.  The agreement doesn't
say that they have to be out in 45 days:

  Following a mailing of a notice of an increase of base prices,
  customer shall have ten days from the effective date of the increase
  to provide NAC with a written request to terminate service.  ...  If
  customer elects to terminate, such notice shall be effective thirty
  days following receipt of customer's notice to terminate.

So, it's 45 + 10 + 30 = 85 days.   

They mention 60 megawatts of power.  It seems to me that the focus
shouldn't be on the easy task of renumbering a /24 in 85 days (is it
really just a /24?), but on moving the servers :-)

There is mention of increased power charges (up to $18,000) and usage
of 60Mw.  Isn't $20/amp/month still a standard charge in co-lo sites?
If so, $18,000 buys 900amps.   With 120V service, we get
(120*900)/1.67 = 65kw.   65kw over 30 twenty-four hour days is
about 47Mw.   So, the customer is getting a deal.

-mark

Re: Points on your Internet driver's license (was RE: Even you can be

[Mark Kent]

Maybe I'm a little slow on the draw, but I've just now realized 
that we've come full circle, in a strange sort of way.

8 to 10 years ago the discussions were dominated by Karl D(1),
where *everything* was defined as to whether is was actionable or not.
Now the discussions are dominated by many people, acting like
Karl D, where their view is solely based on whether
their contract supports either what they do or don't do.

-mark

(1) Actual name not shown to avoid being sued.

Re: Even you can be hacked

[Mark Kent]

 But ultimately, _you_ are responsible for your own systems.

When I detect abusive behavior coming from a customer site then 
it is my responsibility to make sure that doesn't affect the
rest of the world.

Also, if I know how to fix it at source and the customer doesn't know
then it's my responsibility to make sure the customer has the tools
and resources to fix it.  How fast it gets fixed is not a primary
concern because of the previous paragraph.

Parallels to fire/water/electricity/etc. don't quite work
because there is a big difference between the worm that came
out yesterday and the National Electrical Codes that came out
last century.

-mark

Re: Worst cast worm damage estimates: Research

[Mark Kent]

 Nicholas Weaver and Vern Paxson have published a paper estimating the
 worst case scenario of a network worm attack from USD$52 to $103 Billion.

I'ld just like to say that the 52 dollar estimate is _not_ for my network.
We're at least in the $178 to $182.50 range.

-mark

handling ddos attacks

[Mark Kent]

I've been trying to find out what the current BCP is for handling ddos
attacks.  Mostly what I find is material about how to be a good
net.citizen (we already are), how to tune a kernel to better withstand
a syn flood, router stuff you can do to protect hosts behind it, how
to track the attack back to the source, how to determine the nature of
the traffic, etc.

But I don't care about most of that.  I care that a gazillion
pps are crushing our border routers (7206/npe-g1).

Other than getting bigger routers, is it still the case that the best
we can do is identify the target IP (with netflow, for example) and
have upstreams blackhole it?

Thanks,
-mark

Re: CW Move

[Mark Kent]

 What game is this?  I have some gear at SJC1 and I've not heard anything.

http://a.mainstreet.net/mfn.tif

Postmarked early October.  
It would have been hard to get out in less than one month
(we were out as of mid September). 

-mark

Re: Sprint VS. Qwest

[Mark Kent]

 Neither Sprint nor Qwest are serious about earning my business and are not
 providing me with their network peering details.  I was hoping that the
 list might have the collective resources to help me determine who has
 better peering.

Aren't we six years past the point where people ask because they have
a well founded concern?

Here's a good set:  {1, 701, 1239, 3561, 7018}
Buy from any, buy from all.
All will piss you off at one time or another,
all will pass your bits as well as the next guy (at one time or another).

-mark

Re: iBGP next hop and multi-access media

[Mark Kent]

 I've already had several direct replies saying to manually configure the
 172.16 subnet on router A.  Sure, that will work, but I'm looking for a
 solution that doesn't require manual configuration of all the routers
 involved.

Put another physical ethernet interface in router B and
move 172.16.16.0/24 to the new interface.

This will get you over the psychological hurdle you are facing.

-mark

Re: ATT NYC

[Mark Kent]

 Every time you see one of us mention ISIS or OSPF, all it has to do
 with is carrying loopback/infrastructure routes.

I don't think anyone has said to Ralph why the above is done.  Just in
case it isn't obvious: you need to make sure the next-hops are known
on each router by a means other than bgp.

-mark

Re: wcom issues in SF Bay area?

[Mark Kent]

 So, is there a significant Worldcom operational issue that
 has not yet been reported to nanog?  

To answer my own question: Yes, there was a problem on the MFS
ring between S63 and S77 (a BZ ring problem).   Fixed  with 
a card swap yesterday near mid-day.

-mark

wcom issues in SF Bay area?

[Mark Kent]

I've got problems with 30 T1 circuits, on two DS3 hubs.
All are Worldcom (MFS), all out of S63 (55 So. Market, SanJose), 
all have Pacbell tail loops.

Earlier today a worldcom person said to me: I tried to get you status
from the OSC (Sacramento), but they are really, really busy.  I was on
hold for 45 minutes before I got disconnected

So, is there a significant Worldcom operational issue that
has not yet been reported to nanog?  

Or, am I alone with these problems? 

Thanks,
-mark

DNS attack

[Mark Kent]

FYI,

I'm seeing a lot of DNS lookups for all the three letter domain names
for which we are listed as authoritative (we have five).  

The requests look like this:

req: nlookup(foo.com) id 64450 type=255 class=255

 212.100.232.17.domain  myserver.domain:  31881+ ANY ANY? foo.com. (25)
 4500 0035 1e38  ed11 e20a d464 e811
 c7f5 4909 0035 0035 0021  7c89 0100
 0001    0365 6f73 0363 6f6d
  ff00 ff

We get about 400 requests per minute, per attacking machine,
per authoritative name server, per domain.  

This happened on July 25 with these two sources:

194.186.87.197
130.94.23.70

and today, August 25, with this source:

212.100.232.17

Clearly, this is not a problem right now.  But if the
number of attacking machines grows, then any machine that
serves many three-letter domain names might notice.

And who knows, maybe the cretins will get creative and move
to four letter domains!

Just FYI,
-mark

P.S.  I mentioned the two dates above (7/25, 8/25) purely for
  entertainment purposes.  Consistent with the NY Times
  article last weekend about putting too much weight in 
  events that are merely coincidences, I don't mean to imply
  that there is a 25th of the month conspiracy afoot.


  

Re: Max Prefixes Configured on Customer BGP (WAS Re: ALGX problems?)

[Mark Kent]

Joe Wood [EMAIL PROTECTED] typed:
 However, for ISP's that do NOT use any sort of prefix filters, wouldn't
 you prefer that your BGP session was limited to a number of prefixes, in
 case of a routing leak?

We'ld prefer that such ISPs identify themselves here so we can
straighten them out.  Wasn't that your intention when you asked this
question:

How many of you that currently do not filter your customer BGP
sessions have max-prefixes configured?

That seemed to me to be a small trick to get unsuspecting ISPs to
wave their hands Over here!, so that we could whack'em.

-mark

Re: MAE ATM

[Mark Kent]

 I almost forgot about those netedge boxes, seems the one we had in DC 
 was about as reliable as a microwave with tin foil in it.  I cant 
 remember how many times it or a card had been replaced.

There was a general belief that MFS only had one spare on each coast.
When they swapped it in for a faulty box the one they pulled out
became the new spare :-)

-mark

Re: MAE ATM

[Mark Kent]

 How did people interconnect before may 1998, fddi?

fddi, some remote with netedge boxes at either end of an atm link.
There were some 10baseT connections too, there was at least one
low end Catalyst switch dedicated to plain ethernet.

Here is a big hint:

http://www.nanog.org/2.95.NANOG.notes/mae-west.html

-mark

Re: verio arrogance

[Mark Kent]

 I have one downstream ISP customer that explicitly asked for full BGP
 routes to be written into the contract.  Why Verio's customer's wouldn't
 want full routes makes no business sense to me.

The reasons are related to the law of diminishing returns.

-mark

Re: AS path fugliness?

[Mark Kent]

 Anyone else receiving huge as-path (more than 125) causing these:

Yes, but I saw it only once from four different sources:

Through AS1:

Jul  3 07:23:56: %BGP-3-INSUFCHUNKS: Insufficient chunk pools for aspath, requested 
size 266

Through AS6461:

Jul  3 07:22:51: %BGP-3-INSUFCHUNKS: Insufficient chunk pools for aspath, requested 
size 266

Through AS2828:

Jul  3 07:22:52: %BGP-3-INSUFCHUNKS: Insufficient chunk pools for aspath, requested 
size 266

Through AS4513:

Jul  3 07:22:47: %BGP-3-INSUFCHUNKS: Insufficient chunk pools for aspath, requested 
size 266

-mark

Re: ARIN IP allocation question

[Mark Kent]

 Viawest has just told me that their policy is that customers who go
 over a /23 worth of address space must request further space
 directly from ARIN.

What they (Viawest) are saying you is that they are too small to serve
you.  Your domain record says you are in Denver, so I'm guessing you
must have many choices for ISP.  Find another, have them toss a /22 or
more your way, and put Viawest behind you.

-mark

packet inspection and privacy

[Mark Kent]

I recently claimed that, in the USA, there is a law that prohibits an
ISP from inspecting packets in a telecommunications network for
anything other than traffic statistics or debugging.

Was I correct?

I'ld also like to get opinions on privacy policies for network
operators.  It has been suggested that we should adopt a policy that
says that we'll notify customers if:
1) we inspect traffic, 
2) we're aware that an upstream is inspecting traffic 
3) we're required to inspect traffic (by anyone).

Point 3) is just about the same as 1), but it does imply
a slightly different motivation behind the inspection.

Thanks,
-mark

Re: Portable Fire Suppression

[Mark Kent]

 This specific 'unattended server enclosure' is sitting outside
 in the middle of the desert.

How will you protect it from gunshots:

http://sadtomato.net/mojave.html

They removed that phone booth a couple of years ago:

http://www.lvrj.com/lvrj_home/2000/May-23-Tue-2000/news/13631118.html

Are you taking the same spot, sort of analagous to turning
an old CO into a co-location building?

-mark

Re: list problems? + Certification or College degrees?

[Mark Kent]

 The fact that there are actually ways of knowing and
 characterizing the extent of one's ignorance, while still remaining
 ignorant, may ultimately be more interesting and useful to people
 than Yarkovsky

That's just a fancy way of saying a Clint Eastwood line from
one of his movies (Magnum Force?):  A man's gotta know his limitations.

But, imho, it does provide the best summary and/or dismissal 
for the Certification or College degrees? thread.

-mark kent, H.B.

Re: operational: icmp echo out of control?

[Mark Kent]

RAS I can't speak as to what exactly Akamai is doing, but this

I should add that Akamai contacted me with minutes of my initial
post to ask for more data and they said that they are looking
into it... leaving me with the impression that what I was seeing
was not typical.

-mark

Re: Interconnects

[Mark Kent]

[EMAIL PROTECTED] wrote:
 I would expect that if the Equinix exchange participants were IPv6
 hungry ...

Let me toss in a question that may really be dumb...  what are those
that are hungry for IPV6 doing with it?

I figure that organizations that run IPV6 now think they are 
ahead of the game.  Are they?  Is this something that responsible 
ISPs should be doing?   

Would this turn our network into one big NAT area when we have to
translate into IPV4 addresses at the edge to get to the real Internet?

-mark
 !bankrupt, hence !Tier1

Re: genuity - any good?

[Mark Kent]

 I've gotten attractive pricing from Genuity but I haven't used them
 in a couple years.  Is there any reason I wouldn't want to use them
 as a third upstream OC3 provider?

I think they are outstanding.  After using a bunch-o'nsps from 1994 to
1998, including 5 at a time, we picked Genuity when we went to the DS3
level because they consistently had the best cross-country RTT, the
fewest issues, the best trouble ticket system, and friendly and
capable people who interact well with us both by phone and email.

Still no complaints after all this time.

I was concerned when the BBN-GTEI-Genuity (+ nap.net?)
transitions happened, but I saw no degradation of operations.

I've only had one billing glitch in all this time.
Compared to other telco-based NSPs, this is very refreshing
(and time saving).

-mark

Re: Anyone ever used calpop.com?

[Mark Kent]

 Has anyone ever had any experiences with calpop.com for
 colocation services? 

Are they Savvis, or just pretending to be Savvis:

http://www.calpop.com/network.html

I like it where they say

 CalPOP's Network has been rated the #1 rated backbone in the world --

and 

a) the contact for calpop.com is someone at hotmail.com
b) both calpop.com nameservers are on the same /24, likely
   plugged into the same switch and same power supply
c) both calpop.com nameservers report only one NS record,
   which is a third machine sharing the same characteristics
   as in b)
d) The TTL is one hour... I guess that's so they can pick up and
   move to, say, an XO data center real fast.

I think calpop.com is a cabinet inside a Savvis co-lo,
not that there's anything wrong with that.

-mark