Re: Networks ignoring prepends?

[Steve Gibbard]

To expand on what others have said here, I find it helpful to think of BGP as a 
policy enforcement protocol, rather than as a distance vector routing protocol. 
 

To that end, there’s a generally expected hierarchy of routes, and then a lot 
of individuality between networks.  Having done traffic engineering for some 
global CDNs, there’s a bunch of inbound traffic control that you can do by 
letting an understanding of how most other providers think about this guide 
your transit and peering policies, and a remaining portion that generally needs 
to be solved through either discussions, negotiations, or commercial 
arrangements with the sending party or their upstreams.

For the general rules, local-preference trumps everything else.  The number of 
AS path hops comes after local-preference.  Other things being equal networks 
usually like to hand off traffic to a short AS path, and at the closest point 
to its origination (there are valid performance reasons for this) but 
local-preference policies will override both of those.

Local-preferences usually have three default tiers — customer, peering, and 
transit.  In other words, get paid, hand off for free, and pay.  There are 
often some additional peers that can be selected for traffic engineering 
reasons, either internally or by customers using BGP communities.  BUT, those 
BGP communities don’t transit to other ASes, so even if you manage to signal 
one hop up stream, you may still find your upstream provider announcing your 
routes to those who have different ideas.

One example of this from the early days of anycasted DNS root servers involved 
k.root-servers.net  installing a node in Delhi, 
which pulled 60% of its traffic from North America.  This was clearly 
non-optimal.  They had attempted to get routing diversity by getting transit 
from different providers in different parts of the world, but their Delhi node 
was, if I recall correctly, a customer of a customer of a customer of Level3.  
Oops.

So, what do you do about this?

If you’re a global network operator, you probably attempt to maintain 
consistent peering/transit relationships across sites.  That way, AS paths and 
local-preferences should be fairly even, and you can let nearest exit routing 
do its thing.

If you have a smaller network, but have multiple interconnection locations that 
are far enough apart to make a performance difference, make the same transit 
and peering relationships at each one.  Make exceptions only for peers (not 
transit providers) whose customers or services only exist in one of the areas, 
and make sure they don’t announce your routes to their upstreams.  That way you 
won’t trombone traffic.

If you’ve done all that, and traffic is still coming in the wrong place, then 
you start talking to people.  “Hey, I’m buying transit from you in both Asia 
and the Western US, and all my traffic from asian-country-x is coming into San 
Jose.  Why?”  “Well, they only have a 100 Mb/s interconnection to us in Asia.  
We have to traffic engineer around it.”  And then you have to figure out how to 
convince some national telco to want to talk to you more than they want to talk 
to your transit provider.

I think in your case, I would be asking why you have a 5,000 mile, five-prepend 
loop to get to a provide ten miles away.  It suggests that your network is 
doing things 5,000 miles away that are inconsistent with what you're doing 
locally, or that you have upstreams who aren’t interconnecting locally or 
aren’t maintaining sufficient capacity or sufficient political relationships on 
those paths.  All of those would predictably have this result.  The solution is 
likely to take a look at your transit relationships, ask your transit providers 
about their transit relationships, and either supplement or switch to a set of 
transit providers who can provide the routing you want.

-Steve



> On Jan 22, 2024, at 4:49 AM, William Herrin  wrote:
> 
> Howdy,
> 
> Does anyone have suggestions for dealing with networks who ignore my
> BGP route prepends?
> 
> I have a primary ingress with no prepends and then several distant
> backups with multiple prepends of my own AS number. My intention, of
> course, is that folks take the short path to me whenever it's
> reachable.
> 
> A few years ago, Comcast decided it would prefer the 5000 mile,
> five-prepend loop to the short 10 mile path. I was able to cure that
> with a community telling my ISP along that path to not advertise my
> route to Comcast. Today it's Centurylink. Same story; they'd rather
> send the packets 5000 miles to the other coast and back than 10 miles
> across town. I know they have the correct route because when I
> withdraw the distant ones entirely, they see and use it. But this time
> it's not just one path; they prefer any other path except the one I
> want them to use. And Centurylink is not a peer of those ISPs, so
> there doesn't appear to be any community I can use to 

[NANOG-announce] Update on NANOG board and committee nomination process

[Steve Gibbard]

I would like to remind everyone about some important dates that are coming up 
for the NANOG governance process:

* September 17, 2012: The nomination process for NANOG Program 
Committee Candidates begins.
* October 1, 2012 the nomination process for the NANOG Board of 
Directors closes.


The NANOG Program Committee is a group of sixteen individuals from the NANOG 
community who together are responsible for the solicitation and selection of 
material for NANOG meeting programs.

Per the NANOG bylaws, eligible candidates each will serve a two-year term.  To 
be eligible to be appointed as a member of the Program Committee, an individual 
must have attended one NANOG conference within the prior calendar year (12 
months) and be a member in good standing.  Candidates should have a broad 
technical knowledge of Internet operations and be familiar with NANOG meetings. 
 Having constructive opinions and ideas about how NANOG meetings might be 
improved is of high value.  A willingness to recruit presentations for each 
meeting is required.  

Please send nominations to nominati...@nanog.org.  If you are nominating 
another person, please provide that person's name and email address.  If you 
are nominating yourself, please provide a Statement of Intent and a Biography, 
each with a suggested limit of 150 words.  For samples, please see the 2011 
candidate lists (http://www.nanog.org/governance/elections/2011elections/).


The NANOG Board of Directors is a group of six elected members and NANOG's 
Executive Director.   The Board of Directors is responsible for and works 
closely with the Committee Chairs to promote, support, and improve NANOG. The 
Board is responsible for the selection of the Program Committee, the 
Communications Committee, and the Development Committee. The Board is 
responsible to the members ensuring that the NANOG organization remains, open, 
relevant, useful, and financially sound.

Please read the Board Member Responsibilities 
(http://www.nanog.org/governance/BOD_Responsibilities.pdf) and NANOG bylaws 
(https://newnog.org/docs/newnog-bylaws-20110104.pdf) for a complete 
understanding of the expectations placed on Board Members.

To ensure continuity on the Board, three seats out of six become open each
year due to the expiration of 2-year terms.  The Board members whose terms
are expiring in October are:

*  Patrick Gilmore
*  Daniel Golding
*  Michael K. Smith

Patrick has served two 2-year terms and cannot be considered for re-election 
until October 2013 (one year leave).  Daniel is completing the term vacated in 
June 2012 and he can stand for re-election.  Michael is completing the term 
vacated in August 2011 and he can stand for re-election.


How do you Nominate?


You can self-nominate.  If you care about NANOG’s governance and want to take a 
turn at volunteering your time and expertise to help make it better:
1. Make sure you are a NANOG member in good standing
2. Submit your Declaration of Candidacy to electi...@nanog.org.

You can nominate others.
1. Send their contact information to electi...@nanog.org
2. If they accept the nomination, they will be asked to become a NANOG member 
in good standing
3. They will have to submit their Declaration of Candidacy to 
electi...@nanog.org.



As always, if you have a questions, please email nominati...@nanog.org.

Thank you for your support, and your participation in the community.

Thanks,
Steve Gibbard
for the NANOG Board


___
NANOG-announce mailing list
nanog-annou...@nanog.org
https://mailman.nanog.org/mailman/listinfo/nanog-announce

[NANOG-announce] Call for Volunteers -- NANOG Education Committee

[Steve Gibbard]

(my apologies to those receiving a second copy of this.  The first copy ran 
into a mail filtering issue and didn't go out to most of the list)

At the Vancouver meeting in June, I presented a preliminary proposal for a 
NANOG education initiative, which would put together a NANOG-created 
educational program for junior (and possibly more advanced) network operators.  
There was broad support from the community, and now it's time to refine the 
idea and turn it into something that can be implemented.  We are seeking 
volunteers to join the Education Committee and work on the final proposal and 
its implementation.  Among the issues that need to be decided are:

- What format should the classes have?
- What subject matter should they cover, and what should the curriculum be?
- Who should be teaching them -- volunteers from the community or paid 
instructors?
- Where should the classes be taught?  At NANOG venues before, after, or during 
the conferences?  At independent sites at non-conference times?
- Cost structures:  What should the classes cost and what will be included?
- Other sources of financial support:  Tuition?  Sponsorships?  Donations?  
Subsidies from the NANOG conferences?
- And all sorts of other issues

The expected commitment from members of the Education Committee will be as 
follows:

- Attend bi-weekly conference calls
- Research issues as needed, and provide feedback to the group

The goal will be to have a reasonably solid proposal in time for the October 
NANOG meeting, and a final proposal in time for the February meeting.

If you are interested in volunteering for this committee, please contact me.

Thanks,
Steve Gibbard
NANOG Board
s...@newnog.org


___
NANOG-announce mailing list
nanog-annou...@nanog.org
https://mailman.nanog.org/mailman/listinfo/nanog-announce

Re: Network device command line interfaces

[Steve Gibbard]

What this really comes down to, I think, is figuring out how your gut level 
concerns fit into the big picture, and to then put that into terms that the 
people responsible for the big picture can use to make a good decision.

Finances do matter.  Getting your employer to spend money it doesn't have to on 
network equipment generally means there's less money available to spend on 
other things that might matter, like making your network bigger, hiring people 
to help you, or even keeping you employed.  If you want to spend more on 
equipment than the bare minimum, you ought to have a reason.  To get anybody 
else to come to the same conclusion, you ought to be able to explain that 
reason.

That said, I think it's valid to buy something because you're comfortable with 
it, and valid to not buy something because you're not comfortable with it.  I 
don't want to buy new device X because I don't want to have to learn how to use 
it sounds lazy, but most of us are busy, and if the device you're comfortable 
with will do the job for an affordable price, it's generally good not to create 
extra work for yourself.

Saying we shouldn't do that because I don't know how is hard.  It may be 
because something is new and complicated, and nobody has experience with it.  
Or it may be because you're not familiar with it when lots of other people are. 
 You may have a different specialty, or it may be because you're less 
experienced than the people they could have hired if they'd paid or shopped 
around more.  But, your expertise or lack thereof is a legitimate thing to take 
into account when making decisions, as is the likely expertise of people who 
will have to manage the system in the future.

Unfamiliar network equipment is expensive to manage, whether the CLI is well 
done or not.  Even in a one person shop, you won't yet have encountered the 
device's pitfalls -- its easily circumventable bugs, the configurations that 
seem intuitive but aren't, etc.  It's going to take you longer to design and 
configure your networks, and you're going to create problems by doing the wrong 
thing more often.  You're probably going cause some outages, or even buy 
equipment and then find that you missed something and need to buy something 
else.  If you work with a large team, and maybe even have NOC people working 
the night shift in another location supporting the thing, it gets worse.  All 
those people have to be trained on the new device, and come up to speed on it.  

It's also good to understand the reliability requirements for something you're 
building.  We don't have licensing requirements for Network Engineers, but some 
other more established engineering professions do.  If a structural engineer 
signs off on a building despite being unfamiliar with some aspect of the 
construction technology and the building collapses, that can be career ending.

Internet networks that have become pretty important too.  If you're building a 
network where a failure will cause a heart attack victim to not be able to call 
911 from their VOIP phone, it isn't good enough to say I've never seen this 
piece of equipment but I don't have any reason to think it won't work.  If 
you're building a network to connect some office PCs, the stakes are probably 
lower.

And, of course, there's also the option of learning about unfamiliar 
technology. Play with it in your lab.  Put it in a peripheral site that can 
fail without causing too many problems.  Get your NOC staff familiar with it.  
And maybe, in the end, you'll find that you actually like it.  That it does 
something your old hardware doesn't.  That cheap hardware lets you afford a 
level of redundancy, and thus reliability, that was simply unaffordable with 
you're previously preferred equipment.

But that testing and familiarity has a cost, just as buying expensive equipment 
does, and just as running unfamiliar equipment does.  It's a matter of 
balancing it all out, and coming to an agreement with your management on what 
the best strategy is.

-Steve

On Nov 25, 2011, at 8:15 AM, Joel Maslak wrote:

 On Fri, Nov 25, 2011 at 12:01 AM, Robert Bonomi 
 bon...@mail.r-bonomi.comwrote:
 
 
 The trick to deailing with this as a propellorhead[sic] is to include a
 *monetized* estimate of the increased manpower OPEX of using the 'dog to
 work with' box.  And a TCOS figure over the projected lifetime of the
 units.   No need to 'fight' with management about it, just understand
 'how'  they make the decisions, and give them the informatin they need
 to make the decision come out 'your way'.
 
 
 I'd say that the ethical thing to do is to give them the information they
 need to make a decision, not to get it your way.  I see, for instance,
 people buying local closet switches from brand A when brand B is much, much
 cheaper (but lacks the prestige of brand A), had a perfectly workable
 management interface, and will perform identically, with similar support
 offered by both vendors.  But 

[NANOG-announce] Amendments to the NANOG Bylaws

[Steve Gibbard]

NANOG members (and other interested people),

The NANOG bylaws provide an opportunity each year to amend them, if needed.
Bylaw amendments may be enacted by a majority vote of eligible voters (in other 
words, NANOG Members), during NANOG's annual election.  The election is 
scheduled to take place during NANOG 53, from October 9-11, with voting open to 
all NANOG members (http://www.nanog.org/membership_main.html).

The current bylaws are posted at 
http://www.nanog.org/governance/documents/By-Laws_20110104.pdf

An amendment may be put on the ballot by the Board of Directors or by petition. 
 A petition must be signed by at least 30 eligible voters, or 1 percent of 
eligible voters, whichever is greater.

In addition, the Board had the power to make bylaw amendments during NANOG's 
first year as an independent corporation, subject to ratification by the voters 
in the next election.  The Board used that procedure to amend the bylaws and 
institute the current membership structure, so that amendment will now need to 
be accepted or rejected by the membership.

Accordingly, there will be a proposal on the ballot to ratify the current 
membership structure (Section 5 of the bylaws).  The board has also discussed 
submitting a proposal to modify the committee structure called for in the 
bylaws (Section 9), to reflect that  some of the committee functions are now 
being handled by staff.

Any other changes people want to suggest may be discussed on the 
nanog-futu...@nanog.org mailing list.  Anybody wishing to put a bylaw amendment 
on the ballot without board action may do so by submitting a petition to 
nominati...@nanog.org no later than September 26.

Thank you,
Steve Gibbard
NANOG Membership Chair
___
NANOG-announce mailing list
nanog-annou...@nanog.org
https://mailman.nanog.org/mailman/listinfo/nanog-announce

[NANOG-announce] NANOG Membership Update

[Steve Gibbard]

NANOG Community,

As many of you know, NANOG has been taken over by a new membership-based 
non-profit corporation, NewNOG, Inc.  Things are going well so far.  The NANOG 
Intellectual Property transfer is moving along nicely. We've been signing up 
lots of sponsors for the Denver meeting coming up in June, and are getting lots 
of interest in sponsorship for next fall's Philadelphia meeting.  We expect to 
have great programs for those meetings as well.

We began signing up members during the Miami NANOG meeting at the end of 
January, and are now up to 160 members.  In addition to providing some of the 
organization's financial support, the membership makes the decisions on how the 
organization is run.  In effect, the membership is NANOG.

The current list of members is at https://newnog.org/members.php.  The 
membership policy is at https://newnog.org/membership-policy.php.

For those of you who haven't joined yet, we'd like to once again ask you to 
become members.  Membership is $100 per year, with a 10% discount for 
pre-paying for three or more years.  Student membership is $50 per year.  
Members receive a $25 per meeting registration discount.  You can join at 
https://newnog.org/join.php. We will also have a membership table set up at 
NANOG52 in Denver, which you can visit for information or assistance.

For those of you who were having trouble with our PayPal sign-up process, or 
for those outside the United States, we are now also accepting credit card 
payments via Google Checkout.

Thanks for your support,

Steve Gibbard
NANOG Membership Chair
NANOG Development Committee
___
NANOG-announce mailing list
nanog-annou...@nanog.org
https://mailman.nanog.org/mailman/listinfo/nanog-announce

Re: Level 3 Communications Issues Statement Concerning Comcast'sActions

[Steve Gibbard]

On Dec 1, 2010, at 5:47 PM, William Herrin wrote:

 On Wed, Dec 1, 2010 at 3:38 PM, Derek J. Balling dr...@megacity.org wrote:
 On Nov 29, 2010, at 10:25 PM, William Herrin wrote:
 There are a couple forms of shared billing.
 
 There's a third kind you failed to mention that doesn't require equal 
 footing of the parties. The broker.
 
 I might pay an apartment broker $X to help find me an apartment.
 In turn the apartment broker might match me up with an apartment,
 and charge the landlord $Y for a successful tenancy.
 
 Hi Derek,
 
 For the most part the apartment broker process doesn't work quite the
 way you think. Generally he either gets a fee from you to find you the

Regardless of whether the apartment broker comparison holds up, there are many 
examples of what economists call two-sided markets:

http://en.wikipedia.org/wiki/Two-sided_market

They don't all have the same fee-splitting systems, and you can find an example 
to site as precedent for just about any system you could reasonably advocate.  
An example raised in a talk I heard a few years ago was of scholarly journals 
that collect money from both their subscribers and their authors.  The authors 
need to be published in order to get tenure, and the readers pay because they 
want to know what the authors are saying.  Another example is the Golden Gate 
Bridge, which was funded in the 1930s by the rural counties north of the bridge 
(including one ~300 miles north), who wanted connectivity to San Francisco.

It's probably reasonable to generalize a bit and say that in the systems not 
imposed by regulators, the distribution of costs has something to do with how 
much each party cares, within the limits of each party's resources.  Whether 
the response produced by the market is at all fair is another -- far more 
subjective -- question, and that's where regulators come in.

-Steve

Re: [Nanog-futures] Proposed bylaws for NewNOG

[Steve Gibbard]

Thanks for all the feedback received so far (and I'm sure there will be 
much more, which will also be very welcome).

The membership section of this is a product of the Membership Working 
Group, chaired by Kris Foster.  Everything else in the bylaws came from 
the Governance Working Group.  I'll leave addressing the 
membership-related questions to them.

On the other points:

Rose Klimovich said:

 3. On legislation, I think if the government wants input on proposed 
 laws and regulations, Nanog can give that even though we are not a 
 lobbyist.

I'm assuming Rose was referring to the part of section 6 that says, No 
substantial part of the activities of the corporation shall be the 
carrying on of propaganda, or otherwise attempting to influence 
legislation, and the corporation shall not participate in, or intervene in 
(including the publishing or distribution of statements) any political 
campaign on behalf of or in opposition to any candidate for public 
office.

That part came from NewNOG's attorney.  I believe it is standard 501(c)3 
boilerplate, as it looks very similar to what I've seen in the bylaws of 
other 501(c)3 organizations.

My understanding is that there's a critical difference between giving 
advice, which is ok, and attempting to influence legislation, which 
is a disqualifier for 501(c)3 status.  I don't believe answering questions 
when asked for advice would run afoul of that section -- indeed, it's 
arguably an important educational function.  Handing out campaign 
contributions in an attempt to influence the Network Neutrality debate, 
for instance, would be.  There are certainly some grey areas inbetween, 
which might require seeking legal advice, but I don't think this provision 
restricts us any more than any of the other 501(c)3 organizations that 
regularly have representatives testifying before Congress are restricted.

Dan Golding says:

 I think there should be a codified budget and finance committee, to
 provide forward looking budgets as well as audit capability. I think
 these are important things to codify in the bylaws.

That sounds reasonable.

Should I use the following as the language for it, or would the current 
Working Group like the language to say something else?

  9.5 Budget and Finance Committee

  The Budget and Finance Committee will be responsible, along with the
  Executive Director and Board of Directors, for NewNOG's budgeting and
  financial planning.

  9.5.1 Budget and Finance Committee Membership and Selection

  The Budget and Finance Committee will consist of at least three members
  selected by the Board of Directors. Members of the Budget and Finance
  Committee may not serve concurrently on the Board of Directors. The
  chairperson of the Budget and Finance Committee will serve ex officio in
  a non-voting role on the Board of Directors, in order to facilitate
  communication between the two groups.

  Budget and Finance Committee members will serve a two-year term, with
  terms staggered such that as close to half as possible of the terms
  expire each year. No member will serve more than two consecutive terms,
  although additional terms may be served after a one-year interval.

  A Budget and Finance Committee member may be removed before the
  expiration of his or her term if at least five members of the Board of
  Directors  vote for the removal.

Michael Dillon wrote:

 Consider adding worldwide as follows:

 The purpose of NewNOG is to provide forums in the North American
 region for education and the sharing of knowledge for the
 worldwide Internet operations community.
 
 I think this helps clarify that although the focus for venues is on
 North America, that the focus of the work
 is not geographically restricted.

We kept the same language there that's in the current NANOG Charter.  It's 
certainly changable if there's a desire to do so, but I think the current 
language was the result of much negotiation five years ago when it was 
negotiated.

Does anybody else have a strong desire to change that?

Thanks,
Steve

On Sun, 19 Sep 2010, Steve Feldman wrote:

 The NewNOG governance working group, chaired by Steve Gibbard, has 
 published a set of proposed bylaws for the corporation.  These may be 
 found at:

  http://www.newnog.org/docs/newnog-bylaws.pdf

 Please take a few minutes to review these and make any comments or 
 suggestions.

 There will be a question on the ballot during the NANOG election next 
 month to ratify these bylaws.  Everyone eligible to vote in the NANOG 
 election will also be eligible to vote on this.

 Thanks,
Steve (for the NewNOG board)


 ___
 Nanog-futures mailing list
 Nanog-futures@nanog.org
 https://mailman.nanog.org/mailman/listinfo/nanog-futures


--
Steve Gibbard
s...@gibbard.org
+1 415 717-7842 (cell)

___
Nanog-futures mailing list
Nanog-futures@nanog.org
https://mailman.nanog.org/mailman

Re: Out-of-band paging

[Steve Gibbard]

On Wed, 28 Jul 2010, Joel M Snyder wrote:

But... you can take this sort of 'single point of failure' argument almost as 
far as you want.  In the security business (where I spend most of my time), I 
see people do this a lot--they get deep into the ultra-ultra-ultra marginal 
risk, which takes then an enormous amount of money to mitigate.  It's an easy 
rat hole to explore, and often fun.


I think people are getting lost in the weeds here, and confusing 
technologies with paths.


My current employer has been upgrading its transit circuits, and spent 
time in the last few months worrying about diversity of the transit paths. 
But we didn't insist that one provider come in via metro ethernet, one via 
SONET, and one via a GRE tunnel.  What we did was have them bring in 
network maps, and make them sell us circuits that weren't running down the 
same streets as our other providers.


The same goes for your paging network.  If it's running over IP, that's 
not a huge problem.  If anything, if you're an IP engineer, it probably 
makes it easier for you to audit the setup.  Where you do have a problem 
is if it's running over YOUR IP network, but that's just a more accute 
version of the problem you'd have if your paging company were using fiber 
along the same path as somebody you were buying fiber from.


So, for paging, or out of band management, or redundant capacity, the 
rules seem pretty simple.  Buy from somebody who's not your customer. 
Audit whatever information you can get about their network paths to verify 
that they're not sharing segments with you.  And, for good measure, have 
some backup plans in case the notifications don't work.


You probably are better off if you have humans in a NOC, rather than a 
purely automated alerting system.  Those people can notice if you're not 
responding, and be creative.  Maybe they can figure out how to fix 
problems themselves.  If all else fails, they may be able to dispatch 
somebody to your house.  Remember, organizations have been tracking down 
critical personnel for far longer than there have been telephones.


Or are people here worried about a scenario in which the entire world is 
run off of one big interconnected IP network, and that when it fails it's 
not only not possible to make a phone call, but also not possible to get 
across town to alert the people who could fix it?  It seems to me that if 
things really got that bad, it might be pretty hard for even the most 
oblivious on-call person to miss.


-Steve

Re: [Nanog-futures] Moving Forward - What kind of NANOG do we want?

[Steve Gibbard]

On Thu, 1 Jul 2010, Daniel Golding wrote:

 The way forward is to have sharp cut-off from having
 quasi-professional meetings and transition into having real events.
 Real events have real sponsorship models, not a few bucks for a break
 or a beer and gear. Real events are planned a year in advance, not a
 few months. Real events don't require hosts to dedicate a dozen staff
 members - they can just write a check.

I usually find myself agreeing completely with Dan in these future of 
NANOG discussions, but the hard line on sponsorship makes me 
uncomfortable.

I've certainly seen real events that function the way Dan describes. 
ISPCon comes to mind.  I'm sure the model makes a lot of sense for their 
for-profit organizers, but for attendees it tends to put up a lot of 
barriers and prevent the sort of freewheeling culture that, for me, makes 
NANOG and the other less formal operator meetings so useful.

NANOG, or NewNOG, or whatever it ends up being called, needs to bring in 
enough money to keep the organization running.  It doesn't need to -- and 
as a non-profit it probably legally can't -- run a big surplus.  Assuming 
we can make ends meet, I'd hate to prioritize money over creativity.

-Steve

___
Nanog-futures mailing list
Nanog-futures@nanog.org
https://mailman.nanog.org/mailman/listinfo/nanog-futures

Re: [Nanog-futures] Membership, was Transition update

[Steve Gibbard]

I went looking through old e-mails to see if I could figure out where the 
current membership system came from.  The earliest e-mail I could find 
outlining it was this:



Date: Mon, 22 Nov 2004 19:12:26 -0500
From: Daniel Golding dgold...@burtongroup.com
To: Stephen J. Wilcox st...@telecomplete.co.uk,
 Hannigan, Martin hanni...@verisign.com
Cc: nanog-ref...@nac.net
Subject: Re: [nanog-reform] Draft List reform plan


It doesn't have to be one person. Here's a possibility...

- An elected NANOG committee. Merit would have a representative on this
committee

- They appoint a group of FAQ maintainers/list admins. Preferably they 
would have little to do, except for unsubscribing jabbering mail clients 
and what not. This is a group of volunteers. One member of the elected 
committee should be the lead.

- They appoint a program committee to review presentations. This is a 
group of volunteers. One member of the elected committee should be 
coordinating this.

Other members of the elected committee could work with hosts, face the
conference, help steer the agenda. Merit would work with this group,
handling registration, signage, room setup.

I suggest an elected committee of 5-7 with staggered two or three year
terms. Electorate would be anyone who attended a NANOG meeting in the last
year (3 meetings).

- Dan



This looks looks a lot like what we ended up with.


Steve Wilcox then wrote:

On a related note, I was just thinking.. someone mentioned before an 
issue with committee elections in that nanog doesnt have members as such. 
There is a possible solution.. an annual membership subscription, there 
may be other uses to being a nanog member but in this context i'm thinking 
it would give you an electorate. Of course we dont want to increase 
overall costs so soemthing like a $300 annual fee would be given back to 
you as eg $450 of discounts to nanog meets (ie a $50 meeting discount 
assuming 3 meetings/ann as an incentive).


And Dan replied:

Here's the issue with an annual membership subscription - the only 
benefit would be a vote. The result would be a few purchased votes and a 
non-representative Steering Committee. And its not like Merit needs more 
money :)


There was then a lot of discussion about how to keep the voting process 
from being hijacked by organizations sending too many people to the NANOG 
meetings.  Dan suggested that the situation could be prevented by a secret 
ballot, such that employers wouldn't be able to check on how their 
employees voted.  There were a bunch of proposals to prevent employees of 
equipment vendors from voting, to keep the vendors from taking over NANOG. 
Fortunately, that idea didn't go anywhere.  Dan appears to have cut that 
discussion off with this:


Date: Wed, 22 Dec 2004 19:03:56 -0500
From: Daniel Golding dgold...@burtongroup.com
To: Adam Rothschild a...@latency.net, Steve Gibbard 
s...@stevegibbard.com
Cc: nanog-ref...@nac.net
Subject: Re: [nanog-reform] Issues to address


The easiest way to define the electorate is this:

Any person who has attended at least one NANOG conference in the last
calendar year is entitled to vote for steering committee members.

- Dan


That appears to have settled the issue.



Skipping forward to now, I kind of like the idea of having a professional 
organization with a more formalized membership.  At the same time, the 
current system seems to have worked remarkably well, and I'm not sure how 
much sense it makes to mess with it.

-Steve

___
Nanog-futures mailing list
Nanog-futures@nanog.org
https://mailman.nanog.org/mailman/listinfo/nanog-futures

Re: [Nanog-futures] Transition update

[Steve Gibbard]

On Fri, 28 May 2010, Joe Abley wrote:

 I think it's quite within the SC's mandate to do this without a vote, 
 and I also think there's no obvious mechanism within the bylaws to hold 
 a vote on any particular issue (we had our vote; we voted for members of 
 the SC).

The NANOG charter, under which the SC was elected, says NANOG is an 
activity of Merit.  It specifies a process for ammendments, which involves 
having a vote of the membership.

Of course, any group of people can start up a corporation, enter into 
contracts, and host conferences (which none of us have to attend). 
Therefore, the NANOG charter probably doesn't technically apply to this 
situation.  Still, the given that the charter is what gave the SC their 
unofficial legitimacy to contemplate such a move, it would be nice if they 
followed its spirit and held a vote.

I'm also curious about the financial story here.  If the corporation is 
signing hotel contracts, I assume it has some funds.  Are those coming 
from the Merit NANOG accounts, or is somebody else putting up the money? 
If the membership were to vote to stick with Merit as the NANOG 
administrator, is that something Merit would still be willing to do?

For the record, I'll say that if such a vote were held, I would probably 
vote for NANOG to become a standalone organization.  I also don't think 
this move would have come as a surprise to anybody involved in the hallway 
NANOG governance discussions at the Austin NANOG.  Given the questions 
that members of the Steering Committee were asking there, I don't think 
they rushed into at least the beginning of this process.

-Steve

--
Steve Gibbard
s...@gibbard.org
+1 415 717-7842 (cell)

___
Nanog-futures mailing list
Nanog-futures@nanog.org
https://mailman.nanog.org/mailman/listinfo/nanog-futures

Re: [Nanog-futures] [NANOG-announce] The Evolution of NANOG

[Steve Gibbard]

On Thu, 15 Apr 2010, vijay gill wrote:

 I have been to a few and I read the notes. Still not quite sure what
 problem this is hoping to solve.

As I said, I wasn't privvy to most of the discussions surrounding this.  I 
don't really know what's being proposed, or what's transpired between the 
Austin NANOG (when I last discussed this with some of the people involved) 
and now.

The catalyst for some of the discussion in Austin was that Merit had laid 
off some of the staff members involved in running the NANOG organization 
and producing the conferences.  There was a lot of apprehension among some 
who had been previously very happy with the job Merit was doing of running 
NANOG about what this meant, and whether Merit would remain committed to 
NANOG and its open governance process.

That's the end of what I know.  What follows is speculation.

The NANOG meetings, at least, are probably big enough that they do need 
professional coordination, no matter who owns them.  It seems conceivable 
that NANOG Inc. could be an Ann Arbor based organization, hire the 
organizing staff that Merit has laid off, and continue with the meeting 
staff status quo.  Or it could have different staff, be outsourced to some 
other industry organization, or contracted out to one of the many 
conference organizing companies.  I'm interested in seeing which direction 
the Steering Committee wants to go on this, or if they've gotten that far 
yet.

-Steve

___
Nanog-futures mailing list
Nanog-futures@nanog.org
https://mailman.nanog.org/mailman/listinfo/nanog-futures

[Nanog-futures] More information about this morning's announcement?

[Steve Gibbard]

I'm wondering if there's any more information available about the efforts 
to separate NANOG from Merit (as announced by Steve Feldman this morning 
in mail to the main NANOG list).

It sounds like there's already an effort to incorporate an organization. 
What sort of staff and budget is it expected to need?  Where would it be 
based?  Are there specific people lined up to run it, or is that more of 
an implementation detail than is being addressed now?

I understand these things take time, and that the official transition 
proposal won't be released for a few months.  Mostly I guess I'm just 
looking for raw data to satisfy my curiosity.  Steering Committee minutes, 
if that's where this is being discussed, would be fine.  But it looks like 
the most recent minutes on the Steering Committee website are from 
February 9, and don't mention this effort.

Thanks for all the work on this.  It was sounding from discussions at the 
Austin NANOG as if such a separation might be the right thing to do.  I'm 
not questioning the decision; just wanting a better view of the process.

Thanks,
Steve

___
Nanog-futures mailing list
Nanog-futures@nanog.org
https://mailman.nanog.org/mailman/listinfo/nanog-futures

Re: Resilience - How many BGP providers

[Steve Gibbard]

The thing to remember about redundancy is that it's a statistical game 
rather than a magic formula.


You can be reasonably sure that any single component will go down at some 
point.  Nothing works perfectly.  Few things last forever.


If you have two fairly reliable components, and if they're suffciently 
isolated from eachother that they won't be broken by the same event, it's 
much less likely that they'll both break at the same time.  That means 
that if one breaks, and you're not unlucky, you'll have time to fix it 
before the other breaks.


If you have three components, the chances of all three being broken at 
once are even less than the chances of two of them being broken at once. 
With four, you're even safer, and so on and so forth.  But once you get 
beyond two, you hit a point of diminishing returns pretty quickly.


That doesn't mean you should always do two of any given component.  Some 
things may be so important that you're not willing to take that level of 
risk and are willing to spend significantly more money to get a small 
amount more protection.  Some things may be sufficiently unimportant that 
you're willing to deal with occasional outages, and you can get by without 
a spare (few people -- with obvious exceptions who we don't need to hear 
about right now -- have fully redundant home connectivity, for instance). 
It's just a matter of understanding the risks, and doing the cost-benefit 
analysis to determine how much protection you need and how much you're 
willing to pay for it.


-Steve

On Wed, 11 Nov 2009, a...@baklawasecrets.com wrote:




Hi,

After recent discussions on the list, I've been thinking about the affects
of multiple BGP feeds to the overall resilience of Internet connectivity
for my organisation.  So originally when I looked at the design
proposals, there was a provision in there for four connections with the
same Internet provider.  Thinking about it and with the valuable input of
members on this list, it was obvious that multiple connections from the
same provider defeated the aim of providing resilience.

So having come to the decision to use two providers and BGP peer with
both, I'm wondering how much more resilience I would get by peering
with more than two providers.  So will it significantly increase my
resilience by peering with three providers for example, as both of the
upstreams I choose will be multihomed to other providers.  Especially as
I am only looking at peering out of the UK.

Hope the above makes sense.

Adel

[Nanog-futures] Smoke at NANOG meetings

[Steve Gibbard]

I'd like to thank the people at Merit and on the various NANOG committees 
for putting on what was for the most part another great NANOG meeting.

However, for those of us with sensitive resperatory systems, the bar and 
lobby, where much of the important stuff at NANOG happens, were really 
unpleasant places this time.  Even after making a point of leaving when 
the smoke got too thick, I was waking up the next morning with my nose and 
throat incredibly sore.

Given how rare it is to find hotel lobbies in the US or Canada, or even in 
Europe, that still allow smoking, it doesn't seem like it would be 
difficult to adopt a policy of only holding NANOG meetings in non-smoking 
hotels.  I'd like to request that Merit or the Steering Committee adopt 
such a policy.

-Steve

___
Nanog-futures mailing list
Nanog-futures@nanog.org
http://mailman.nanog.org/mailman/listinfo/nanog-futures

Re: Follow up to previous post regarding SAAVIS

[Steve Gibbard]

On Wed, 12 Aug 2009, Richard A Steenbergen wrote:


I would make the opposite argument, my business would NEVER go to any
network which didn't support IRR (and a bunch of other simple but
important things, like a full set of non-secret BGP communities). It's
amazing the number of networks that excludes in this day and age. And
not even because omg IRR is good because someone told me so and we
should support it, but because I've seen FAR too much grief caused by
humans typoing prefix-lists, or taking days to process them. It is the
height of absurdity that this would ever be considered an acceptable
solution to the problem.


I'd be very hesitant to use an upstream that didn't use any filtering 
method.  But, as convenient as I find the IRR system to be (from the 
customer perspective, at least), I'm quite happy that a couple of our 
upstreams use other mechanisms instead.


I've had prefixes fall out of the IRR a couple of times, leading to 
outages of IRR-using transit providers.  I've had transit providers screw 
up manually maintained prefix-lists, or had mis-communications resulting 
in the wrong thing getting removed.  With multiple transit providers and 
multiple systems, they tend not to all have the same filtering problem at 
the same time.  That's a very good thing.


I hope the recommendation that comes out of this discussion is to filter 
somehow, rather than to use any particular filter-generation mechanism. 
Diversity and redundancy are good things, in processes as well as 
hardware.


-Steve

Re: Recommendations for Hong Kong datacenter, and a sanity check for my geopolitical conclusions ?

[Steve Gibbard]

My take on this would be that DNS especially, and the volume of mail that 
can be handled via a few 1 and 2u servers, are pretty easy to duplicate. 
As such, I suspect you're overthinking some of the risk management pieces. 
In any of the places you mentioned, you're more likely to have random 
accidental power or network connectivity outages than to be dislodged by a 
tsunami, hurricane, or military coup.  No matter where you go, if you 
design your service such that it can fail over to your network sites 
elsewhere in the world, you should be fine.


I ran a 30-location DNS network that included servers in some fairly 
unstable places for about four years.  Power outages in one location or 
another happened a couple times a week sometimes.  The ones we worried 
about were the ones where the equipment didn't successfully reboot itself 
afterward.  Hardware failures happened periodically -- again often enough 
that I don't have a clear count.  We had one location that we lost 
connectivity to due to a coup for maybe a week, once.


The real questions to be asking are where you'll get the best network 
connectivity and support.  For network connectivity, Hong Kong, Singapore, 
and Tokyo will all be decent choices.  Tokyo can be difficult if you don't 
have a Japanese speaker on staff.  Hong Kong and Singapore are both full 
of people who speak good English.  Last time I looked at it, transit 
connectivity was cheaper in Hong Kong.  Peering was easier in Hong Kong as 
well, since everybody was on the HKIX rather than being split between two 
exchanges (SOX and Equinix) as they were in Singapore.  But it's been a 
few years since I've dealt with stuff in either place, so the situation 
may have changed.


As for facilities, my usual shopping technique is to figure out who I want 
to connect to, figure out where they are, and then figure out which 
building has the best combination of price and remote hands support.  If 
there are any discernable differences in the level of back-up power they 
provide, you may want to take that into consideration too.  And then 
remember, your equipment will be far away.  Things will happen to it that 
you don't expect.  Some of those will be hard to fix from a distance. 
Make sure you're able to fail over to equipment in other places if you 
need to, because if you do this enough, you will lose a site somewhere 
eventually.


-Steve

On Fri, 24 Jul 2009, George Sanders wrote:




I will be expanding a small network infrastructure service (read: DNS 
and mail ... a few 1u and 2u servers) to Hong Kong next year.


We don't have any particular customer base in Hong Kong - rather, we 
have customers all over southeast asia and would like to serve them 
better, as well as attract more SE Asia customers.


I chose Hong Kong for the following reasons:

- South Korea is alternately happy with / upset with Japan, and I don't 
want to deal with that


- Japan is is alternately happy with / upset with South Korea, and I 
don't want to deal with that


- Mainland China is out of the question, for obvious reasons

- The smaller (Thailand, Vietnamese, Phillipines, etc.) countries all 
have their own particular issues (recent coup in Thailand, etc.)


So the choice came down to Hong Kong or Singapore, and I chose Hong Kong 
because it seems easier to just get things done there.  I realize that 
in the long term there is a greater risk of social paradigm shift in 
Hong Kong because of mainland China, but in the short run it seems that 
Hong Kong is more functional than Singapore.


Any comments on the above thought process ?


The obvious follow-up is, which datacenter ?

I need a full service center that will give me rackspace and let me just 
plug ethernet into their switch.  I am not interested in brokering my 
own connectivity, nor am I interested in running my own routers.  I want 
to pay one bill to one organization and get one cable.  The end.


I think there are further considerations though ... I read details of 
one very modern, very sexy datacenter housed in a skyscraper, but my 
research showed me that this building has been built on land reclaimed 
from the sea, and there is reasonable concern that the sand 
underpinnings could liquify, to a degree, in a seismic event.  I'd also 
like to be more than a few feet above sea level.  Honestly, as sexy as 
it would be to be in a slick tower right on the bay in Central Hong 
Kong, I would much rather find some nondescript, one story building, 
miles from the coast and a few hundred feet above sea level.


What recommendations might someone have ?

Thank you very much for any comments or suggestions you may have.

Re: One /22 Two ISP no BGP

[Steve Gibbard]

On Fri, 6 Feb 2009, Jason Biel wrote:


As I mentioned earlier, you'll want to have one provider announce the /22
unweighted and the other announce it weighted.  Just pick the better of the
two providers as the primary.  Don't base it soley off bandwidth, but check
your SLA and any recent outage occurances.

Traffic will flow in via the primary until that link to you drops, the
provider will remove the route, and traffic will come in the back up route.


This is unlikely to work, on a couple of levels.

Given the same prefix-length on both announcements, you're unlikely to 
have much luck keeping traffic off your back-up path as Jason suggests. 
This means you'll need to have ways to withdraw the routes through both 
providers if their respective links fail, rather than just being able to 
withdraw the routes from one.


I'm not sure what he means by doing a weighted announcement.  If he 
means using the Cisco weight attribute, it's local to the router where 
it's set and won't propagate upstream.  Your upstream providers could use 
that to control how traffic exits their networks, but not how traffic gets 
to them.


Indeed, given two routing announcements of the same route to two different 
upstream providers who connect to the rest of the Internet in different 
ways, the announcing network has very little control over which route will 
be followed.  Once an announcement is out there, routing decisions get 
made according to the policies of the networks sending traffic in the 
announcing network's direction, which are generally based more on customer 
relationships than on topological distance or anything that can be set on 
the announcing end.


The usual way to attempt to influence inbound traffic flow is with AS path 
prepending -- making one path into a network look artifically long so that 
the other will look comparitively short.  This partly works.  Those who 
don't have any other reason to prefer one path over the other will prefer 
the shortest one.  But it's not going to shift 100 percent of inbound 
traffic.  The upstream provider, and their upstream providers, and anybody 
upstream from them, will probably all be using the BGP local-preference 
attribute to prefer paths they get paid for over paths they don't, and 
local-preference trumps AS path length no matter how many prepends are 
put into a path.


As others here have suggested, you could have the provider that won't do 
BGP with you tie their own BGP announcement to your interface, such that 
if the interface facing you goes down the route will go away.  Or you 
could have them use Cisco's conditional routing feature to only announce 
your route if they stop seeing your route being announced through your 
other provider.  The problem with both of these approaches is that they 
depend on some BGP routing flexibility on the part of your upstream 
provider, and if your upstream provider were flexible in terms of how they 
handle BGP for customers we wouldn't be having this discussion.


If you did want to follow Jason's suggestion of having primary/backup 
providers, such that inbound routing decisions are made based on whether 
the primary one is up, the tool you've probably got available is to 
announce more and less specific routes.   Barring filters in your 
upstream providers' networks, a more specific route will always be chosen 
over a less specific one.  So, if you've got a /22, you could have your 
non-BGP-speaking provider announce it as a /22 on your backup link, and 
announce it yourself as two /23s on your BGP-aware primary link.  That 
should more or less work, at the cost of having two more routes in the 
global routing table and getting you some dirty looks from peers who will 
consider it irresponsible.


That said, if you've got the resources, I think tunneling over the 
uncooperative provider to somebody who will do BGP with you on the 
mainland is probably a better answer.


-Steve

Re: [Nanog-futures] [NANOG-announce] Election reminder - charter amendments

[Steve Gibbard]

On Thu, 2 Oct 2008, Philip Smith wrote:

 Hello everyone,

 Please take a moment to look at the current charter amendment proposals
 for the October ballot at:

  http://www.nanog.org/charter/

 If you have comments on the proposals, please post them on the
 nanog-futures list or send them to [EMAIL PROTECTED] in the next few days.

A and B lok fine to me, although given that serious mischeif would require 
the participation of multiple steering committee members, I'm not sure how 
important it is that an individual one be easil removable.

Most of the charter clean-up thing (C) looks fine to me.  I notice that 
the last clean-up point in C turns the power the membership has currently 
to recommend changes to the charter into the power to actually change the 
charter.  My recollection is that the recommended wording is there 
because having ultimate authority over the charter was important to Merit, 
who might agree to change it in response to recommendations.  Has that 
changed?

-Steve

___
Nanog-futures mailing list
Nanog-futures@nanog.org
http://mailman.nanog.org/mailman/listinfo/nanog-futures

Re: Atrivo/Intercage: Now Only 1 Upstream

[Steve Gibbard]

On Wed, 17 Sep 2008, David Ulevitch wrote:

Reputation based on src_addr is /so/ 2005.  ASN has a few more legs 
perhaps... but...


All the growth in Internet-connected compute clouds (EC2, AppNexus, GoGrid, 
etc.) makes any system based around IP reputation decidedly less useful.


At the end of the day, nobody is going to drop packets for amazon's IP space.


While I can't speak for the others on your list, we have been putting a 
fair amount of thought into abuse detection and mitigation at GoGrid.  We 
are well aware of the problems we would have if our address space were to 
end up with a bad reputation.  If stuff does get through that shouldn't, 
please contact [EMAIL PROTECTED] and we'll take care of it.


-Steve

Re: InterCage, Inc. (NOT Atrivo)

[Steve Gibbard]

On Fri, 12 Sep 2008, Patrick W. Gilmore wrote:

Going back a bit in case you forgot, we were discussing the fact you have NO 
RIGHT to connect to my network, it is a privilege, not a right.  You 
responded with: If I have either a peering agreement ... then that contract 
supports my 'rights' under that contract persuant to my responsibilities 
being fulfilled.  Then you posted this contract as an example of those 
rights.  From the contract you claim to be a great model:


Calm down, Patrick. :)

It's probably correct that any individual player in this industry not 
under other regulatory restrictions can refuse to do business with 
somebody they don't like, sometimes.  For the industry as a whole to make 
a group decision to not do business with somebody who may be a competitor 
seems more legally risky.  Engaging in that sort of thing without getting 
some good legal advice first would certainly make me nervous.


Since this appears to be somebody who is contracting with lots of US 
providers, their identity is presumably known.  This discussion has now 
been going on for long enough that it's presumably passed the emergency, 
act now; think later, phase.  Should what they're doing be a law 
enforcement issue, rather than a they've got cooties issue?


-Steve

Re: BGP Attack - Best Defense ?

[Steve Gibbard]

On Fri, 29 Aug 2008, Scott Weeks wrote:

I am signed up for the Prefix Hijack Alert System 
(phas.netsec.colostate.edu) and would be alerted in about 6 hours (or 
less?) about a prefix announcement change.


I then would deaggregate (as little as possible) to be able to announce 
the same more specific as the attacker.


Announcing the same prefix length as the attacker would get you back some 
portion of your traffic, rather than all of it.  You'd really want to 
announce something more specific than what the attacker is announcing.


Of course, then you'd need to get your upstreams to accept the more 
specific, which might mean modifying filters.  How quickly can you get 
your upstreams to do that?


Also, please don't be like Covad.  If you deaggregate to deal with a 
highjacking, make your deaggregation temporary, and clean it up when it's 
not needed anymore.


I would then try to contact the ASs still using the attack path to get 
it stopped.  (Yell help on NANOG? ;-)


If you try to contact networks that are innocently hearing the 
announcement, rather than those involved in propagating it, you'll have a 
lot of networks to contact.  A better move would be to contact those 
originating the announcement (unless you think they're involved in 
something malicious), and then their upstreams, and if that doesn't work, 
their upstreams' upstreams.


Calling an upstream provider's NOC to ask them to modify a customer's 
filters generally gets met with lots of skepticism.  You'll almost 
certainly be told that you have to be the customer whose filter it is to 
ask to have it modified.  You'll need to be quite firm, and will probably 
need to ask to speak to somebody higher up than the front-line tech who 
answers the phone.  The very few times I've had to do this, I've also 
found it quite useful to deemphasize their receiving of the prefix from a 
customer, and emphasize that they were announcing it to the rest of the 
world.  You are announcing our prefix, and you are not authorized to do 
so, is a useful line.


-Steve

Re: EC2 and GAE means end of ip address reputation industry? (Re: Intrustion attempts from Amazon EC2 IPs)

[Steve Gibbard]

On Sun, 22 Jun 2008, Paul Vixie wrote:


it seems that amazon has succeeded where google and microsoft failed.  with
e-mail only services like hotmail and gmail, it was still possible to treat
an IP address as having a reputation, and to therefore blackhole hotmail
and gmail (and other free e-mail services) due to the spam emanating from
them, even though they are shared IP addresses and also emit much non-spam
traffic.


Even assuming Amazon will do as bad a job of policing EC2 as Paul suspects 
they will, I'm not at all convinced that customers would miss EC2 more 
than they'd miss mail from Hotmail or GMail.


Paul has said in the past that he refuses e-mail from the various free 
webmail services.  If that works for him, great, but I suspect the typical 
e-mail service customer wouldn't consider the resulting spam savings worth 
the potential downside.  If I did that on my own servers, I'd probably 
miss out on most of the e-mail I care most about receiving, since my 
friends and relatives seem to like free webmail services.  Given the 
number of legitimate free webmail users out there, and the number of 
people who like getting mail from them, I suspect any service provider who 
tried to block them would end up with a lot of angry former customers.


Likewise, anybody blocking EC2 would miss out on whatever bad stuff might 
be coming out of EC2, but would miss out on being able to access services 
hosted there as well.  Would they miss it more than they'd miss their 
friends on GMail?  That seems far from guaranteed.


So yeah, if big shared services that include important stuff aren't being 
adequately policed, that's probably a problem for IP address reputation 
services.  But that's not really a new problem being introduced by EC2.


-Steve

Re: [Nanog-futures] [Outages] Outages have an Outage? (fwd)

[Steve Gibbard]

I've been seeing two kinds of discussion on the NANOG mailing list about 
the Outages mailing list.  I've seen people post asking about various 
outages and be redirected to the Outages list, and I've seen outage 
notifications saying that the Outages list has been down for days (or 
maybe longer) at a time.

Is the Outages list of interest to the NANOG community?  Should NANOG, 
which if I'm remembering correctly has a nice shiny new professionally 
managed mail server take over the operation of the Outages list in order 
to improve its reliability?

-Steve

On Tue, 17 Jun 2008, Gadi Evron wrote:

 Lightning storm, subsequent commercial power failure. UPS not up due to 
 restructing.

 We are working on getting backup servers alive, as to DNS we used to 
 secondary at vixie's, but due to IP changes and movements removed that for 
 now.

 A comedy of mistakes.

 Details below.


 -- Forwarded message --
 Date: Tue, 17 Jun 2008 21:18:45 +
 From: Randy Vaughn [EMAIL PROTECTED]
 To: Jason Iannone [EMAIL PROTECTED]
 Cc: [EMAIL PROTECTED]
 Subject: Re: [Outages] Outages have an Outage?


 Hi Jason et al!

 The rack restructuring was only able to proceed as far as one
 rack and did not make to the point of attaching the isotf
 server to a UPS.  As luck has it, our provider's location
 lost commercial power this morning.

 The remainder of the rack restructuring will be announced once
 we have a firm timeline established for that activity.


 On a slightly different matter.  I appreciate the problems caused by
 high list volume.  Digest mode is one potential solution,  list
 moderation is another.

 R

 On 14:55 Tue 17 Jun , Jason Iannone wrote:
 We weren't able to resolve anything on amigostecnicos.net.  isotf
 admin care to elaborate?
 ___
 Outages mailing list
 [EMAIL PROTECTED]
 http://isotf.org/mailman/listinfo/outages
 ___
 Outages mailing list
 [EMAIL PROTECTED]
 http://isotf.org/mailman/listinfo/outages


___
Nanog-futures mailing list
Nanog-futures@nanog.org
http://mailman.nanog.org/mailman/listinfo/nanog-futures

RE: Hauling gear around a NANOG meeting

[Steve Gibbard]

I hesitate to weigh in here, but my observation after several years of 
doing a fair bit of traveling to a wide variety of places is this:  In any 
big city, anywhere in the world, there will be plenty of people ready with 
lectures on how this is a big city, and is therefore a dangerous place. 
You need to be careful.  Often, this will be repeated with escalating 
tones of alarm if it becomes clear that I've been ignoring it.  Sometimes 
the claim will be that their city is especially dangerous, and sometimes 
the claim will be that it's dangerous just like any other big city. 
Sometimes it takes on the form of this is a really safe city, but don't 
go out at night.  It doesn't matter.  Some cities really are dangerous, 
and some seem quite safe, but there's no quantifiable difference between 
lectures received in places that really are dangerous and places that 
aren't.


-Steve

On Fri, 23 May 2008, Paul Stewart wrote:


A lot of it is common sense - New York is a GREAT city .. no question
and very safe overall.  But common sense will tell you not to take a
leisure walk through Harlem at 3AM .. having said that, I've walked
through Central Park (65th St.) at various times of the night and never
had a problem, but then again that's different too...

Travel in herds and mind your own business - don't travel at 3AM (on
foot) and you'll be fine..;)  That really goes for any city when you
think about it...

Take care,

Paul

-Original Message-
From: Alex Rubenstein [mailto:[EMAIL PROTECTED]
Sent: Thursday, May 22, 2008 5:06 PM
To: Rod Beck; David Diaz; Martin Hannigan
Cc: nanog@nanog.org
Subject: RE: Hauling gear around a NANOG meeting


I hate to break the news to the New York bashers, but New York is one

of

the safest American cities. This is not a controversial statement.


While I generally agree with what Rod is saying, saying NYC is safe is
like saying all routers are cisco

There are safe areas, and there are not safe areas. I don't know how the
Brooklyn side of the Brooklyn bridge rates, but I don't think I'd be
overly concerned. And, since people going to NANOG tend to have a
herding instinct, there shouldn't be a problem.



New York has a lower incidence of crime than Miami, Detroit, Seattle,
Los Vegas, Houston, Atlanta, DC, Los Angeles, and Philadelphia.


Yes, but in at least most of those locations, my Florida or Utah CCW is
valid.








The information transmitted is intended only for the person or entity to which it 
is addressed and contains confidential and/or privileged material. If you received this 
in error, please contact the sender immediately and then destroy this transmission, 
including all attachments, without copying, distributing or disclosing same. Thank 
you.

Re: [NANOG] OSPF minutia, and, technote publication venues

[Steve Gibbard]

On Tue, 6 May 2008, Nathan Ward wrote:

 This stuff about customers and things sounds too hard.

 Steve, have you actually had to do anycast without having control of
 the routing hop in front of your service providing hosts, or is this
 getting unnecessarily complicated? I'd imagine that the ability to
 install routing equipment would be a pre-requisite for any anycast
 service deployment..

Yes I have.  Or rather, I've done the network infrastructure for anycast 
services without having administrative control of the anycasted servers. 
PCH's anycast platform hosts some blade servers for some other DNS 
infrastructure operators (in addition to the name servers PCH operates 
itself).  Those operators operate their own servers.  PCH operates the 
routing infrastructure.  There is filtering in place to limit the routing 
announcements from the servers.

But also, most of the larger organizations I've worked for have had 
separate systems and network engineering groups.  In general, the network 
groups haven't wanted to let the systems engineers configure the routers, 
and the systems groups haven't wanted to let network engineers configure 
the servers (with good reason).  Filtering of routing announcements from 
anycast servers would be useful in that environment too.


To address Paul's point about multipath BGP, I never saw Cisco's 
implementation of it causing a problem even with full routing tables.  I 
haven't used any other implementations.

In the Cisco version (and at least for EBGP; I haven't looked at this with 
IBGP), it only applies to otherwise identical AS paths.  Multiple 
directly-connected DNS servers sourcing the same announcement with the 
same AS path and other BGP attributes get load balanced between.  Paths 
learned from different peers had different AS paths and do not get 
balanced between.  I suppose there probably is load balancing in cases 
where there are multiple sessions with the same peer at the same exchange. 
That's a relatively rare case in this implementation, and using hash based 
rather than per-packet load balancing makes it not really matter.

-Steve

___
NANOG mailing list
NANOG@nanog.org
http://mailman.nanog.org/mailman/listinfo/nanog

Re: [NANOG] Did Youtube not pay their domain bill?

[Steve Gibbard]

On Sun, 4 May 2008, Paul Vixie wrote:

 [EMAIL PROTECTED] (Steve Gibbard) writes:
 The right solution is to design the anycast servers to be as sure as
 possible that the route will go away when you want it gone, but to have
 multiple non-interdependent anycast clouds in the NS records for each
 zone.  If the local node in one cloud does fail improperly, something will
 still be responding on the other cloud's IP address.

 the need for multiple independent anycast clouds is an RFC 2182 topic, but
 joe's innovation both in ISC-TN-2004-1 and in his earlier ISC-TN-2003-1 (see
 http://www.isc.org/pubs/tn/isc-tn-2003-1.txt is that if each anycast cluster
 is really several servers, each using OSPF ECMP, then you can lose a server
 and still have that cluster advertising the route upstream, and only when you
 lose all servers in a cluster will that route be withdrawn.

This is getting into minutia, but using multipath BGP will also accomplish 
this without having to get the route from OSPF to BGP.  This simplifies 
things a bit, and makes it safer to have the servers and routers under 
independent control.

But yes, Joe's ISC TechNote is an excellent document, and was a big help 
in figuring out how to set this up a few years ago.

-Steve

___
NANOG mailing list
NANOG@nanog.org
http://mailman.nanog.org/mailman/listinfo/nanog

Re: [Nanog] ATT VP: Internet to hit capacity by 2010

[Steve Gibbard]

On Mon, 21 Apr 2008, Sean Donelan wrote:

 The rest of the story?

 http://www.usatoday.com/tech/products/services/2008-04-20-internet-broadband-traffic-jam_N.htm

   By 2010, the average household will be using 1.1 terabytes (roughly
   equal to 1,000 copies of the Encyclopedia Britannica) of bandwidth a
   month, according to an estimate by the Internet Innovation Alliance in
   Washington, D.C. At that level, it says, 20 homes would generate more
   traffic than the entire Internet did in 1995.

 How many folks remember InternetMCI's lack of capacity in the 1990's
 when it actually needed to stop installing new Internet connections
 because InternetMCI didn't have any more capacity for several months.

I've been on the side arguing that there's going to be enough growth to 
cause interesting issues (which is very different than arguing for any 
specific remedy that the telcos think will be in their benefit), but the 
numbers quoted above strike me as an overstatement.

Let's look at the numbers:

iTunes video, which looks perfectly acceptable on my old NTSC TV, is .75 
gigabytes per viewable hour.  I think HDTV is somewhere around 8 megabits 
per second (if I'm remembering correctly; I may be wrong about that), 
which would translate to one megabyte per second, or 3.6 gigabytes per 
hour.

For iTunes video, 1.1 terabytes would be 1,100 gigabytes, or 1,100 / .75 = 
1,467 hours.  1,467 / 30 = 48.9 hours of video per day.  Even assuming we 
divide that among three or four people in a household, that's staggering.

For HDTV, 1,100 gigabytes would be 1,100 / 3.6 = 306 hours per month.  306 
/ 30 = 10.2 hours per day.

Maybe I just don't spend enough time around the leave the TV on all day 
demographic.  Is that a realistic number?  Is there something bigger than 
HDTV video that ATT expects people to start downloading?

-Steve

___
NANOG mailing list
NANOG@nanog.org
http://mailman.nanog.org/mailman/listinfo/nanog

Re: [Nanog-futures] The Peering BOF and the Fallout?

[Steve Gibbard]

I'm sorry to be a bit contrarian here, but...

Looking at the crowd that assembles for the peering BOF, it's clearly one 
of the more popular things on the NANOG program.  It may not draw the raw 
numbers of people that the general session does, but it does tend to pack 
whatever room it's in.  People in the room tend to be attentive and 
engaged, whether or not they have anything to do with peering.  It's a lot 
of fun, and it's clear to me that enough attendees want it on the program 
for it to be worthwhile.

That said, while the latest one struck me as a vast improvement over the 
last few, I can't say I've actually learned much from most of the recent 
peering BOFs or from the last few exchange point operator forums I've been 
to.  The agendas tend to strike me as entertaining but recycled filler, 
perhaps useful for getting people into a room and talking, but not nearly 
what they could be.

When I think back to the peering BOFs and exchange operator-sponsored 
forums of several years ago, I used to come out of them with some better 
understanding of how peering worked.  There were talks on things like how 
much of peering traffic was P2P back when that was new and scary.  Large 
parts of the program were made up of peering personals, where I would 
learn who was looking for what sort of peers.  In addition, there were 
exchange operator-sponsored forums, in which people would give talks about 
peering-related issues they had faced and how they had solved them, 
observations about how peering worked in other parts of the world, views 
into highly secretive tier 1 peering operations, and the like.

The exchange operator-sponsored forums are now gone, having been replaced 
by parties where the content consists of fake game shows.  The peering BOF 
content now consists of things like the great debates, which while it's 
entertaining to to see people trying to justify extreme positions, never 
feel to me like they get anywhere close to establishing what the right 
answer to the question being debated -- presumably somewhere between the 
two extremes -- would be.

So, I wouldn't suggest that the current peering BOF or exchange 
operator-sponsored forums go away.  They're good fun social events and 
NANOG could often use more of those.  But I don't think we've run out of 
new things to say, or new issues to address, in the areas of peering and 
other forms of interconnection.  It would be nice if there were some more 
serious forums as well.

(And yes, I know, this counts as sniping from the sidelines.  The big 
impediment to what I'm asking for here is presumably having somebody step 
up and organize it).

-Steve

On Sun, 24 Feb 2008, Ren Provo wrote:

 On behalf of the NANOG PC:

 Nothing has been submitted in the NANOG tool and nothing has been declined.

 The survey results from NANOG42 this week have not been made available to
 the PC yet.

 We would like to review community feedback on this topic.

 Hallway discussions this past week in San Jose suggest some would like to
 see a more diverse selection of topics at the very least.
 Bill was asked on Wednesday not to make commitments until we, the NANOG PC,
 are able to review feedback and perhaps expand the cramped format into a
 track.

 Thanks, -Ren Provo, NANOG Program Committee, Vice-Chair

 On Sun, Feb 24, 2008 at 8:00 PM, Patrick W. Gilmore [EMAIL PROTECTED]
 wrote:

 On Feb 24, 2008, at 12:57 PM, Joel Jaeggli wrote:
 Chris Malayter wrote:

 Would you ask the PC to release the minutes from the SJC nanog and
 any
 meeting since.

 Given that the pc last met on tuesday at lunch, I think the minutes
 when
 released will prove to be a poor source the sort information you're
 looking for.

 Let's stop dancing around the issue.  There was discussion regarding
 the Peering BoF amongst the SC  PC.  There is no reason to hide this
 fact - just the opposite.  And there were at least some provisional
 outcomes from those discussions.  I am unclear on why those decisions
 are not being announced to the community.

 The question is where we stand in the process.

 If the PC does not have an official stance, then we should all stop
 speculating until there is an official stance or (hopefully) an
 official request for input from the community.

 If the PC has an official stance, then the community needs to hear it
 ASAP.

 Either way, gossiping on a mailing list is not the right way.  We had
 a revolution, let's follow our own rules.  As Randy like to proclaim
 every 14 ms, let's have some transparency.  What was said, why was it
 said, and what decisions were made?

 SC / PC members, please step up, so we can all go back to arguing over
 leaking deaggs. :)

 --
 TTFN,
 patrick


 ___
 Nanog-futures mailing list
 Nanog-futures@nanog.org
 http://mailman.nanog.org/mailman/listinfo/nanog-futures



___
Nanog-futures mailing list
Nanog-futures@nanog.org

Re: [Nanog-futures] Objection: RE: [admin] Re: EU Official: IP Is Personal

[Steve Gibbard]

On Tue, 29 Jan 2008, Robert E. Seastrom wrote:


 Pete Templin [EMAIL PROTECTED] writes:

 And seriously, can we stop with the if you don't like it, you must
 volunteer to serve on it to effect your desired changes mantra?

 Why?  The people who bellyache and the people who have skin in the
 game are by and large a disjoint set.  As someone who's put up (in
 more ways than one), I encourage those who are not willing to put up
 to shut up.

Speaking as somebody who has put up a few times, and who has been more 
recently shutting up most of the time...

 For the record, I don't care if that particular thread dies; it'd
 strayed off-topic.  However, I think the policy interpretation is too
 strict and warrants clarification.

 Reasonable people may disagree with any particular MLC action,
 however, I don't think that overall policy interpretation is too
 strict right now.

It seems to me that there are two issues, topicality and quality.

I'm not generally finding the NANOG list worth reading these days, and 
that makes me sad.  I don't think I've noticed anything particularly 
off-topic recently.  The mailing list committee must be doing a good job 
of dealing with that sort of thing.  What I am seeing is discussion 
threads going on and on and on, long after there's nothing new left to 
say.  Mostly this seems to be a fairly small group of people who appear to 
feel compelled to voice strong opinions over and over again on every topic 
that comes up, whether it's something they know anything about or not.  I 
don't think those people add any value to the discussion, and I don't 
think the hordes of people who generally jump in to argue with them from 
different but equally uninformed perspectives do either.  But, most of the 
time those people are on-topic.  They're just not useful or interesting.

I'd be quite happy to see the list administrators going to some of the 
most frequent posters and asking them to post less, whether on topic or 
not.

-Steve

___
Nanog-futures mailing list
Nanog-futures@nanog.org
http://mailman.nanog.org/mailman/listinfo/nanog-futures

Myanmar Internet turned off

[Steve Gibbard]

There have been several news stories today about Myanmar's government 
turning off the country's Internet connectivity to suppress news coming 
out of the country (for instance: 
http://www.nytimes.com/2007/10/04/world/asia/04info.html?ref=world). 
Doing some poking at it earlier today, here's what I found:


The .MM top level domain has disappeared.  It's served by three name
servers:
;; AUTHORITY SECTION:
mm. 172800  IN  NS  NS-MM.RIPE.NET.
mm. 172800  IN  NS  NS.NET.mm.
mm. 172800  IN  NS  NS0.MPT.NET.mm.

;; ADDITIONAL SECTION:
NS.NET.mm.  172800  IN  A   202.153.125.17
NS0.MPT.NET.mm. 172800  IN  A   203.81.64.20
NS-MM.RIPE.NET. 172800  IN  A   193.0.12.151

ns0.mpt.net.mm is in Myanmar, part of the network of Myanma Post  
Telecommunication.  It's unreachable.


ns.net.mm is in address space registered to Powerbase DataCenter Services 
(HK) Ltd. in Hong Kong.  It's also unreachable, which makes it difficult 
to confirm whether its physical location matches its registered location. 
It may also be in Myanmar.


ns-mm.ripe.net is in Amsterdam.  It's reachable, but is responding to all 
queries with a SERVFAIL response.  Presumably, this means it hasn't been 
able to get updates from a master server for the .MM domain for long 
enough that its data has expired.


Looking at the rest of Myanmar's connectivity to the outside world, Myanma 
Post  Telecommunication has two IP address blocks registered to it: 
203.81.64.0/19 and 203.81.160.0/20.  Both of those blocks were in the 
global Internet routing table on September 27, but but have not been since 
September 28 (according to daily snapshots of route-views data).  It's 
pretty safe to say that Myanma Post  Telecommunication has completely 
turned off its connection to the outside world.  This is no doubt 
following the example set by the King of Nepal during the coup there a 
couple years ago.


The New York Times story says there are two ISPs in Myanmar.  Myanma Post 
 Telecommunication is the only one with IP addresses registered to a 
mailing address within the country, so I'm not sure who the other one is, 
or what its status is.


-Steve

RE: proposed NANOG charter amendments

[Steve Gibbard]

On Thu, 27 Sep 2007, Michael K. Smith - Adhost wrote:


Proposal 2:

Shall program committee members be permitted to skip rating
presentation proposals that do not fall into their areas of

expertise?


Wording:

Change the third paragraph of Section 8.3.2 as follows:

Old version:  Each member of the Program Committee must review all
presentations submitted for each meeting. The Chair may excuse a
member from one meeting's review cycle due to extenuating
circumstances, but if a member misses two meetings in a row, he or
she may be removed from the committee.

New version:  Each member of the Program Committee must review all
presentations submitted for each meeting and rate those
presentations which fall into their areas of expertise. The Chair
may excuse a member from one meeting's review cycle due to
extenuating circumstances, but if a member misses two meetings in a
row, he or she may be removed from the committee.



It seems like you're talking about two totally different things here,
punitive actions for missing meetings and requirements for PC Members to
audit presentations.  Unless it's really worth defining both separately
I would scratch the whole thing.  If you need to define punitive
actions, why not just say a simple majority may take action to remove a
Program Committee Member for cause.  The first sentence, taken alone,
could be listed as a portion of PC Members assigned duties (if such a
thing exists).


Since we're amending an existing document, this would be two separate 
changes.


The one being proposed here is supposed to reduce the requirements for 
Program Committee members (although, rereading it, I see that in the old 
version there wasn't actually a requirement for members to rate 
anything, as opposed to just reviewing things, so maybe it accidentally 
adds more requirements instead).  It does that by adding a few words to an 
existing paragraph.


The other thing being debated is the language about the procedure for 
removing Program Committee members.  That's in the same paragraph, because 
it's how we wrote it a few years ago, but it's not really related to this 
proposal.  If it needs to change, that should probably be done via a 
separate amendment.


I don't have strong feelings on the merits of either of the proposed 
amendments (and as long as I'm drafting charter changes at the request of 
an elected committee I'm not part of, any strong feelings I might develop 
should probably be kept out of this).  I can talk to some degree about the 
initial intent of the charter, with the caveat that there were lots of 
people involved and we may not have all intended the same thing.


My intent, which was fairly influential since I was the last one to do 
major editing on the document, was to to make sure the Steering Committee 
selection process was very clear, to give them some power, and beyond that 
to give them a pretty free hand.  They were going to be elected, while 
those of us working on the charter were self-appointed, so any decisions 
they made were going to have a lot more legitimacy than the decisions we 
were making.  Others had some very strong concerns about the operation of 
the Program Committee, so that got specified in probably more detail than 
I would have preferred.


So, in that vein, it seems to me that that the vagueness in the Program 
Committee member-removal text is ok.  The Steering Committee is in charge, 
and if there's a desire to remove a member of the Program Committee, the 
procedure ought to be whatever they say it is.


But, that said, we also intended the charter to be changeable if people 
felt the need to do so.  That's what the Amendments procedure is there 
for.


-Steve

Re: large organization nameservers sending icmp packets to dns servers.

[Steve Gibbard]

On Tue, 7 Aug 2007, Donald Stahl wrote:

It has nothing to do with judging how one runs their network or any other 
such nonsense. The RFC's say TCP 53 is fine. If you don't want to follow the 
rules, fine, but have the temerity to admit that it is stupid.


I don't want to wade into this particular argument, which doesn't seem to 
be going anywhere useful.  But I think the style of the argument causes 
some problems that trickle into network operations, and should be 
addressed.


The problem with this argument is that, while it may be entirely correct, 
it's unlikely to convince the people who matter.  The people who matter 
are the people who write the checks for the networks we work on.


Successful managers (and successful engineers) generally get pretty good 
at doing cost benefit analyses.  Since there are many decisions where 
there isn't one obvious answer, they learn instead to think in terms of 
each choice providing some benefits and having some costs, and doing the 
things where the benefits outweigh the costs.


In the firewall case, as Kevin said, there are probably people going to 
the decision makers and talking about the importance of keeping things 
closed up.  Every open firewall rule, they'll say, creates the potential 
for an attack.  Any attack could cause down time, unauthorized sharing of 
confidential data, loss of files people have spent the last several years 
working on, and more.  Therefore, the cost of an open firewall rule could 
potentially be millions of dollars.  The value of any service enabled by a 
hole in the firewall had better be more than that.


Is this argument valid?  Maybe not.  But the money people who make the 
decisions probably don't have the technical expertise to analyse it. 
Even if they suspect that the case for the policy is overstated, they'll 
associate some cost with ignoring the advice of their security people, as 
they probably should.


So, what's somebody who objects to such an argument to do?

You could go to management and say, the security people are wrong.  The 
standard says we must open more ports.  To not do so would be wrong. 
But you may not like the choice this presents management with.  On one 
side, they've got you telling them to follow an arbitrary standard, 
because not doing so would be wrong.  On the other side, they're being 
told that taking your advice could cost millions of dollars.  Losing 
millions of dollars as a result of a refusal to heed warnings would 
probably get them fired, or worse.  Pointing at an arbitrary standard 
after things had gone wrong probably wouldn't get them very far.


Alternatively, you too could start speaking their cost benefit language. 
You could assail the security peoples' cost figures, although at that 
point you'd be asking them to distrust other employees and they might 
wonder if they should distrust you instead.  Or you could point out the 
costs of leaving the port closed, or possible benefits of leaving it open. 
If you can tell them that some fraction of their customers aren't able to 
get to them because of the closed port, and that those would be customers 
represent some large amount of revenue, you'll show that there's actual 
benefit to having the port open.  If that benefit is greater than the 
potential loss they're being told about, you might actually win the 
argument.  If you have some evidence to back up your numbers, you may have 
more credibility, and be able to win the argument with lower numbers.


Or, you may find that you're not as right as you thought you were.  You 
may find that what you were advocating doesn't seem to have any concrete 
benefit, and that what the other side was saying has some merit.  That may 
not happen in this case, but sooner or later you'll probably find one 
where it does.


-Steve