Re: Looking for geo-directional DNS service

[Steve Gibbard]

On Tue, 15 Jan 2008, Patrick W.Gilmore wrote:


On Jan 15, 2008, at 12:00 PM, Bill Woodcock wrote:
[...]

If you're doing things on the Internet, instead of the physical world,
topological distance is presumably of much greater interest than whatever
geographic proximity may coincidentally obtain.


Unless you define "topologically nearest" as "what BGP picks", that is 
incorrect.  And even if you do define topology to be equivalent to BGP, that 
is not what is of the greatest interest.  "Goodput" (latency, packet loss, 
throughput) is far more important.  IMHO.


If you don't like my example, then ignore Ashburn and take a random, 
medium-sized network.  Now assume an anycast node which is topologically 
(i.e. latency, bit-miles, throughput, whatever your definition) closer 
through transit, compared to a node topologically farther away through 
peering.  Which is chosen?  And this is not even close to an unusual 
situation.


This in no way means anycast sux.  It just means anycast is not, by a long 
shot, guaranteed to give you the "closest" node by any reasonable definition. 
(Sorry, I don't think "node BGP picks" is "reasonable".  You are welcome to 
disagree, but the point still stands that other definitions of "reasonable" 
are not satisfied.)


There are many different ways to set up Internet topology.  Some of these 
achieve geographic proximity, and some don't.  Network topology that 
doesn't match geographic proximity (common in Southern Africa, South 
America, and to a degree in the central US) leads to some unavoidable 
performance issues (speed of light, constraints on long distance 
capacity, etc.).  A distribution system following topology in such an 
environment won't do nearly as well as one that follows topology in a 
better interconnected area, but following topology should still produce 
better performance than not doing so.  If traffic from ISP A to ISP B in 
Region 1 goes through Region 2, ISP B will be served better by content in 
Region 2 than by content in ISP A.  So, following topology is good.


There are many different ways to set up an anycast system, and how a 
system is set up has a lot of influence on what node BGP on the networks 
that connect to it are going to pick.  If somebody setting up an anycast 
system plugs a bunch of nodes into random networks scattered around the 
world, they're not going to do very well on geographic or topological 
proximity.  Chances are, they'll end up with situations like the K Root in 
India that was at one point getting most of its traffic from North 
America.  But if an anycast system is set up with the right transit and 
peering policies, it can do a decent job of matching topology.


I went into this in a lot more detail in the paper at: 
http://www.pch.net/resources/papers.php?dir=/anycast-performance


Will a well-designed anycast system do as well as Akamai?  Probably not. 
Akamai does actual testing of paths rather than using theory to decide 
what the paths will probably look like, which should give them a much 
better view of places where reality doesn't match theory.  They've also 
got a lot more locations than anybody else doing this, which means they 
should typically be able to get much closer to where the content needs to 
go.  But Akamai has lots of patents and lots of proprietary software 
making their decisions about where to source things from.  They charge 
their customers quite a bit for this service, and the cost savings their 
technology and wide footprint should produce go to the receiving networks 
who don't have to carry the traffic very far, rather than to the content 
provider who would hot potato the traffic off at the closest possible 
point anyway.  So, the decision for somebody deciding whether to use 
Akamai, use one of its less advanced competitors, or make their own, may 
come down to whether they can produce something good enough, rather than 
whether they can produce something as good or better.


-Steve

Re: Looking for geo-directional DNS service

[Joe Greco]

> [EMAIL PROTECTED] (Joe Greco) writes:
> > ...
> > So, anyways, would it be entertaining to discuss the relative merits of
> > various DNS implementations that attempt to provide geographic answers 
> > to requests, versus doing it at a higher level?  (I can hear everyone 
> > groaning now, and some purist somewhere probably having fits)
> 
> off topic.  see .

Possibly, but I found myself removed from that particular party, and the
request was on NANOG, not on dns-operations.  I was under the impression 
that dns-operations was for discussion of DNS operations, not 
implementation choices.  Whether NANOG is completely appropriate remains 
to be seen; I haven't heard a ML complaint though.  There would ideally 
be a list for implementation and design of such things, but I've yet to 
see one that's actually useful, which is, I suspect, why NANOG got a 
request like this.

Besides, if you refer back to the original message in this thread, where I
was driving would be much closer to being related to what the OP was 
interested in.

Hank was saying:

> What I am looking for is a commercial DNS service.
> [...]
> Another service I know about is the Ultradns (now Neustar) Directional DNS:
> http://www.neustarultraservices.biz/solutions/directionaldns.html
> But this service is based on statically defined IP responses at each of
> their 14 sites so there is no proximity checking done.

So there are three basic ways to go about it,

1) Totally static data (in which case anycast and directionality are not a
   consideration, at least at the DNS level), which does not preclude doing
   things at a higher level.

2) Simple anycast, as in the Directional DNS service Hank mentioned, which
   has thoroughly been thrashed into the ground as to why it ain't great,
   which it seems Hank already understood.

3) Complex DNS implementations.  Such as ones that will actually do active
   probes, etc.  Possibly combined with 1) even.

I was trying to redirect the dead anycast horse beating back towards a 
discussion of the relative merits of 1) vs 3).  The largest problems with 
3) seem to revolve around the fact that you generally have no idea where 
a request /actually/ originated, and you're pinning your hopes on the 
client's resolver having some vague proximity to the actual client. 
Redirection at a higher level is going to be desirable, but is not always 
possible, such as for protocols like NNTP.

I'm happy to be criticized for guiding a conversation back towards being
relevant...  :-)

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.

Re: Looking for geo-directional DNS service

[Paul Vixie]

[EMAIL PROTECTED] (Joe Greco) writes:
> ...
> So, anyways, would it be entertaining to discuss the relative merits of
> various DNS implementations that attempt to provide geographic answers 
> to requests, versus doing it at a higher level?  (I can hear everyone 
> groaning now, and some purist somewhere probably having fits)

off topic.  see .
-- 
Paul Vixie

Re: Looking for geo-directional DNS service

[Joe Greco]

> Unless you define "topologically nearest" as "what BGP picks", that is  
> incorrect.  And even if you do define topology to be equivalent to  
> BGP, that is not what is of the greatest interest.   
> "Goodput" (latency, packet loss, throughput) is far more important.   
> IMHO.

Certainly, but given some completely random transaction, there's still
going to be a tendency for anycast to be some sort of improvement over
pure random chance.  1000 boneheaded anycast implementations cannot be
wrong.  :-)  That you don't get it right every time doesn't make it
wrong every time.

I'm certainly not arguing for anycast-only solutions, and said so.  I'll
be happy to consider it as a first approximation to getting something to
a topologically "nearby" network, though as I also said, there needs to
be some care taken in the implementation.

Anycast can actually be very powerful within a single AS, where of course
you have some knowledge of the network and predictability.  You lose some
(probably a lot) of that in the translation to the public Internet, but
I'm going to go out on a bit of a limb and guess that if I were to stick an
anycast node in Chicago, Sydney, and Amsterdam, I'm very likely to be able
to pick my networks such that I get a good amount of localization.

Of course, nobody's perfect, and it probably needs to be a data-driven 
business if you really want well-optimized redirection.  However, that's
a bit of magic.  Even the fabled Akamai used to direct us to some ISP up
in Minnesota...  (BFG)

So, anyways, would it be entertaining to discuss the relative merits of
various DNS implementations that attempt to provide geographic answers 
to requests, versus doing it at a higher level?  (I can hear everyone 
groaning now, and some purist somewhere probably having fits)

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.

Re: Looking for geo-directional DNS service

[Paul Vixie]

[EMAIL PROTECTED] ("Patrick W.Gilmore")]
> And even if you do define topology to be equivalent to BGP, that is not
> what is of the greatest interest.  "Goodput" (latency, packet loss,
> throughput) is far more important.  IMHO.

in my less humble justified true belief, this is 100% truth.

> This in no way means anycast sux.  It just means anycast is not, by a
> long shot, guaranteed to give you the "closest" node by any reasonable
> definition.  (Sorry, I don't think "node BGP picks" is "reasonable".  ...

i also second this notion.

in our (ISC's) current use of anycase (for f-root and other dns servers),
anycast is a crutch for not having a global backbone, but wanting f-root to
have global representation and extreme replication.  informal studies don't
show as much locality as we'd like -- but by peering aggressively everywhere
and by setting no-export on our route almost everywhere, we've been able to
localize and isolate ddos effects, which is all we were trying to accomplish.

but note, f-root is a normal dns server, it has an absolute mapping between
 and .  i don't believe in stupid dns tricks
(where that mapping is relativized for TE purposes), and one of the reasons
for my disbelief is that many ISP's in f-root's ~40 IXP locations do not
peer with us, and their traffic is therefore answered in remote (to them)
places where TE can't be predicted.  in other words, people doing "stupid dns
tricks" are probably counting on anycast to do something f-root doesn't care
about (and which i think BGP won't do even on its best day.)
-- 
Paul Vixie

Re: Looking for geo-directional DNS service

[Patrick W.Gilmore]

On Jan 15, 2008, at 3:03 PM, Joe Greco wrote:


Except Hank is asking for true topological distance (latency /
throughput / packetloss).

Anycast gives you BGP distance, not topological distance.

Say I'm in Ashburn and peer directly with someone in Korea where he
has a node (1 AS hop), but I get to his node in Ashburn through my
transit provider (2 AS hops), guess which node anycast will pick?


Ashburn and other major network meet points are oddities in a very  
complex
network.  It would be fair to note that anycast is likely to be  
reasonably
effective if deployed in a manner that was mindful of the overall  
Internet

architecture, and made allowances for such things.


You are not disagreeing with me.  I was responding to Woody who said:

On Jan 15, 2008, at 12:00 PM, Bill Woodcock wrote:

Yes, and that's how anycast works: it directs traffic to the
_topologically nearest_ server.

[...]

If you're doing things on the Internet, instead of the physical world,
topological distance is presumably of much greater interest than  
whatever

geographic proximity may coincidentally obtain.


Unless you define "topologically nearest" as "what BGP picks", that is  
incorrect.  And even if you do define topology to be equivalent to  
BGP, that is not what is of the greatest interest.   
"Goodput" (latency, packet loss, throughput) is far more important.   
IMHO.


If you don't like my example, then ignore Ashburn and take a random,  
medium-sized network.  Now assume an anycast node which is  
topologically (i.e. latency, bit-miles, throughput, whatever your  
definition) closer through transit, compared to a node topologically  
farther away through peering.  Which is chosen?  And this is not even  
close to an unusual situation.


This in no way means anycast sux.  It just means anycast is not, by a  
long shot, guaranteed to give you the "closest" node by any reasonable  
definition.  (Sorry, I don't think "node BGP picks" is "reasonable".   
You are welcome to disagree, but the point still stands that other  
definitions of "reasonable" are not satisfied.)



In general, anycast is better than not-anycast, and can be optimized  
to be better than non-anycast for nearly all user by someone with  
enough clue + money + time.  This is not in question.  It is  
essentially impossible to guarantee anycast is better than any other  
solution for all applications and all end users, especially over time  
as the Internet changes.  This is not in question either.


--
TTFN,
patrick


Anycast by itself probably isn't entirely desirable in any case, and  
could
ideally be paired up with other technologies to "fix" problems like  
this.


I haven't seen many easy ways to roll-your-own geo-DNS service.  The  
ones
I've done in the past simply built in knowledge of the networks in  
question,
and where such information wasn't available, took "best guess" and  
then may
have done a little research after the fact for future queries.  This  
isn't

as comprehensive as doing actual latency / throughput / pl checking.

... JG
--
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance  
[and] then I
won't contact you again." - Direct Marketing Ass'n position on e- 
mail spam(CNN)
With 24 million small businesses in the US alone, that's way too  
many apples.

Re: Looking for geo-directional DNS service

[Joe Abley]

On 15-Jan-2008, at 12:50, Patrick W. Gilmore wrote:


Anycast gives you BGP distance, not topological distance.


Yeah, it's topology modulated by economics :-)


Joe

Re: Looking for geo-directional DNS service

[Joe Greco]

> Except Hank is asking for true topological distance (latency /  
> throughput / packetloss).
> 
> Anycast gives you BGP distance, not topological distance.
> 
> Say I'm in Ashburn and peer directly with someone in Korea where he  
> has a node (1 AS hop), but I get to his node in Ashburn through my  
> transit provider (2 AS hops), guess which node anycast will pick?

Ashburn and other major network meet points are oddities in a very complex
network.  It would be fair to note that anycast is likely to be reasonably
effective if deployed in a manner that was mindful of the overall Internet
architecture, and made allowances for such things.

Anycast by itself probably isn't entirely desirable in any case, and could
ideally be paired up with other technologies to "fix" problems like this.

I haven't seen many easy ways to roll-your-own geo-DNS service.  The ones
I've done in the past simply built in knowledge of the networks in question,
and where such information wasn't available, took "best guess" and then may
have done a little research after the fact for future queries.  This isn't
as comprehensive as doing actual latency / throughput / pl checking.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.

Re: Looking for geo-directional DNS service

[Patrick W. Gilmore]

On Jan 15, 2008, at 12:00 PM, Bill Woodcock wrote:

 On Tue, 15 Jan 2008, Hank Nussbacher wrote:

The Ultradns (now Neustar) Directional DNS service is based on
statically defined IP responses at each of their 14 sites so there
is no proximity checking done.


Yes, and that's how anycast works: it directs traffic to the
_topologically nearest_ server.  So as long as there's a DNS server
topologically near your data server, your users will get the  
topologically

nearest of your servers.  Which is why so many content folks _do_ roll
their own: to ensure fate-sharing between the DNS traffic which
effectively selects the data server, and the eventual data traffic.

If you're doing things on the Internet, instead of the physical world,
topological distance is presumably of much greater interest than  
whatever

geographic proximity may coincidentally obtain.


Except Hank is asking for true topological distance (latency /  
throughput / packetloss).


Anycast gives you BGP distance, not topological distance.

Say I'm in Ashburn and peer directly with someone in Korea where he  
has a node (1 AS hop), but I get to his node in Ashburn through my  
transit provider (2 AS hops), guess which node anycast will pick?


--
TTFN,
patrick

Re: Looking for geo-directional DNS service

[Bill Woodcock]

  On Tue, 15 Jan 2008, Hank Nussbacher wrote:
> The Ultradns (now Neustar) Directional DNS service is based on 
> statically defined IP responses at each of their 14 sites so there 
> is no proximity checking done.

Yes, and that's how anycast works: it directs traffic to the 
_topologically nearest_ server.  So as long as there's a DNS server 
topologically near your data server, your users will get the topologically 
nearest of your servers.  Which is why so many content folks _do_ roll 
their own: to ensure fate-sharing between the DNS traffic which 
effectively selects the data server, and the eventual data traffic.

If you're doing things on the Internet, instead of the physical world, 
topological distance is presumably of much greater interest than whatever 
geographic proximity may coincidentally obtain.

-Bill

Re: Looking for geo-directional DNS service

[Hank Nussbacher]

At 12:14 AM 16-01-08 +1300, jamie baddeley wrote:

Yes, but that would require them to run a DNS server at each of their 4 
locations.  They do not want to run their own DNS.  They want it outsourced.


Thanks,
-Hank


Thought about anycasting? Broad as a barn door, but if you add health
checking of the services and  integrate that into what your DC router
announces you get closer to what you want.

jamie

On Tue, 2008-01-15 at 12:55 +0200, Hank Nussbacher wrote:
> I am looking for a commercial DNS service that provides
> geo-directionality.  Suppose I have 4 data centers scattered thruout the
> world and want users to hit the closest data center based on proximity
> checks (pings, TTLs, latency, load, etc.).  I know one can "roll their
> own", using various geo-locational data from companies like Maxmind.  I am
> *not* interested in that.  I am *not* interested in applicances like the
> Cisco ACE GSS 4400 either (that do this as well):
> http://www.cisco.com/en/US/products/hw/contnetw/ps4162/
>
> What I am looking for is a commercial DNS service.
>
> Is the Akamai Edgescape service the closest to what I want:
> http://www.akamai.com/html/technology/products/edgescape.html
> Is anyone using it?  Can you recommend it?
>
> Another service I know about is the Ultradns (now Neustar) Directional DNS:
> http://www.neustarultraservices.biz/solutions/directionaldns.html
> But this service is based on statically defined IP responses at each of
> their 14 sites so there is no proximity checking done.
>
> Thanks,
> Hank
>

Looking for geo-directional DNS service

[Hank Nussbacher]

I am looking for a commercial DNS service that provides 
geo-directionality.  Suppose I have 4 data centers scattered thruout the 
world and want users to hit the closest data center based on proximity 
checks (pings, TTLs, latency, load, etc.).  I know one can "roll their 
own", using various geo-locational data from companies like Maxmind.  I am 
*not* interested in that.  I am *not* interested in applicances like the 
Cisco ACE GSS 4400 either (that do this as well):

http://www.cisco.com/en/US/products/hw/contnetw/ps4162/

What I am looking for is a commercial DNS service.

Is the Akamai Edgescape service the closest to what I want:
http://www.akamai.com/html/technology/products/edgescape.html
Is anyone using it?  Can you recommend it?

Another service I know about is the Ultradns (now Neustar) Directional DNS:
http://www.neustarultraservices.biz/solutions/directionaldns.html
But this service is based on statically defined IP responses at each of 
their 14 sites so there is no proximity checking done.


Thanks,
Hank