Re: AWS S3 DNS load balancer

2021-06-16 Thread Andras Toth
AWS or S3 is not the only service where you will see a single IP returned for a DNS query, www.microsoft.com and www.apple.com (via Akamai) do the same - see further below. When you look up .s3.amazonaws.com you get back an answer that directs you to the correct region where the S3 bucket is locat

Re: AWS S3 DNS load balancer

2021-06-15 Thread Christopher Morrow
On Tue, Jun 15, 2021 at 10:33 AM Christopher Morrow wrote: > > On Tue, Jun 15, 2021 at 8:07 AM Karl Auer wrote: > >> On Tue, 2021-06-15 at 11:37 +, Deepak Jain wrote: >> > (I’m talking specifically about S3 not Route5x or whatever the DNS >> > product is). >> >> Route53. >> >> Not sure what

Re: AWS S3 DNS load balancer

2021-06-15 Thread Dan Halperin via NANOG
Hi Deepak. Amazon documents the IPs for their public and private cloud services: https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html (I know this because Batfish uses these in its reachability analysis, for example, "Make sure all outgoing flows to S3 are permitted by the firewall".)

RE: AWS S3 DNS load balancer

2021-06-15 Thread Deepak Jain
You can't use DNS to get "all" service IP's of a service like S3 or a CDN for traffic engineering purposes. That will not work, ever (for services of such scale). The hackery is assuming you can build a list of service IP's by querying DNS. > There are a lot of reasons why someone may want th

RE: AWS S3 DNS load balancer

2021-06-15 Thread Deepak Jain
I've just taken a squiz at an S3-based website we have, and via the S3 URL it is a CNAME with a 60-secod TTL pointing at a set of A records with 5-second TTLs. Any one dig returns the CNAME and a single IP address: dig our-domain.s3-website-ap-southeast-2.amazonaws.com. our-domain.s3-website

Re: AWS S3 DNS load balancer

2021-06-15 Thread Lukas Tribus
Hello, > AWS is doing Geo-based load balancing and spitting things out, > and networks with eyeballs are doing their own things for traffic > management and trying to do shortest paths to things – and responsible > operators want to minimize the non-desirable and non-deterministic > behaviors. Y

Re: AWS S3 DNS load balancer

2021-06-15 Thread Karl Auer
On Tue, 2021-06-15 at 10:33 -0400, Christopher Morrow wrote: > Maybe Deepak means: > "When I ask for an S3 endpoint I get 1 answer, which is 1 of a set > of N. > Why would >the 'loadbalancer' send me all N?" I've just taken a squiz at an S3-based website we have, and via the S3 URL it is a C

Re: AWS S3 DNS load balancer

2021-06-15 Thread Lukas Tribus
Hello, On Tue, 15 Jun 2021 at 13:37, Deepak Jain wrote: > Is this a “normal” or expected solution or just some local hackery? It's absolutely normal and expected for a huge service like this to keep round robin at the DNS server side. YMMV with client side DNS based round robin (Amazon needs to

RE: AWS S3 DNS load balancer

2021-06-15 Thread Deepak Jain
Maybe Deepak means: "When I ask for an S3 endpoint I get 1 answer, which is 1 of a set of N. Why would the 'loadbalancer' send me all N?" (I don't know a aws s3 url to test this out with, an example from Deepak would be handy) Regards, K. -- ~~

Re: AWS S3 DNS load balancer

2021-06-15 Thread Christopher Morrow
On Tue, Jun 15, 2021 at 8:07 AM Karl Auer wrote: > On Tue, 2021-06-15 at 11:37 +, Deepak Jain wrote: > > (I’m talking specifically about S3 not Route5x or whatever the DNS > > product is). > > Route53. > > Not sure what you mean by "S3 DNS". I wasn't aware S3 had any DNS > functionality at al

Re: AWS S3 DNS load balancer

2021-06-15 Thread nanog
The IP addresses for S3 do not change very often, and are region specific (as you would expect). You are correct that this can cause problems for clients that never re-resolve (eg Java networkaddress.cache.ttl=-1) You may be interested in the (periodically updated) list of AWS IP ranges by usi

Re: AWS S3 DNS load balancer

2021-06-15 Thread Karl Auer
On Tue, 2021-06-15 at 11:37 +, Deepak Jain wrote: > (I’m talking specifically about S3 not Route5x or whatever the DNS > product is). Route53. Not sure what you mean by "S3 DNS". I wasn't aware S3 had any DNS functionality at all... on the other hand, there is much indeed that I do not know.

AWS S3 DNS load balancer

2021-06-15 Thread Deepak Jain
They seem to do something a little unusual where every DNS request provides a different IP out of a small pool with those IPs not changing very frequently. (I’m talking specifically about S3 not Route5x or whatever the DNS product is). Basically like round robin, but instead of providing all of