Re: Update on RFC7871 - Client Subnet in DNS Support
+1 This will also work nicely with the effort to have a localized TR DNS selection. If Traffic Routers are selected the same way a traffic server is selected, by the client proximity from CZF or Geo, then using EDNS0 client-subnet will enable this choice to be more accurate based on real client subnet rather than DSN resolver. On Fri, Jun 2, 2017 at 10:46 PM, David Neuman <david.neuma...@gmail.com> wrote: > +1, excited to see this one come through. > > On Fri, Jun 2, 2017 at 12:15 PM, Eric Friedrich (efriedri) < > efrie...@cisco.com> wrote: > > > We are planning to add support for RFC7871 to Traffic Router. Here is a > > brief description of the feature. Comments appreciated! > > > > Background > > Clients do not make DNS requests directly to TR. Typically TR requests > > come from DNS resolvers within the infrastructure. Today, Cache Group > > selection for DNS Delivery Services is based on the IP address of the DNS > > resolver making the request to TR. RFC7871 includes the client subnet in > an > > EDNS0 option within the DNS query. When the ECS OPT is present (and the > > feature is enabled via a TR parameter), Traffic Router will use this IP > in > > place of the source IP of the DNS packet (the IP of the resolver). This > IP > > will be used in the CZF and maxmind lookups. > > > > Requirements > > > > 1. If DNS query includes ECS option in the Optional Record, Traffic > > Router will use the IP address included in the ECS option as the client > > address for Cache Group Selection > > 2. If there are multiple ECS options in the Optional Record, the one > > with the longest IP prefix - i.e. the ECS option with largest Source Net > > Mask will be used > > 3. If DNS query includes ECS Option, then DNS response from Traffic > > Router will also include the ECS Option. In the response the Scope Net > Mask > > is set to be same as the Source Net Mask. This is required by RFC 7871 > for > > DNS caching to work properly. > > 4. It is assumed that customers/operators will ensure that Source Net > > Mask for a subnet specified in the ECS is at greater than or equal to the > > net mask for the corresponding subnet entry in the CZF file. e.g. if ECS > > Option has 192.168.10.0/8, then 192.168.0.0/16 in CZF will work, but > > 192.168.10./28 will not work. > > 5. Add a TR parameter to disable use of ECS field even when present > > > > Design > > > > To support this feature new logic will be added to NameServer.query() > > function. The new logic will be responsible for parsing ECS option in the > > OptionalRecord of the incoming DNS Request, and retrieving the Client IP > > address and the associated Source Net Mask (Scope Net Mask must be 0 in > the > > incoming Request per RFC 7871). Please note that the underlying DNS > library > > xbill/dnsjava already has support for parsing the ECS Options. These > > functions from the library will be leveraged. > > > > 1. Message.getOPT().getOptions(EDNSOption.Code.CLIENT_SUBNET) will > > return list of ClientSubnetOption for the incoming Request (Message) > > 2. ClientSubnetOption has public methods to retrieve netmask and > Client > > IP address: getSourceNetmask(), getAddress() > > > > If ECS option is present, then the IP address retrieved from the ECS > > option, and will be passed as the Client IP address to the Traffic Router > > (through getZone call) for CZF/geo lookup > > > > If ECS option is present, then ClientSubnetOption will be created and > > included in the DNS response. In the Response the Scope Net Mask of the > > ClientSubnetOption is set as the Source Net Mask > > > > Testing > > We’ll test against all of the various CZF, National, Regional, VPN > > Blocking features and will do our best to check with DNSSEC > > > > —Eric > > > > > > > > > > > > > -- *Ori Finkelman*Qwilt | Work: +972-72-2221647 | Mobile: +972-52-3832189 | o...@qwilt.com
Re: Getting CZF data from BGP?
BIRD http://bird.network.cz is widely used as a BGP listener, though from a quick look at OpenBMP it does look like it is the way to go, depending on availability of BMP in existing networks. On Wed, May 31, 2017 at 10:27 PM, Ori Finkelman <o...@qwilt.com> wrote: > +1 > Wouldn't this require the BGP to run on each server ? that way the server > could learn the BGP distances from different routes. > As it may be the case that routes could be reached from different servers, > with different costs, it may also require a central entity to collect from > all server and decide which subnet should go to which server, and build the > CZF. > > > > On Wed, May 31, 2017 at 3:57 PM, Steve Malenfant <smalenf...@gmail.com> > wrote: > >> +1 on this. Although no experience with BGP. I've been looking into >> OpenBMP >> lately which seems to support BGP-LS. >> >> On Tue, May 30, 2017 at 1:48 PM, Jan van Doorn <j...@knutsel.com> wrote: >> >> > Definitely the first, the second maybe as a stretch goal. >> > >> > On Tue, May 30, 2017 at 11:23 AM Eric Friedrich (efriedri) < >> > efrie...@cisco.com> wrote: >> > >> > > Hey Jan- >> > > >> > > Are you looking to build a static CZF based off of BGP inputs? >> > > >> > > Are you looking for something that will listen to BGP and create a >> > > “real-time CZF” that responds to routing/CG changes? >> > > >> > > Or something else? >> > > >> > > > On May 30, 2017, at 1:00 PM, Jan van Doorn <j...@knutsel.com> wrote: >> > > > >> > > > Hi, >> > > > >> > > > We are considering building something to get the CZF data directly >> from >> > > the >> > > > network, possibly using BGP and open source routing software. >> Before we >> > > get >> > > > going, wanted to ask if anyone has any experience with this, or >> tools >> > > > already built to do this? >> > > > >> > > > Rgds, >> > > > JvD >> > > >> > > >> > >> > > > > -- > > *Ori Finkelman*Qwilt | Work: +972-72-2221647 <072-222-1647> | Mobile: > +972-52-3832189 <052-383-2189> | o...@qwilt.com > -- *Ori Finkelman*Qwilt | Work: +972-72-2221647 | Mobile: +972-52-3832189 | o...@qwilt.com
Re: Getting CZF data from BGP?
+1 Wouldn't this require the BGP to run on each server ? that way the server could learn the BGP distances from different routes. As it may be the case that routes could be reached from different servers, with different costs, it may also require a central entity to collect from all server and decide which subnet should go to which server, and build the CZF. On Wed, May 31, 2017 at 3:57 PM, Steve Malenfant <smalenf...@gmail.com> wrote: > +1 on this. Although no experience with BGP. I've been looking into OpenBMP > lately which seems to support BGP-LS. > > On Tue, May 30, 2017 at 1:48 PM, Jan van Doorn <j...@knutsel.com> wrote: > > > Definitely the first, the second maybe as a stretch goal. > > > > On Tue, May 30, 2017 at 11:23 AM Eric Friedrich (efriedri) < > > efrie...@cisco.com> wrote: > > > > > Hey Jan- > > > > > > Are you looking to build a static CZF based off of BGP inputs? > > > > > > Are you looking for something that will listen to BGP and create a > > > “real-time CZF” that responds to routing/CG changes? > > > > > > Or something else? > > > > > > > On May 30, 2017, at 1:00 PM, Jan van Doorn <j...@knutsel.com> wrote: > > > > > > > > Hi, > > > > > > > > We are considering building something to get the CZF data directly > from > > > the > > > > network, possibly using BGP and open source routing software. Before > we > > > get > > > > going, wanted to ask if anyone has any experience with this, or tools > > > > already built to do this? > > > > > > > > Rgds, > > > > JvD > > > > > > > > > -- *Ori Finkelman*Qwilt | Work: +972-72-2221647 | Mobile: +972-52-3832189 | o...@qwilt.com
CCR A record TTL parameter changes is not reflected in CRConfig.json
Hi,
I am using TC 1.7.
I have an HTTP delivery service and would like to change the TTL of the A
records traffic router is responding with,
When changing the parameters tld.ttls.A or tld.ttls. it has no effect
on the TTLs passed in the CRConfig.json. It always stays 3600, which is
different than the behavior for the tld.ttls.NS which is properly reflected.
For example, setting
tld.ttls.A 60
tld.ttls. 60
tld.ttls.NS40
Creates the following CRConfig.json
"my-service": {
"protocol": {
"redirectToHttps": "false",
"acceptHttps": "false"
},
"domains": ["my-service.my-cdn.my-domain.com"],
"missLocation": {
"long": "-87.627778",
"lat": "41.881944"
},
"ttls": {
"": "3600",
"SOA": "86400",
"A": "3600",
"NS": "40"
},
Is there a different configuration I am missing, or it's a known issue ?
Thanks,
Ori
--
*Ori Finkelman*Qwilt | Work: +972-72-2221647 | Mobile: +972-52-3832189 |
o...@qwilt.com
Delivery Services compliance to SVA Open Caching model (TC-55)
Dear All, *Executive summary:* we would like to promote PR 108 <https://github.com/apache/incubator-trafficcontrol/pull/108> (or similar functionality) described in [image: Improvement] TC-55 Multi Delivery Services With Same Domain Name and Different Path Prefixes <https://issues.apache.org/jira/browse/TC-55>, to be pulled back to master, after review and required adaptations. >From here on it's the use case description justifying this PR. As presented at the previous meetup, we would like to go forward making TC compatible with the Streaming Video Alliance <https://www.streamingvideoalliance.org/> Open Caching <https://www.streamingvideoalliance.org/technical-work/working-groups/open-caching/> architecture. In very high level, open caching means that another CDN (the typical use case is a commercial CDN) would be able to delegate traffic to open caches at the ISP premises. The full specs for those who are interested can be found on the SVA website, under Open Caching <https://www.streamingvideoalliance.org/technical-work/working-group-output/> . One of the main gaps are the differences in host name and URL structure. In open caching there is no interdependence of host names between the delegating CDN and the surrogate open cache system. In other words, the hostname of the ISP request router and caches is not indicative for the origin service, and all the information needed for service matching is in the URL path. *An example:* CDN *foo *is redirecting traffic of it's content provider customer example.com to ISP *bar*. Client manifest request from *foo *CDN: GET /vod/sports/final-index.m3u8 Host: foo.example.com (note that typically though the CDN is handling the traffic, the domain is of the content provider, example.com in this case). Foo CDN redirect response: HTTP/1.1 302 Found location http://tr.bar.com/*foo.example.com <http://foo.example.com/>*/vod/sports/final-index.m3u8 (note that the hostname is not indicative, the delegating host name is in the first path segment). Client request to traffic router: GET /*foo.example.com <http://foo.example.com/>*/vod/sports/f inal-index.m3u8 Host: tr.bar.com Traffic Router redirect response: HTTP/1.1 302 Found location http://server1.bar.com/ <http://server.bar.com/>*foo.example.com <http://foo.example.com/>*/vod/sports/final-index.m3u8 >From here the client requests the manifest and then the media files directly from the cache server (server1). *What is the reason for this approach ?* Think of it not in Traffic Control context but rather as a general use case of delegating HTTP traffic between different systems. Note that the delivery services are define by the CDN (foo) and are pushed to the ISP caching systems, which means the CDN needs to have the flexibility to define the DS to their own needs. Decoupling the ISP host names (traffic router and cache server) from the delegating host name has the following benefits: 1. No need for DNS changes for each new delivery service. This is even more important when using DNSSEC. 2. No need for certificate updates for each new delivery service. If the ISP hostnames are fixed, or at least fixed per partner CDN, then there is only one cert (per partner CDN). 3. Allows more flexibility when integrating with 3rd party commercial CDNs. *What is missing ?* The main gap is allowing a delivery service to match the service against path regexp, or a combination of host regexp and path regexp. This means that different DSs may have the same host regexp but with different path regexp. This was already suggested several times in the past and specifically in TC- 55 <https://issues.apache.org/jira/browse/TC-55>. We propose to re-visit TC- <https://issues.apache.org/jira/browse/TC-55>55 <https://issues.apache.org/jira/browse/TC-55> ,make any required changes in the original proposal and actual implementation in PR 108. We would highly appreciate any inputs and comments. Many thanks, Ori -- *Ori Finkelman*Qwilt | Work: +972-72-2221647 <072-222-1647> | Mobile: +972-52-3832189 <052-383-2189> | o...@qwilt.com
Re: Backup Cache Group Selection
Hi again, I understand now that the "backupList" feature does not exist yet. What is the status of this feature ? is it planned ? Thanks, Ori On Mon, May 8, 2017 at 4:18 PM, Ori Finkelman <o...@qwilt.com> wrote: > Hi, > Following up on this one, it seems that both czf attributes described in > this thread, the "coordinates" and the "backupList" are not documented in > the official docs in > http://trafficcontrol.incubator.apache.org/docs/latest/admin/traffic_ops_ > using.html#the-coverage-zone-file-and-asn-table > > Is there a plan to update the documentation ? should I open a JIRA for it ? > > Thanks, > Ori > > On Thu, Mar 30, 2017 at 8:45 PM, Jeff Elsloo <jeff.els...@gmail.com> > wrote: > >> Yes, that's correct. >> -- >> Thanks, >> Jeff >> >> >> On Thu, Mar 30, 2017 at 11:20 AM, Eric Friedrich (efriedri) >> <efrie...@cisco.com> wrote: >> > Thanks Jeff- >> > Could I think of it as the following? Echoing back to be sure I >> understand... >> > >> > If there is a lat/long for a cache group in the CZF file, any client >> hit to that CG should use the CZF lat/long as it client’s lat/long instead >> of using geolocation. >> > >> > For the purposes of finding closest cache group, the client’s location >> (from CZF as above or from Geolocation provider) will be compared against >> the location of the cache’s as configuration in Traffic Op’s CG record? >> > >> > —Eric >> > >> > >> >> On Mar 30, 2017, at 1:07 PM, Jeff Elsloo <jeff.els...@gmail.com> >> wrote: >> >> >> >> It could now be considered the "average" of the location of the >> >> clients within that section of the CZF, however, it should be noted >> >> that the addition of the geo coordinates to the CZF is relatively new. >> >> Previously we never had the ability to specify lat/long on those >> >> cachegroups, and we solely relied on those specified in edgeLocations, >> >> meaning that the matches had to be 1:1. Adding the coordinates allowed >> >> us to cover edge cases and miss scenarios and stick to the CZF >> >> whenever possible. Previously when we had no coordinates, and we had a >> >> hit in the CZF but not corresponding hit within the edgeLocations >> >> (health, assignments, etc), we would fall back to the Geolocation >> >> provider. >> >> -- >> >> Thanks, >> >> Jeff >> >> >> >> >> >> On Thu, Mar 30, 2017 at 5:29 AM, John Shen (weifensh) >> >> <weife...@cisco.com> wrote: >> >>> Thanks Jeff and Oren for the discussion. I agree now that lat/long >> from CZF is the “average” location of clients, and lat/long from Ops is the >> location of a certain Cache Group. So it appears to be reasonable to use >> them as source and dest to calculate the distance. >> >>> >> >>> Thanks, >> >>> John >> >>> >> >>> >> >>> On 30/03/2017, 6:55 PM, "Oren Shemesh" <or...@qwilt.com> wrote: >> >>> >> >>>Jeff, having read this conversation more than once, I believe >> there is a >> >>>misunderstanding regarding the ability to provide coordinates for >> cache >> >>>groups both in the CZF and in the TO DB. >> >>> >> >>>Here is what I believe is a description which may help >> understanding the >> >>>current behaviour: >> >>> >> >>>The coordinates specified in the CZF for a cache group are not >> supposed to >> >>>be the exactly same as the coordinates in the TO DB for the same >> cache >> >>>group. >> >>>This is because they do not represent the location of the caches >> of the >> >>>group. >> >>>They represent the (average) location of clients found in the >> subnets >> >>>specified for this cache group. >> >>> >> >>>This, I believe, explains both the behaviour of the code (Why the >> >>>coordinates from the CZF are used for the source, but the >> coordinates from >> >>>the TO DB are used for the various candidate cache groups), and >> the fact >> >>>that there is a 'duplication'. >> >>> >> >>>Is this description true ? >> >>> >> >>> >&
Re: Backup Cache Group Selection
;>>>>>> TR will send a client request to the cache group that is > >>>>>> closest to > >>>>>>>>> the > >>>>>>>>>>>>>> original mapped group. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Example CZF w/ cache location > >>>>>>>>>>>>>> - > >>>>&
