I think adding new DS types for this makes sense because traditionally the DS type determines the value of go_direct as well as how content is cached (disk/ram/not cached at all). If we make the field directly configurable on the Delivery Service, then we now have the complexity of prohibiting certain go_direct values in certain types of delivery services, and that adds to the cognitive load required to create/edit a delivery service.
For example, it's a bad experience for the user to find out through an API error that they are prohibited from setting go_direct to certain values in certain scenarios. If we just create new types for that instead, the user doesn't have to worry about conflicting settings in specific DS types and rather just has to choose the DS type that they want (where the proper/best settings are chosen for them under the hood). Basically we'd need 4 new types (naming could be different): HTTP_LIVE_NATNL_BYPASS DNS_LIVE_NATNL_BYPASS HTTP_BYPASS DNS_BYPASS The new types would mimic the matching original types except for setting go_direct=true, which would allow the edges to fetch from the origin when all the mids are down. - Rawlin On Wed, Jun 20, 2018 at 6:33 AM, Vijay Anand <[email protected]> wrote: > All, > > The PR given below is a perl implemention for making parent.config's > go_direct directive configurable via Delivery service > https://github.com/apache/trafficcontrol/pull/2407 > > This PR has been hosted to discuss about various approaches to make > go_direct configurable. GoLang implementation will be added once we > finalize on the approach. > > Background: > --------------- > Right now it is hard coded as false in parent.config for DS of types other > than HTTP_LIVE, HTTP_NO_CACHE, DNS_LIVE and hence for such delivery > services, if there occurs a problem in the network / in the Mids and they > become unreachable / offline, all the requests fail because of this hard > coded GO_DIRECT setting. > > By making this configurable, we are giving a choice to the operators to > fetch directly from origin under such scenarios. > > > Question: > ------------ > Originally it was thought of as a new column in Deliveryservice table > (Go_Direct). But then Rawlin suggested adding a new delivery serice type to > avoid some of the conflict scenario disccussed in the PR. Eric and Rob > feels that adding new DS type is not a desired one. > > Request your views to get a consensus on the suitable approach for this. > > Thanks, > Vijayanand S > > > > ---------- Forwarded message ---------- > From: Vijay Anand <[email protected]> > Date: Thu, Jun 7, 2018 at 6:45 PM > Subject: Re: Making parent.config's go_direct directive configurable via > Delivery service > To: [email protected] > > > Rawlin, > > Yes, I am using a version which Eric referred to (i.e) Cisco's version. And > looks like in this code it is actually possible to create MSO groups > (origin server) which may not contain the org_serv_fqdn. So do you think, > MSO enabling and Go Direct = True as mutually exclusive will work? > > Thanks, > Vijayanand S > > On Mon, Jun 4, 2018 at 8:13 PM, Rawlin Peters <[email protected]> > wrote: > >> Also, it's very possible this might be a nuance between the version of >> TO you are running and the vanilla open source TO. We don't assign >> delivery services to "groups"; we assign them to individual servers >> (although that might mean every single server in a cachegroup). I >> think I recall Eric mentioning "server groups" at Cisco. Might that be >> the case? >> >> - Rawlin >> >> On Mon, Jun 4, 2018 at 8:36 AM, Rawlin Peters <[email protected]> >> wrote: >> > It might be possible to create an MSO delivery service with an >> > `orgServerFqdn` that doesn't match the [hostname + domain name] of a >> > server it's assigned to, but I'm not sure it would work as expected. >> > In a bug I recently fixed, I found a bit of code [1] that makes me >> > think that scenario would lead to a bad parent.config. The issue [2] >> > was that having a trailing slash in `orgServerFqdn` led to an empty >> > parent field in parent.config (because before the fix, TO didn't find >> > an origin Server that matched the `orgServerFqdn` of the MSO delivery >> > service. >> > >> > - Rawlin >> > >> > [1] https://github.com/apache/incubator-trafficcontrol/blob/mast >> er/traffic_ops/app/lib/API/Configs/ApacheTrafficServer.pm#L2407 >> > [2] https://github.com/apache/incubator-trafficcontrol/issues/2062 >> > >> > On Fri, Jun 1, 2018 at 6:57 AM, Vijay Anand >> > <[email protected]> wrote: >> >> Hi Rawlin, >> >> >> >> I thought, we can always create an origin server group (for MSO) which >> wont >> >> necessarily contain the origin server which we configure while creating >> a >> >> DS and that is the logic behind adding this conflict. " MSO = true and >> >> go_direct = true". IF that is not the case, we dont need this conflict. >> >> >> >> Thanks, >> >> Vijayanand S >> >> >> >> >> >> >> >> On Thu, May 31, 2018 at 9:08 PM, Rawlin Peters <[email protected] >> > >> >> wrote: >> >> >> >>> When configuring MSO currently (note this will change soon once I get >> >>> the MSO implementation refactored to use Origins rather than Servers), >> >>> you have to set `orgServerFqdn` to an FQDN that matches at least one >> >>> Server with a combined (hostname, domain_name) matching that >> >>> `orgServerFqdn` used in the Delivery Service. That Delivery Service is >> >>> then assigned to that Server and any other Server acting as an origin. >> >>> So the request flow for an MSO Delivery Service should still flow to >> >>> both the origin Server matching the DS's `orgServerFqdn` and any other >> >>> origin Server that is assigned that DS. >> >>> >> >>> - Rawlin >> >>> >> >>> On Thu, May 31, 2018 at 9:12 AM, Vijay Anand >> >>> <[email protected]> wrote: >> >>> > Hi Rawlin, >> >>> > >> >>> > Adding CNAME Alias for sharing an origin to resolve the conflicts >> looks >> >>> > good and it should work. So that , if a conflict is detected, the >> >>> operator >> >>> > has to setup up CNAME alias for that particular origin. >> >>> > >> >>> > For MSO, when some one configures an MSO, he would probably meant to >> use >> >>> an >> >>> > origin server(s) which is different from Delivery service's >> orgin_fqdn. >> >>> So >> >>> > when go_direct is true and mid cache is offline, Edge will go to >> Delivery >> >>> > service configured orgin_fqdn which is not an intented behaviour if >> some >> >>> > one configured MSO. >> >>> > >> >>> > Thanks, >> >>> > Vijayanand S >> >>> > >> >>> > On Thu, May 31, 2018 at 3:17 AM, Rawlin Peters < >> [email protected]> >> >>> > wrote: >> >>> > >> >>> >> Hi Vijayanand S, >> >>> >> >> >>> >> Generally we've found it's bad practice to have multiple delivery >> >>> >> services sharing the same origin due to the conflicts in >> configuration >> >>> >> on the caches serving those delivery services like you mentioned. >> But >> >>> >> this can be fixed by setting up CNAME DNS records for the shared >> >>> >> origin and using a distinct CNAME in each delivery service. In fact >> >>> >> I've discussed duplicate origins here fairly recently due to my >> effort >> >>> >> to refactor the Origin implementation, and the tentative plan was to >> >>> >> phase-in a uniqueness constraint on Origin FQDN so that there will >> be >> >>> >> no possibility of conflicts that we experience today with duplicate >> >>> >> Origin FQDNs. >> >>> >> >> >>> >> Would that fix your issue? >> >>> >> >> >>> >> The `go_direct` option isn't hardcoded per se but is determined by >> the >> >>> >> delivery service type in order to bypass or use the mid tier. So for >> >>> >> HTTP_NO_CACHE, HTTP_LIVE, and DNS_LIVE, we bypass the mid tier >> because >> >>> >> that's what those types are for. Why do you want to bypass the mid >> >>> >> tier for MSO? >> >>> >> >> >>> >> - Rawlin >> >>> >> >> >>> >> On Wed, May 30, 2018 at 8:58 AM, Vijay Anand >> >>> >> <[email protected]> wrote: >> >>> >> > Hi All, >> >>> >> > >> >>> >> > Planning for a PR on making parent.config's go_direct directive >> >>> >> > configurable via Delivery service. Right now, go_direct is >> >>> >> > being hard coded. >> >>> >> > >> >>> >> > Given below is a brief write up on the implementing the same. >> >>> >> > >> >>> >> > Assumption: >> >>> >> > All DS-es sharing an origin server should have same value for >> >>> go_direct. >> >>> >> > >> >>> >> > Implementation: >> >>> >> > Add a new column 'go_direct' in Deliveryservice table. Its value >> >>> defaults >> >>> >> > to False. >> >>> >> > Delivery service UI (Traffic Ops) and API will be enhanced to >> support >> >>> >> this >> >>> >> > new column. >> >>> >> > >> >>> >> > Conflicts: >> >>> >> > 1. DS Type HTTP_NO_CACHE and go direct False >> >>> >> > 2. DS Type HTTP_LIVE and go direct False >> >>> >> > 3. DS Type DNS_LIVE and go direct False >> >>> >> > 4. MSO = true and go_direct = true >> >>> >> > >> >>> >> > >> >>> >> > When we have more than 2 DS-es sharing same origin, then updating >> one >> >>> >> > particular DS's go_direct value will result >> >>> >> > in conflict, since all DS-es sharing an origin should have same >> value >> >>> for >> >>> >> > go_direct. >> >>> >> > >> >>> >> > Such conflicts should be resolved by deleting and recreating these >> >>> DS-es >> >>> >> > with new value for go_direct. >> >>> >> > >> >>> >> > This method of deleting and recreating DS-es is preferred over >> >>> updating >> >>> >> all >> >>> >> > the affected DS-es implicitly. >> >>> >> > >> >>> >> > would like your comments on this. >> >>> >> > >> >>> >> > Thanks, >> >>> >> > Vijayanand S >> >>> >> >> >>> >>
