All, The PR given below is a perl implemention for making parent.config's go_direct directive configurable via Delivery service https://github.com/apache/trafficcontrol/pull/2407
This PR has been hosted to discuss about various approaches to make go_direct configurable. GoLang implementation will be added once we finalize on the approach. Background: --------------- Right now it is hard coded as false in parent.config for DS of types other than HTTP_LIVE, HTTP_NO_CACHE, DNS_LIVE and hence for such delivery services, if there occurs a problem in the network / in the Mids and they become unreachable / offline, all the requests fail because of this hard coded GO_DIRECT setting. By making this configurable, we are giving a choice to the operators to fetch directly from origin under such scenarios. Question: ------------ Originally it was thought of as a new column in Deliveryservice table (Go_Direct). But then Rawlin suggested adding a new delivery serice type to avoid some of the conflict scenario disccussed in the PR. Eric and Rob feels that adding new DS type is not a desired one. Request your views to get a consensus on the suitable approach for this. Thanks, Vijayanand S ---------- Forwarded message ---------- From: Vijay Anand <vijayanand.jayaman...@gmail.com> Date: Thu, Jun 7, 2018 at 6:45 PM Subject: Re: Making parent.config's go_direct directive configurable via Delivery service To: dev@trafficcontrol.apache.org Rawlin, Yes, I am using a version which Eric referred to (i.e) Cisco's version. And looks like in this code it is actually possible to create MSO groups (origin server) which may not contain the org_serv_fqdn. So do you think, MSO enabling and Go Direct = True as mutually exclusive will work? Thanks, Vijayanand S On Mon, Jun 4, 2018 at 8:13 PM, Rawlin Peters <rawlin.pet...@gmail.com> wrote: > Also, it's very possible this might be a nuance between the version of > TO you are running and the vanilla open source TO. We don't assign > delivery services to "groups"; we assign them to individual servers > (although that might mean every single server in a cachegroup). I > think I recall Eric mentioning "server groups" at Cisco. Might that be > the case? > > - Rawlin > > On Mon, Jun 4, 2018 at 8:36 AM, Rawlin Peters <rawlin.pet...@gmail.com> > wrote: > > It might be possible to create an MSO delivery service with an > > `orgServerFqdn` that doesn't match the [hostname + domain name] of a > > server it's assigned to, but I'm not sure it would work as expected. > > In a bug I recently fixed, I found a bit of code [1] that makes me > > think that scenario would lead to a bad parent.config. The issue [2] > > was that having a trailing slash in `orgServerFqdn` led to an empty > > parent field in parent.config (because before the fix, TO didn't find > > an origin Server that matched the `orgServerFqdn` of the MSO delivery > > service. > > > > - Rawlin > > > > [1] https://github.com/apache/incubator-trafficcontrol/blob/mast > er/traffic_ops/app/lib/API/Configs/ApacheTrafficServer.pm#L2407 > > [2] https://github.com/apache/incubator-trafficcontrol/issues/2062 > > > > On Fri, Jun 1, 2018 at 6:57 AM, Vijay Anand > > <vijayanand.jayaman...@gmail.com> wrote: > >> Hi Rawlin, > >> > >> I thought, we can always create an origin server group (for MSO) which > wont > >> necessarily contain the origin server which we configure while creating > a > >> DS and that is the logic behind adding this conflict. " MSO = true and > >> go_direct = true". IF that is not the case, we dont need this conflict. > >> > >> Thanks, > >> Vijayanand S > >> > >> > >> > >> On Thu, May 31, 2018 at 9:08 PM, Rawlin Peters <rawlin.pet...@gmail.com > > > >> wrote: > >> > >>> When configuring MSO currently (note this will change soon once I get > >>> the MSO implementation refactored to use Origins rather than Servers), > >>> you have to set `orgServerFqdn` to an FQDN that matches at least one > >>> Server with a combined (hostname, domain_name) matching that > >>> `orgServerFqdn` used in the Delivery Service. That Delivery Service is > >>> then assigned to that Server and any other Server acting as an origin. > >>> So the request flow for an MSO Delivery Service should still flow to > >>> both the origin Server matching the DS's `orgServerFqdn` and any other > >>> origin Server that is assigned that DS. > >>> > >>> - Rawlin > >>> > >>> On Thu, May 31, 2018 at 9:12 AM, Vijay Anand > >>> <vijayanand.jayaman...@gmail.com> wrote: > >>> > Hi Rawlin, > >>> > > >>> > Adding CNAME Alias for sharing an origin to resolve the conflicts > looks > >>> > good and it should work. So that , if a conflict is detected, the > >>> operator > >>> > has to setup up CNAME alias for that particular origin. > >>> > > >>> > For MSO, when some one configures an MSO, he would probably meant to > use > >>> an > >>> > origin server(s) which is different from Delivery service's > orgin_fqdn. > >>> So > >>> > when go_direct is true and mid cache is offline, Edge will go to > Delivery > >>> > service configured orgin_fqdn which is not an intented behaviour if > some > >>> > one configured MSO. > >>> > > >>> > Thanks, > >>> > Vijayanand S > >>> > > >>> > On Thu, May 31, 2018 at 3:17 AM, Rawlin Peters < > rawlin.pet...@gmail.com> > >>> > wrote: > >>> > > >>> >> Hi Vijayanand S, > >>> >> > >>> >> Generally we've found it's bad practice to have multiple delivery > >>> >> services sharing the same origin due to the conflicts in > configuration > >>> >> on the caches serving those delivery services like you mentioned. > But > >>> >> this can be fixed by setting up CNAME DNS records for the shared > >>> >> origin and using a distinct CNAME in each delivery service. In fact > >>> >> I've discussed duplicate origins here fairly recently due to my > effort > >>> >> to refactor the Origin implementation, and the tentative plan was to > >>> >> phase-in a uniqueness constraint on Origin FQDN so that there will > be > >>> >> no possibility of conflicts that we experience today with duplicate > >>> >> Origin FQDNs. > >>> >> > >>> >> Would that fix your issue? > >>> >> > >>> >> The `go_direct` option isn't hardcoded per se but is determined by > the > >>> >> delivery service type in order to bypass or use the mid tier. So for > >>> >> HTTP_NO_CACHE, HTTP_LIVE, and DNS_LIVE, we bypass the mid tier > because > >>> >> that's what those types are for. Why do you want to bypass the mid > >>> >> tier for MSO? > >>> >> > >>> >> - Rawlin > >>> >> > >>> >> On Wed, May 30, 2018 at 8:58 AM, Vijay Anand > >>> >> <vijayanand.jayaman...@gmail.com> wrote: > >>> >> > Hi All, > >>> >> > > >>> >> > Planning for a PR on making parent.config's go_direct directive > >>> >> > configurable via Delivery service. Right now, go_direct is > >>> >> > being hard coded. > >>> >> > > >>> >> > Given below is a brief write up on the implementing the same. > >>> >> > > >>> >> > Assumption: > >>> >> > All DS-es sharing an origin server should have same value for > >>> go_direct. > >>> >> > > >>> >> > Implementation: > >>> >> > Add a new column 'go_direct' in Deliveryservice table. Its value > >>> defaults > >>> >> > to False. > >>> >> > Delivery service UI (Traffic Ops) and API will be enhanced to > support > >>> >> this > >>> >> > new column. > >>> >> > > >>> >> > Conflicts: > >>> >> > 1. DS Type HTTP_NO_CACHE and go direct False > >>> >> > 2. DS Type HTTP_LIVE and go direct False > >>> >> > 3. DS Type DNS_LIVE and go direct False > >>> >> > 4. MSO = true and go_direct = true > >>> >> > > >>> >> > > >>> >> > When we have more than 2 DS-es sharing same origin, then updating > one > >>> >> > particular DS's go_direct value will result > >>> >> > in conflict, since all DS-es sharing an origin should have same > value > >>> for > >>> >> > go_direct. > >>> >> > > >>> >> > Such conflicts should be resolved by deleting and recreating these > >>> DS-es > >>> >> > with new value for go_direct. > >>> >> > > >>> >> > This method of deleting and recreating DS-es is preferred over > >>> updating > >>> >> all > >>> >> > the affected DS-es implicitly. > >>> >> > > >>> >> > would like your comments on this. > >>> >> > > >>> >> > Thanks, > >>> >> > Vijayanand S > >>> >> > >>> >
