Fwd: Making parent.config's go_direct directive configurable via Delivery service
replies inline On Wed, Jun 27, 2018 at 10:59 AM Eric Friedrich (efriedri) wrote: > > Regardless of adding a new DS type, we’ll always have incompatibility between > features. In this case, we could create new DS types for the _BYPASS, but we > would then still need to block enabling MSO on those delivery services. <<>> True, we should probably prohibit MSO on every type of DS that bypasses the mid tier because it won't work. > A few more comments inline > > On Jun 27, 2018, at 12:08 PM, Rawlin Peters wrote: > > > > Alright, I'm going to try to walk through this from the dev > > perspective a bit too. Say we add a go_direct column (non-null, > > default false) to the deliveryservice table. We can't add a new > > required field to the API without breaking it, so the API backend has > > to do a check like this before inserting the DS into the DB: > > > > if go_direct is undefined: > >if ds_type in (HTTP, DNS, HTTP_LIVE_NATNL, DNS_LIVE_NATNL, etc...): > >go_direct = false > >else: > >go_direct = true > EF>The API field would of course be optional like everything else going into > DS table. > > I’d prefer to see this written as: > If go_direct is undefined: > go_direct = ds_type.getGoDirectDefault() > > Much safer and if you add a new DS Type, the Go interface would enforce > implementing that default > (Maybe we don’t have a DSType interface… we should) <<>> I agree, the code would be better that way. The example was more just to show that we would need _something_ like that in TO/TP if we didn't have new types. We do have a DSType type in Go TO, but we'd have to add a "GoDirectDefaulter" interface that every DSType would have to implement. > > That means we would need to have some form of that code in the DS API > > as well as Traffic Portal so that someone doesn't accidentally shoot > > themselves in the foot and create a DS that bypasses the mid tier > > without them realizing it. With new DS types, we don't need > > conditional checks like that or a new column in the DS table. Since > > the types would still be in the DNS/HTTP families, we wouldn't have to > > change CRConfig or everywhere else that discriminates between > > DNS/HTTP. We'd really only have to change the code that generates > > parent.config to check for the new types and set go_direct > > accordingly. > EF> If we added new DS Types I would think we would want to update the > CRConfig as well. It would be pretty confusing to me to have all but 4 of the > DS types represented in the CRConfig. If we wanted to change this convention > separately, I’d suggest following Rob’s earlier advice and adding individual > fields for specific behaviors the TR cares about. I really don’t want to try > and troubleshoot Traffic Ops and wonder why my HTTP_LIVE_NATNL_BYPASS > delivery service is showing up as HTTP_LIVE_NATL in the CRConfig. If I saw > that, I’d think I had the DS type set wrong. We’d also need to update the DS > type documentation with a mapping from TO DS Type to the type as it would > appear in the CRConfig. This mapping could get complicated (i.e. why does TR > find out which DS’s are live and vod today or which DS’ are > national/regional, but not which are bypass). <<>> What I meant by not having to change CRConfig was not having to change CRConfig _generation_. Right now all the DNS-type deliveryservices are put into the DNS-routed bucket, and every other type of DS is put into the HTTP-routed bucket (ANY_MAP is ignored, so this includes all the HTTP_* and STEERING types). So by adding new HTTP_ or DNS_ types, they will get placed into the proper routing protocol buckets without changing CRConfig generation code. TR does not distinguish between _LIVE or whatever, because those differences only affect ATS edge/mid config. > > By adding new type-conflicting fields like go_direct, it also makes it > > slightly more difficult to add new DS types in the future (which we > > may have to do for anycast-routed DSes) because we'd have to maintain > > all the instances of the code snippet above and make sure the new > > types are handled properly with all the type-conflicting fields. > > > > We'd also have to add an asterisk to the DS types in the documentation like: > > HTTP_LIVE_NATNL: same as HTTP_LIVE except the mid-tier is NOT bypassed* > > *unless it IS bypassed by setting go_direct=true on the Delivery Service > EF> We should add that today for MSO :-) Also, I don’t think we need to > asterisk the DS table in this case because its a failure recovery behavior. > We don’t say that the an HTTP_LIVE_NATL has an * with the text “unless all > edges are down then requests are 302’d to the Bypass FQDN" > > > > Which would make choosing a DS type way more confusing IMO. It's like > > this should obsolete the _LIVE types because you could just choose > > _LIVE_NATNL but set go_direct=true to make it _LIVE? > EF> Nope, its only on failure that this happens its not a steady-state >
Fwd: Making parent.config's go_direct directive configurable via Delivery service
All, The PR given below is a perl implemention for making parent.config's go_direct directive configurable via Delivery service https://github.com/apache/trafficcontrol/pull/2407 This PR has been hosted to discuss about various approaches to make go_direct configurable. GoLang implementation will be added once we finalize on the approach. Background: --- Right now it is hard coded as false in parent.config for DS of types other than HTTP_LIVE, HTTP_NO_CACHE, DNS_LIVE and hence for such delivery services, if there occurs a problem in the network / in the Mids and they become unreachable / offline, all the requests fail because of this hard coded GO_DIRECT setting. By making this configurable, we are giving a choice to the operators to fetch directly from origin under such scenarios. Question: Originally it was thought of as a new column in Deliveryservice table (Go_Direct). But then Rawlin suggested adding a new delivery serice type to avoid some of the conflict scenario disccussed in the PR. Eric and Rob feels that adding new DS type is not a desired one. Request your views to get a consensus on the suitable approach for this. Thanks, Vijayanand S -- Forwarded message -- From: Vijay Anand Date: Thu, Jun 7, 2018 at 6:45 PM Subject: Re: Making parent.config's go_direct directive configurable via Delivery service To: dev@trafficcontrol.apache.org Rawlin, Yes, I am using a version which Eric referred to (i.e) Cisco's version. And looks like in this code it is actually possible to create MSO groups (origin server) which may not contain the org_serv_fqdn. So do you think, MSO enabling and Go Direct = True as mutually exclusive will work? Thanks, Vijayanand S On Mon, Jun 4, 2018 at 8:13 PM, Rawlin Peters wrote: > Also, it's very possible this might be a nuance between the version of > TO you are running and the vanilla open source TO. We don't assign > delivery services to "groups"; we assign them to individual servers > (although that might mean every single server in a cachegroup). I > think I recall Eric mentioning "server groups" at Cisco. Might that be > the case? > > - Rawlin > > On Mon, Jun 4, 2018 at 8:36 AM, Rawlin Peters > wrote: > > It might be possible to create an MSO delivery service with an > > `orgServerFqdn` that doesn't match the [hostname + domain name] of a > > server it's assigned to, but I'm not sure it would work as expected. > > In a bug I recently fixed, I found a bit of code [1] that makes me > > think that scenario would lead to a bad parent.config. The issue [2] > > was that having a trailing slash in `orgServerFqdn` led to an empty > > parent field in parent.config (because before the fix, TO didn't find > > an origin Server that matched the `orgServerFqdn` of the MSO delivery > > service. > > > > - Rawlin > > > > [1] https://github.com/apache/incubator-trafficcontrol/blob/mast > er/traffic_ops/app/lib/API/Configs/ApacheTrafficServer.pm#L2407 > > [2] https://github.com/apache/incubator-trafficcontrol/issues/2062 > > > > On Fri, Jun 1, 2018 at 6:57 AM, Vijay Anand > > wrote: > >> Hi Rawlin, > >> > >> I thought, we can always create an origin server group (for MSO) which > wont > >> necessarily contain the origin server which we configure while creating > a > >> DS and that is the logic behind adding this conflict. " MSO = true and > >> go_direct = true". IF that is not the case, we dont need this conflict. > >> > >> Thanks, > >> Vijayanand S > >> > >> > >> > >> On Thu, May 31, 2018 at 9:08 PM, Rawlin Peters > > >> wrote: > >> > >>> When configuring MSO currently (note this will change soon once I get > >>> the MSO implementation refactored to use Origins rather than Servers), > >>> you have to set `orgServerFqdn` to an FQDN that matches at least one > >>> Server with a combined (hostname, domain_name) matching that > >>> `orgServerFqdn` used in the Delivery Service. That Delivery Service is > >>> then assigned to that Server and any other Server acting as an origin. > >>> So the request flow for an MSO Delivery Service should still flow to > >>> both the origin Server matching the DS's `orgServerFqdn` and any other > >>> origin Server that is assigned that DS. > >>> > >>> - Rawlin > >>> > >>> On Thu, May 31, 2018 at 9:12 AM, Vijay Anand > >>> wrote: > >>> > Hi Rawlin, > >>> > > >>> > Adding CNAME Alias for sharing an origin to resolve the conflicts > looks > >>> > good and it should work. So that , if a conflict is detected, the > >>> operator > >>> > has to setup up CNAME alias for that particular origin. > >>> > > >>> > For MSO, when some one configures an MSO, he would probably meant to > use > >>> an > >>> > origin server(s) which is different from Delivery service's > orgin_fqdn. > >>> So > >>> > when go_direct is true and mid cache is offline, Edge will go to > Delivery > >>> > service configured orgin_fqdn which is not an intented behaviour if > some > >>> > one configured MSO. > >>> > > >>> > Thanks, > >>> >
