Hi Rawlin, Can you please add a few word for the motivation behind basing the steering target selection on the location of the client? As the content goes through the caches, isn't it the job of the cache to select the best origin for the cache? Why the client should be the one to take the origin location into consideration? Why the target DSes have different origins in the first place? Are they have different characteristics additionally to their location? Thanks, Nir
---------- Forwarded message ---------- From: Rawlin Peters <rawlin.pet...@gmail.com> Date: Mon, Mar 12, 2018 at 9:46 PM Subject: Delivery Service Origin Refactor To: dev@trafficcontrol.incubator.apache.org Hey folks, As promised, this email thread will be to discuss how to best associate an Origin Latitude/Longitude with a Delivery Service, primarily so that steering targets can be ordered/sent to the client based upon the location of those targets (i.e. the Origin), a.k.a. Steering Target Geo-Ordering. This is potentially going to be a pretty large change, so all your feedback/questions/concerns are appreciated. Here were a handful of bad ideas I had in order to accomplish this DS Origin Lat/Long association (feel free to skip to PROPOSED SOLUTION below): 1. Reuse the current MSO (multisite origin) backend (i.e. add the origin into the servers table, give it a lat/long from its cachegroup, assign the origin server to the DS) Pros: - reuse of existing db schema, probably wouldn't have to add any new tables/columns Cons: - MSO configuration is already very complex - for the simple case of just wanting to give an Origin a lat/long you have to create a server (of which only a few fields make sense for an Origin), add it to a cachegroup (only name and lat/long make sense, won't use parent relationships, isn't really a "group" of origins), assign it to a server profile (have to create one first, no parameters are needed), and finally assign that Origin server to the delivery service (did I miss anything?) 2. Add Origin lat/long columns to the deliveryservice table Pros: - probably the most straightforward solution for Steering Target Geo-Ordering given that Origin FQDN is currently a DS field. Cons: - doesn't work well with MSO - could be confused with Default Miss Lat/Long - if two different delivery services use colocated origins, the same lat/long needs entered twice - adds yet another column to the crowded deliveryservice table 3. Add origin lat/long parameters to a Delivery Service Profile Pros: - Delivery Services using colocated origins could share the same profile - no DB schema updates needed Cons: - profile parameters lack validation - still doesn't support lat/long for multiple origins associated with a DS 4. Add the lat/long to the steering target itself (i.e. where you choose weight/order, you'd also enter lat/long) Pros: - probably the easiest/quickest solution in terms of development Cons: - only applies lat/long to a steering target - using the same target in multiple Steering DSes means having to keep the lat/long synced between them all - lat/long not easily reused by other areas that may need it in the future PROPOSED SOLUTION: All of those ideas were suboptimal, which is why I think we need to: 1. Split Locations out of the cachegroup table into their own table with the following columns (cachegroup would have a foreign key to Location): - name - latitude - longitude 2. Split Origins out of the server and deliveryservice tables into their own table with the following columns: - fqdn - protocol (http or https) - port (optional, can be inferred from protocol) - location (optional FK to Location table) - deliveryservice FK (if an Origin can only be associated with a single DS. Might need step 3 below for many-to-many) - ip_address (optional, necessary to support `use_ip_address` profile parameter for using the origin's IP address rather than fqdn in parent.config) - ip6_address (optional, necessary because we'd have an ip_address column for the same reasons) - profile (optional, primarily for MSO-specific parameters - rank and weight - but I could be convinced that this is unnecessary) - cachegroup (optional, necessary to maintain primary/secondary relationship between MID_LOC and ORG_LOC cachegroups for MSO) 3. If many-to-many DSes to Origins will still be possible, create a new deliveryservice_origin table to support a many-to-many relationship between DSes and origins - the rank/weight fields for MSO could be added here possibly, maybe other things as well? 4. Consider constraints in the origin and deliveryservice_origin table - must fqdn alone be unique? fqdn, protocol, and port combined? The process for creating a Delivery Service would change in that Origins would have to be created separately and added to the delivery service. However, to aid migration to the new way of doing things, our UIs could keep the "Origin FQDN" field but the API backend would then create a new row in the Origin table and add it to the DS. More Origins could then be added (for MSO purposes) to the DS via a new API endpoint. MSO configuration would change at least in how Origins are assigned to a DS ("server assignments" would then just be for EDGE-type servers). Cachegroup creation also changes in that Locations need to be created before associating them to a Cachegroup. However, our UIs could also stay the same with the backend API updated to create a Location from the Cachegroup request and tie it to the Cachegroup. I know there are a lot of backend and frontend implications with these changes that would still need to be worked out, but in general does this proposal sound good? Questions/concerns/feedback welcome and appreciated! - Rawlin