+1 on not copying the DS-server assignments API. I'd also suggest that the
proposed `deliveryServices` and `caches` arrays use xmlID and hostname,
respectively. Idk what server is #2, but I know which one is 'edge'.
Also I, for one, would benefit greatly from a diagram of all these proposed
relationships. And so would the docs :P
________________________________________
From: Rawlin Peters <rawlin.pet...@gmail.com>
Sent: Tuesday, May 14, 2019 6:07 PM
To: dev@trafficcontrol.apache.org
Subject: [EXTERNAL] Re: New Feature: Cache Assignment Groups
This is pretty close to what I was envisioning for Flexible
Cachegroups/Topologies, except I've been throwing around the name
"Service Group" instead of "Cache Assignment Group" since in the
Flexible Cachegroups world, delivery services would be assigned to a
Topology which is basically a tree of Service Groups.
Allowing both mids and edges in a Cache Assignment Group would kind of
go against the Service Group idea, because in a Topology you would
have mids and edges in separate Service Groups that are then related
to each other. If we could make a Cache Assignment Group only contain
either mids or edges (but not both), then I think that would help the
transition into Topologies. Maybe we could add a type on the Cache
Assignment Group to enforce that (e.g. EDGE and MID). I think your
Cache Assignment Groups would still work with that; you'd just have to
assign both the EDGE groups and MID groups separately to the delivery
service.
For the API proposal, I know they map pretty closely to the existing
deliveryservice-server assignments APIs, but I think those endpoints
are often called out as matching the DB schema too closely, making the
API kind of clunky to use. What would you think about unifying those
three endpoints into one endpoint that can handle all of those things?
Basically, the Cache Assignment Group json object could look like this instead:
{
"name": "foo",
...
"deliveryServices": [1, 2, 3],
"caches": [9, 8, 7]
}
which would say "this Cache Assignment Group is assigned to these
delivery services and contains these caches" all in one API resource.
The underlying DB schema could remain as proposed. We could also
support query parameter filters to do things like getting all the
Cache Assignment Groups that are assigned to a particular delivery
service or cache.
One thing that I was hoping to accomplish with Flexible Cachegroups is
doing away with the legacy deliveryservice-server assignments, so
anything we can do with Cache Assignment Groups to help go that
direction would be good. Could we possibly deprecate
deliveryservice-server assignments in favor of Cache Assignment
Groups? We would have to support both assignment strategies for at
least one major release, but I could see us potentially migrating to
this model completely and removing the legacy deliveryservice-server
assignments. What do you think?
Also just so you're aware, @rob05c has some open PRs for rewriting
some of the ATS configs (including parent.config and remap.config)
into Go:
https://protect2.fireeye.com/url?k=9ae15df19fb85f51.9ae17a45-35aa0aa015428654&u=https://github.com/apache/trafficcontrol/pull/3075
(basically done,
will probably merge soon)
https://protect2.fireeye.com/url?k=cc74b0745a56b5e4.cc7497c0-dcfbd695342d1b5c&u=https://github.com/apache/trafficcontrol/pull/2991
(still marked WIP,
not sure how far along it is)
We should probably prioritize getting those in so that your changes
could be made in what we have there so far.
- Rawlin
On Mon, May 13, 2019 at 6:25 AM Eric Friedrich -X (efriedri - TRITON
UK BIDCO LIMITED c/o Alter Domus (UK) Limited -OBO at Cisco)
<efrie...@cisco.com.invalid> wrote:
>
> As it stands, you can choose a subset of edge and mid caches, which is a big
> change from where we stand today.
>
> I think it can lay the groundwork for n-tier hierarchies. Right now one of
> these groups can contain both edges and mids. We could extend the group to
> have a parent/child relationship with other groups (any group that has
> children I believe would need to contain only Mids).
>
>
> —Eric
>
>
>
>
> > On May 12, 2019, at 8:40 PM, Dave Neuman <neu...@apache.org> wrote:
> >
> > This is exciting, thanks Eric.
> > Any idea how well this fits in with flexible cache groups?
> >
> > On Thu, May 9, 2019 at 10:51 Eric Friedrich <efriedr...@synamedia.com>
> > wrote:
> >
> >> GH Issue Link: https://github.com/apache/trafficcontrol/issues/3557
> >>
> >> Background
> >> ----------
> >> Edge caches can currently be assigned to delivery services in three ways:
> >> - By Server Profile
> >> - By Cache Group
> >> - By Individual Cache
> >>
> >> The mids of the assigned edges are chosen based only on the parent and
> >> secondary_parent cachegroup settings.
> >>
> >> Use Cases
> >> ----------------
> >> Create a more flexible method of assigning mid/edge caches to delivery
> >> services. This allows users to solve the following problems:
> >> - Dedicating a subset of caches to particular content providers/CDN
> >> services
> >> - Dedicating a subset of caches to particular content types (i.e. PDL or
> >> HTTPS content)
> >> - Specifying exactly which mid-caches are used for a particular delivery
> >> service.
> >>
> >> The existing "assign by profile" is not sufficient because servers can
> >> only be in a single Profile. This proposal introduces a new Cache
> >> Assignment Group, of which a server can be in more than one. Also, there is
> >> no existing way to influence the selection of mid-caches on a DS-by-DS
> >> basis.
> >>
> >> The first two uses cases can be accomplished with individual server
> >> assignments, but this is cumbersome when you need to match assignments of
> >> lots of servers across lots of delivery services. Let's have the DB
> >> relations manage these logical groupings for us instead.
> >>
> >>
> >> Requirements
> >> ------------
> >> 1) Create Cache Assignment Group in Traffic Ops with a name and description
> >> 2) Allow many to many association of caches to cache assignment groups
> >> 3) Allow many to many association of cache assignment groups to Delivery
> >> Services.
> >> 4) Consider the union of individual cache assignments (in
> >> deliveryservice_server table) and CAG assignments (in new
> >> deliveryservice_assignmentgroup table) when creating remap.config and
> >> CRConfig.json
> >>
> >> 5) Mid-Cache Override: If a mid cache is present in a cache assignment
> >> group, edge caches in that cache assignment group will disregard the parent
> >> and secondary_parent hierarchy. Instead those edges will use only the mids
> >> from that cache assignment group. (No secondary parent support yet). If no
> >> mid is present, edge caches will continue to use parent and
> >> secondary_parent as today.
> >>
> >>
> >> Proposed API changes
> >> --------------------
> >>
> >> /api/1.4/cacheassignmentgroups
> >> Methods: POST, GET (Get all)
> >> /api/1.4/cacheassignmentgroups/:id
> >> Methods: PUT, GET, DELETE
> >>
> >> Structure:
> >> FieldTypeMethod Present
> >> namestringPOST, PUT, GET
> >> descriptionstringPOST, PUT, GET
> >> cdnIdintPOST, PUT, GET
> >> idintGET
> >> lastUpdated stringGET
> >>
> >>
> >> /api/1.4/cacheassignmentgroupserver
> >> Methods: GET, POST
> >>
> >> Structure:
> >> FieldTypeDescription
> >> cagIdintID of assignment group to which servers will be assigned
> >> replaceboolTrue to overwrite existing assignments, false to only add
> >> additional
> >> serversint array Server ids to assigned into the Cache Assignment
> >> Group
> >>
> >>
> >> /api/1.4/deliveryservicecacheassignmentgroup
> >> Methods: GET, POST
> >>
> >> Structure:
> >> FieldType Description
> >> dsIdint ID of delivery service to which cache assignment groups will
> >> be assigned
> >> replacebool True to overwrite existing assignments, false to only add
> >> additional
> >> cacheassignmentgroups int array CAG IDs to assign into the Delivery
> >> Service
> >>
> >>
> >> Proposed DB Schema Changes
> >> -----------------
> >> New Table: cacheassignmentgroup
> >>
> >> FieldTypeNotes
> >> idbigintPrimary Key, auto increment
> >> nametextUnique
> >> descriptiontext
> >> cdn_idbigintForeign Key to cdn(id), trigger to pick one as default
> >> last_updatedtimestampDefault now(), trigger to autoupdate
> >>
> >> New Table: cacheassignmentgroup_server
> >> FieldTypeNotes
> >> cacheassignmentgroupbigintForeign Key to cacheassignmentgroup(id)
> >> serverbigintForeign Key to server(id)
> >>
> >> PrimaryKey/Unique constraint on {cag, server}
> >>
> >> New Table: deliveryservice_cacheassignmentgroup
> >> FieldTypeNotes
> >> deliveryservice bigintForeign Key to deliveryservice(id)
> >> cacheassignmentgroup bigintForeign Key to cacheassignmentgroup(id)
> >>
> >> PrimaryKey/Unique constraint on {deliveryservice, cacheassignmentgroup}
> >>
> >>
> >> Limitations
> >> -----------
> >> If multiple delivery services share the same Origin FQDN and are
> >> assigned to the same caches, ATS requires they share the same set of
> >> parents. Traffic Ops does not currently enforce this restriction.
> >> Validation for this will be added. (I don't think we can accomplish it with
> >> a DB constraint, but if we could that would be awesome. For now, planning
> >> to do it in API validation).
> >>
> >> Notes
> >> ------
> >> I expect this will take at least 4 PRs (TO API/DB changes, Portal, TO ATS
> >> config Gen, TO CRConfig Gen) with maybe some intermediate steps to do some
> >> minor refactoring. We have a working implementation in Perl TO, but the
> >> config generation is pretty messy. As part of this I'll be porting the API
> >> to Go and trying to create a cleaner, more modular implementation of the
> >> remap and parent config generation.
> >>
> >> We also have a feature coming up after this which will modify the "cache
> >> unit of assignment". In that next feature, we'll allow assignment of a
> >> specific IP address to a Delivery Service (or Cache Assignment Group), so
> >> please keep that in mind as you review the API and schema proposals.
> >> Specifically, anywhere we assign a server now, will soon change to server
> >> and/or optionally ip.
> >>
> >>
> >> This message and any attachment are confidential and may be privileged or
> >> otherwise protected from disclosure. If you are not the intended recipient,
> >> please notify the sender immediately and delete this message and any
> >> attachment from your system. Do not copy them or disclose the contents to
> >> any other person.
> >>
>
