I'd be a fan of adding a servers array to DS objects. We don't need the whole server object in each entry, just some identifying information (name, id, type should be sufficient I would think). ________________________________________ From: Eric Friedrich -X (efriedri - TRITON UK BIDCO LIMITED c/o Alter Domus (UK) Limited -OBO at Cisco) <[email protected]> Sent: Tuesday, May 21, 2019 8:09 AM To: [email protected] Subject: Re: [EXTERNAL] [PROPOSAL] Cache Assignment Group REST API
I like this, but I think we still have a challenge with tenancy. Two of the very common questions with these groups are 1) Which servers are assigned to this Delivery Service (either individually or through a CAG). Using the existing proposed API, users would first need to call /deliveryService?id= to get a list of CAGs. Then iterate of that list of CAGs calling /server?cag= for each name/id. Not very user friendly. Instead, this question could be answered using /servers?deliveryService=<dsId> But /server still needs to be tenant-aware I dont see a way to put it into the more tenant-friendly /deliveryservice endpoint without adding a “servers” field to the existing DS query params/response. 2) Which delivery services are assigned to this server (needed for remap.config generation but operators also like to see this info) This could be answered with /servers?id=<serverId> If we are OK adding the DS assignments into the server as well Similarly to 1, it could also go into /deliveryservice with the addition of an assigned servers field in the response. —Eric > On May 19, 2019, at 7:33 PM, Chris Lemmons <[email protected]> wrote: > > I like where Rawlin is headed and I think you can take it even one > step simpler. What if you moved the cacheAssignmentGroup into the > server data instead? So it would look like this: > > Server: > { > "id": 1, > "cags": ["foo", "quux"] > ... > } > > Delivery Service: > { > "xmlId": "bar", > "cags": ["foo","bar"], > "tenantId": 7, > ... > } > > If we need it for performance, we could map names to numbers under the > hood, of course. This way tenancy works the way we want it to, and we > can use capabilities to restrict who can adjust server assignments, > which are, as has been observed, multi-tenant. A delivery service is > assigned to a server if one of its cags matches one of the server's > cags. > > For some bonus utility, you could allow most mutating operations to > operate on cags as well, for example to set a group admin-down at > once. > > In the future, if we need really fine control, we could potentially > allow boolean logic on the cag fields, which might be useful for > things like "cags": ["coreprod", "newprod & !canary"], which would > match any server that had a coreprod tag, or a server with newprod but > not canary. It could be used for managing "requirements", since not > all caches might support the same set or versions of plugins > (especially during a rollout of an upgrade). Imagine "cags": > ["prod_east & urisigning16"] for ensuring that the DS is only assigned > to servers with the correct version of urisigning. That sort of logic > could let operators set up very powerful and flexible assignment > systems, even in excess of what we can think up today. We'd need to > design that feature carefully, though, since we probably want to be > able to ask questions like "list all the DSs with the newprod cag", > which can be tricky to answer if we get too clever on our fields. > > On Sun, May 19, 2019 at 2:20 PM Jeremy Mitchell <[email protected]> wrote: >> >> I think what Rawlin has proposed makes a lot of sense and would simplify >> things w.r.t. tenancy. I like simplification. :) >> >> jeremy >> >> On Fri, May 17, 2019 at 1:59 PM Rawlin Peters <[email protected]> >> wrote: >> >>> I've been thinking about this a bit more, and I had a different idea >>> about CAGs w.r.t. tenancy. >>> >>> Maybe the CAGs shouldn't contain the list of delivery services they're >>> assigned to, and instead, the delivery service is enhanced to include >>> the list of CAGs its assigned to. Also, instead of a CAG being >>> tenantable, maybe we just keep tenancy in the delivery service so that >>> if a tenant has access to a delivery service they are free to change >>> their DS's CAGs at will. >>> >>> So we'd end up with something like: >>> CAG: >>> { >>> "name": "foo", >>> "servers": [1,2,3], >>> ... (no tenant ID) >>> } >>> Delivery Service: >>> { >>> "xmlId": "bar", >>> "cacheAssignmentGroups": [7, 8, 9], >>> "tenantId": 7, >>> ... >>> } >>> >>> I'm thinking that "logical server groups" shouldn't really be a >>> tenantable thing, because they are inherently multi-tenant, and we >>> might end up creating a crazy number of overlapping CAGs for all >>> tenants. Adding a new server into multiple overlapping CAGs per tenant >>> might get out of hand. This would allow us to keep the total number of >>> "logical server groups" down but still prevent a tenant from seeing >>> another tenant's CAGs. >>> >>> What do you think? >>> >>> - Rawlin >>> >>> >>> >>> >>> On Thu, May 16, 2019 at 10:18 AM Jeremy Mitchell <[email protected]> >>> wrote: >>>> >>>> yeah, maybe i overcomplicated it with my example and got it wrong. >>>> >>>> if the CAG belongs to tenant 2 and john is the only one that belongs to >>>> tenant 2 (sally belongs to 2.1 and linda to 2.1.1), then john is really >>> the >>>> only one that can modify the CAG. and since the CAG only contains DS's >>> from >>>> tenant 2, 2.1 or 2.2, he can modify ALL the ds's in that CAG... >>>> >>>> so disregard what i said in my example about what sally and linda can >>>> modify because they can't if you add tenantId to CAG. >>>> >>>> so i think >>>> >>>> 1. add a tenantID to a CAG >>>> 2. enforce user tenancy on CAG post/put/delete >>>> 3. on post/put, ensure the tenancy of the assigned ds's are compatible >>> with >>>> the CAG tenant >>>> >>>> jeremy >>>> >>>> >>>> On Thu, May 16, 2019 at 9:08 AM Eric Friedrich -X (efriedri - TRITON UK >>>> BIDCO LIMITED c/o Alter Domus (UK) Limited -OBO at Cisco) >>>> <[email protected]> wrote: >>>> >>>>> Jeremy- >>>>> Going back to your original example of the tree below. >>>>> >>>>> If DS baz is assigned to tenantId 2, then tenantIds 2.1, 2.1.1, 2.2 >>> cannot >>>>> modify baz, right? >>>>> >>>>> If a CAG is created also with tenantId2, then I would expect the same >>>>> behavior- only John as part of the 2 tenant can modify that CAG. >>>>> Similarly, this CAG could only contain DS’ that belong to our are >>> children >>>>> of tenant 2. >>>>> >>>>> This seems to match existing behavior more closely? >>>>> >>>>> —Eric >>>>> >>>>>> On May 16, 2019, at 10:43 AM, Jeremy Mitchell <[email protected] >>>> >>>>> wrote: >>>>>> >>>>>> no, capabilities are very different than tenancy. >>>>>> >>>>>> capabilities dictate what you "can do" - i.e. you can modify CAG's >>> if you >>>>>> have the cacheassignmentgroup-write capability >>>>>> tenancy dictates what you "can do it to". in this case, if CAG's >>> have a >>>>>> tenantId and you have the cacheassignmentgroup-write capability, >>> then you >>>>>> can ONLY modify CAG's that fall in your tenancy scope. >>>>>> >>>>>> although different, capabilities and tenancy work hand in hand to >>> limit >>>>>> what a user can do (permissions) and what they can do that to >>> (scope). >>>>>> >>>>>> because CAG's have an embedded tenantable resource (delivery >>> services), >>>>>> this makes it a bit trickier. not only should tenancy dictate which >>> CAG's >>>>>> can be modified by the user, it should also dictate how those CAG's >>>>> should >>>>>> be modified (i.e. which delivery services can be impacted by a >>>>> modification) >>>>>> >>>>>> jeremy >>>>>> >>>>>> >>>>>> >>>>>> On Wed, May 15, 2019 at 3:13 PM Eric Friedrich -X (efriedri - TRITON >>> UK >>>>>> BIDCO LIMITED c/o Alter Domus (UK) Limited -OBO at Cisco) >>>>>> <[email protected]> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>>> On May 15, 2019, at 4:16 PM, Fieck, Brennan < >>> [email protected] >>>>>> >>>>>>> wrote: >>>>>>>> >>>>>>>> You could just set tenant permissions based on the owning tenant >>> of the >>>>>>> Delivery Service. >>>>>>>> Should the child of a tenant be able to modify cache assignments of >>>>> said >>>>>>> tenant's Delivery Services? >>>>>>>> I wouldn't think so. >>>>>>> EF> Isn’t this what the capabilities are for? If a user has >>>>>>> “cacheassignmentgroup-write” capability, then they can modify the >>>>>>> assignments for any delivery services in that tenant >>>>>>> >>>>>>>> ________________________________________ >>>>>>>> From: Jeremy Mitchell <[email protected]> >>>>>>>> Sent: Wednesday, May 15, 2019 1:22 PM >>>>>>>> To: [email protected] >>>>>>>> Subject: [EXTERNAL] Re: [PROPOSAL] Cache Assignment Group REST API >>>>>>>> >>>>>>>> example tenant tree: >>>>>>>> >>>>>>>> root >>>>>>>> - 1 (foo) >>>>>>>> -- 1.1 (bar) >>>>>>>> - 2 (baz) >>>>>>>> -- 2.1 (bee) >>>>>>>> --- 2.1.1 (bang) >>>>>>>> -- 2.2 (bop) >>>>>>>> >>>>>>>> the foo ds belongs to tenant 1, bar ds belongs to tenant 1.1, etc. >>>>>>>> >>>>>>>> a CGA is created with tenantId=2 which means it can only have the >>> baz, >>>>>>> bee, >>>>>>>> bang and bop ds's in it. John belongs to the 2 tenant and adds all >>> 4 >>>>>>> (baz, >>>>>>>> bee, bang, bop). >>>>>>>> >>>>>>>> Sally belongs to the 2.1 tenant, and only sees [bee, bang] in the >>> CGA >>>>> and >>>>>>>> does an update with [], so it blows away bee (the ds in her >>> tenant) and >>>>>>>> bang (plus any child tenant ds's) >>>>>>>> >>>>>>>> Linda belongs to the 2.1.1 tenant, and sees [] in the CGA (because >>>>> sally >>>>>>>> blew them all away) and does an update with [goo]. now the CAG has >>> baz >>>>>>> and >>>>>>>> goo ds's. >>>>>>>> >>>>>>>> so basically, a user can only modify the ds's that they can see >>> based >>>>> on >>>>>>>> their tenant (and subtenants). and a CAG can only have certain >>> ds's in >>>>> it >>>>>>>> based on it's tenant... >>>>>>>> >>>>>>>> I "think" that would work....sounds a bit complicated but i really >>> feel >>>>>>>> like tenancy probably belongs on a CAG because of its relationship >>> with >>>>>>>> ds's. plus, then it would be nice to create CAG's for tenants. for >>>>>>> example, >>>>>>>> create a trial tenant and some trial ds's and some trial users and >>> they >>>>>>>> have no choice but to use the trial CAG that has 2 caches in it or >>>>>>>> something. >>>>>>>> >>>>>>>> jeremy >>>>>>>> >>>>>>>> On Wed, May 15, 2019 at 1:01 PM Eric Friedrich -X (efriedri - >>> TRITON UK >>>>>>>> BIDCO LIMITED c/o Alter Domus (UK) Limited -OBO at Cisco) >>>>>>>> <[email protected]> wrote: >>>>>>>> >>>>>>>>> What if the tenantId on the cacheassignmentgroup does not match >>> the >>>>>>>>> tenantID on one of the included delivery services? >>>>>>>>> >>>>>>>>> On May 15, 2019, at 2:55 PM, Jeremy Mitchell < >>> [email protected]> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> I got an idea. If you made a cachegroupassignment a "tenantable" >>>>>>>>> resource, >>>>>>>>>> you could avoid the problem i mentioned above (having ds's >>> hidden for >>>>>>>>>> tenancy reasons). so this instead: >>>>>>>>>> >>>>>>>>>> {"id": 1, >>>>>>>>>> "name": "name1", >>>>>>>>>> "description": "description1", >>>>>>>>>> tenantId: 2, >>>>>>>>>> "cdnId": 1, >>>>>>>>>> "servers": [1,2,...n], >>>>>>>>>> "deliveryServices": [10, 20, 30, 35] >>>>>>>>>> "lastUpdated": "", >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> this has a nice benefit as well. i.e. certain tenants (content >>>>>>> providers) >>>>>>>>>> have access to certain cachegroupassignment configurations. >>>>>>>>>> >>>>>>>>>> jeremy >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, May 15, 2019 at 12:43 PM Jeremy Mitchell < >>>>>>> [email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Just be careful that a GET /api/1.4/cacheassignmentgroups?id=4 >>>>> doesn't >>>>>>>>>>> return a filtered list of delivery services because of tenancy >>>>>>>>>>> >>>>>>>>>>> {"id": 1, >>>>>>>>>>> "name": "name1", >>>>>>>>>>> "description": "description1", >>>>>>>>>>> "cdnId": 1, >>>>>>>>>>> "servers": [1,2,...n], >>>>>>>>>>> "deliveryServices": [10, 20, 30], <-- maybe there are really 5 >>>>>>> delivery >>>>>>>>>>> services but 2 of them (40 and 50) are hidden from you due to >>>>> tenancy >>>>>>>>>>> "lastUpdated": "", >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> and a subsequent PUT with the same json (plus a new delivery >>> service >>>>>>>>> that >>>>>>>>>>> is added): >>>>>>>>>>> >>>>>>>>>>> {"id": 1, >>>>>>>>>>> "name": "name1", >>>>>>>>>>> "description": "description1", >>>>>>>>>>> "cdnId": 1, >>>>>>>>>>> "servers": [1,2,...n], >>>>>>>>>>> "deliveryServices": [10, 20, 30, 35] >>>>>>>>>>> "lastUpdated": "", >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> doesn't wipe out 40 and 50. >>>>>>>>>>> >>>>>>>>>>> if you do this, it begs the question. how do you remove ALL ds >>>>>>>>> assignments >>>>>>>>>>> from a cache assignment group? >>>>>>>>>>> >>>>>>>>>>> also, how about >>>>>>>>>>> >>>>>>>>>>> DELETE /api/1.4/cacheassignmentgroups?id= >>>>>>>>>>> >>>>>>>>>>> instead of >>>>>>>>>>> >>>>>>>>>>> DELETE /api/1.4/cacheassignmentgroups/{id} >>>>>>>>>>> >>>>>>>>>>> jeremy >>>>>>>>>>> >>>>>>>>>>> On Wed, May 15, 2019 at 12:22 PM Eric Friedrich -X (efriedri - >>>>> TRITON >>>>>>> UK >>>>>>>>>>> BIDCO LIMITED c/o Alter Domus (UK) Limited -OBO at Cisco) >>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Feature description >>>>>>>>>>>> >>>>>>>>>>>> -------------------- >>>>>>>>>>>> >>>>>>>>>>>> Mail Thread: >>>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> https://lists.apache.org/thread.html/35ee49f4be1c30ff4a12c71e02897aee0e0d3d2f356640ab69ba247e@%3Cdev.trafficcontrol.apache.org%3E >>>>>>>>>>>> < >>>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> https://lists.apache.org/thread.html/35ee49f4be1c30ff4a12c71e02897aee0e0d3d2f356640ab69ba247e@ >>>>>>>>>>>> <dev.trafficcontrol.apache.org>> >>>>>>>>>>>> >>>>>>>>>>>> Github Issue: >>> https://github.com/apache/trafficcontrol/issues/3557 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> API Proposal >>>>>>>>>>>> >>>>>>>>>>>> ------------ >>>>>>>>>>>> >>>>>>>>>>>> POST,GET /api/1.4/cacheassignmentgroups/ >>>>>>>>>>>> >>>>>>>>>>>> PUT /api/1.4/cacheassignmentgroups/{id} >>>>>>>>>>>> >>>>>>>>>>>> {"id": 1, >>>>>>>>>>>> "name": "name1", >>>>>>>>>>>> "description": "description1", >>>>>>>>>>>> "cdnId": 1, >>>>>>>>>>>> "servers": [1,2,...n], >>>>>>>>>>>> "deliveryServices": [10, 20, 30], >>>>>>>>>>>> "lastUpdated": "", >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> DELETE /api/1.4/cacheassignmentgroups/{id} >>>>>>>>>>>> >>>>>>>>>>>> -- No request body >>>>>>>>>>>> >>>>>>>>>>>> This API is tenant-aware by delivery service. >>>>>>>>>>>> >>>>>>>>>>>> Query Parameters (all optional) >>>>>>>>>>>> If multiple filter parameters are specified, they are AND'd >>>>> together >>>>>>>>>>>> - id: Filter for a specific entry >>>>>>>>>>>> - servers: Filter all entries containing this server >>>>>>>>>>>> - deliveryService: Filter all entries containing this DS >>>>>>>>>>>> >>>>>>>>>>>> - limit: Return maximum number of entries (default 20) >>>>>>>>>>>> - page: Return page n, with each page having limit number of >>>>> entries >>>>>>>>>>>> (default 0) >>>>>>>>>>>> >>>>>>>>>>>> Body Parameters: >>>>>>>>>>>> - id: Numeric identifier, automatically assigned on creation. >>>>>>>>> [Read-Only, >>>>>>>>>>>> not allowed in PUT/POST] >>>>>>>>>>>> - name: Name of the cache assignment group >>>>>>>>>>>> - description Description of the cache assignment group >>>>>>>>>>>> - cdnId: ID of the CDN the cache assignment group belongs to >>>>>>>>>>>> - servers: List of server IDs. >>>>>>>>>>>> - deliveryServices: List of delivery service IDs. All caches >>> in the >>>>>>>>>>>> servers list will be assigned to these delivery services >>>>>>>>>>>> - lastUpdated: Timestamp this cache assignment group was last >>>>>>> updated. >>>>>>>>>>>> [Read-Only, not allowed in PUT/POST] >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> >>>
