On Thu, Jul 9, 2020 at 11:20 PM Brian Bouterse <bmbou...@redhat.com> wrote: > > This is all done. I've pushed my code to the links below. You can now use > get_current_authenticated_user in both viewsets and tasks and it will give > you the user, which lets you use signals to automatically add object-level > permissions anywhere automatically. Also tasks have RBAC themselves > (including cancelation and deletion restriction), and support queryset > filtering. > > https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC > https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC > > Here are a few random topics I've been thinking about: > > ## What should the default policy be? > > I propose Pulp ship with a very permissive policy, and this is effectively > with RBAC off (also a requirement, e.g. Katello's usage). Each viewset would > have an AccessPolicy but it would by default ship with the following single > rule which allows everything. > > { > "action": ["*"], > "principal": "authenticated", > "effect": "allow", > }
I think, we can satisfy Katello as well with a very restrictive policy. They are using the one and only available account who happens to be admin anyway. > > ## How can we adopt user-controllable policies? > > The highest goal here is to have users in total control of the policy. This > is in-line with what drf-access-policy recommends: > https://rsinger86.github.io/drf-access-policy/loading_external_source/ > > I propose we store the json policy for each Viewset, e.g. > FileRemoteAccessPolicy which governs File Remotes as a blob that can by > editable via the API itself. This allows us to ship a simple policy like ^ > and allow users to do what they want. We can provide many example policies > also. > > [idea] We could make a sub-resource called "access_policy" so for FileRemote > for example you could read the policy here: > > GET /pulp/api/v3/remotes/file/file/access_policy > PUT /pulp/api/v3/remotes/file/file/access_policy > > It would not support PATCH because we don't allow partial policy changes, > it's one blob. It wouldn't support DELETE because there is always a policy. > It wouldn't support POST because you're modifying a resource in-place. > > ## Requirements driving this > > In my discussions with galaxy_ng their deployment on the cloud versus > on-premise will need a drastically different policy. This tells me that we > need pluggable policies at initial launch. > > ## My main concerns at this point: > > 1) Galaxy_ng also needs flexibility between their cloud versus on-premise > deployment regarding what the obj-level permissions created are. If ^ is > implemented the user can have total control over the policy, but zero control > over what permissions are assigned to newly created objects. For example that > happens in this signal here: > https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC?expand=1#diff-bdb7fbb7c563d6651a309aff97001cb6R8-R12 > > 2) Also can an admin "fix" object-level permissions? Right now they are only > in the DB. I suspect the answer is for us to turn on more django-guardian > admin integration: > https://github.com/django-guardian/django-guardian#admin-integration > > I have to think about this last part more... Meanwhile I plan to go forward > with the user-controllable policies next. Input and feedback is welcome. > > Cheers, > Brian > > > > > On Wed, Jul 8, 2020 at 4:53 PM Brian Bouterse <bmbou...@redhat.com> wrote: >> >> My next goal is to have object-level permissions assigned through signals so >> that anywhere you save the model the permissions are correctly created. To >> do this I need to get a few things working: >> >> 1) Move the permissions creation to the signals [done] >> 2) Have the user be well-known, this is accomplished by pulpcore picking up >> this dependency https://github.com/PaesslerAG/django-currentuser [done] >> 3) Have the user information persisted in tasks so signals can work there >> also ... there are two obvious options >> >> option a) Have tasks themselves have RBAC which we know we need anyway, then >> query the RBAC permissions inside a task to determine which user has the >> object level permissions for this task. This is what I'm currently >> prototyping. It at least will provide a mechanism we can use for now until >> we find something better. >> >> option b) Add a user field to the Task itself and have the RBAC assign >> permissions to them. The concern with this is that while this is nice, it >> would be nicer if we had this type of visibility on everything in Pulp from >> an auditing project dedicated to keeping this type of info. Option (b) is >> still viable, just maybe not the best way. Comments/feedback is welcome. >> >> Also I've collaborated with @alikins from galaxy_ng project and together we >> wrote this requirements doc for their plugin: >> https://hackmd.io/JisLkfyeT2myAD2khEwU0A >> >> On Wed, Jul 1, 2020 at 6:43 PM Brian Bouterse <bmbou...@redhat.com> wrote: >>> >>> The demo advertisement for tomorrow is here: >>> https://www.redhat.com/archives/pulp-dev/2020-June/msg00076.html >>> >>> On Wed, Jul 1, 2020 at 6:41 PM Brian Bouterse <bmbou...@redhat.com> wrote: >>>> >>>> Another productive RBAC day! See the latest code at the links below. >>>> Here's what's new: >>>> >>>> * policy is now shorter thanks to machinery checking both model-level and >>>> object-level permissions with one call. The other two are also available >>>> * sync is now restricted on both 'modify_repo_content' permissions AND >>>> read permission on the remote being used to sync >>>> * modify is now restricted on 'modify_repo_content' permission >>>> * moved the permission checking machinery to be "global checks" >>>> * added data migration that sets is_staff=True, so the django-admin >>>> interface can be used (this is getting a slight rework tomorrow morning >>>> tho) >>>> >>>> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC >>>> https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC >>>> >>>> Tomorrow's demo is advertised here. It will also include an overview of >>>> some of the unsolved problems with some possible solutions. >>>> >>>> Cheers, >>>> Brian >>>> >>>> >>>> On Tue, Jun 30, 2020 at 5:08 PM Brian Bouterse <bmbou...@redhat.com> wrote: >>>>> >>>>> Today I accomplished a few more things: >>>>> >>>>> * finished my ldap notes: https://hackmd.io/ED9UpscNSRW86Le3xNzVeg >>>>> * moving the checks from a mixin to be "global checks" so they are >>>>> available everywhere, this is a feature from drf-access-policy: >>>>> https://rsinger86.github.io/drf-access-policy/reusable_conditions/ >>>>> * added a has_obj_or_module_perms method allowing policy writers to just >>>>> use that instad of carrying "two entries" in the policy, one for >>>>> model-level, one for object-level >>>>> >>>>> Need to: >>>>> * clean up the "sync" policy code >>>>> * Add global condition check facilities for the perms of a 'remote' param >>>>> * add policy language restricting the /modify/ endpoint also for >>>>> FileRepository >>>>> * push my code >>>>> >>>>> New Challenge: We need to also have the permissions assignments happen >>>>> for objects created by tasks. django-guardian recommends this happen >>>>> inside signals >>>>> (https://django-guardian.readthedocs.io/en/latest/userguide/assign.html#assigning-permissions-inside-signals). >>>>> The challenge (thanks @mdellweg for identifying) is that the user/group >>>>> context information is well-known in the viewset but not in a task. >>>>> Soooooo ... the idea is: >>>>> >>>>> 1. Switch the perms addition to the model itself via signals so it's >>>>> automatic everywhere (including in tasks) >>>>> 2. Preserve the user and group "request context" into the tasking system. >>>>> I can see a straightforward path to how to do this so I plan to prototype >>>>> this soon also. >>>>> >>>>> Feedback is welcome! >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Fri, Jun 26, 2020 at 6:16 PM Brian Bouterse <bmbou...@redhat.com> >>>>> wrote: >>>>>> >>>>>> Today I got the "sync" RBAC working, but I need to give it some more >>>>>> thought. The extra challenge with this parts is that "having permission >>>>>> to read a Remote" is already defined in one place, on >>>>>> FileRemoteAccessPolicy, yet the AccessPolicy that needs to perform the >>>>>> enforcement is FileRepositoryAccessPolicy for its "sync" action. This is >>>>>> a bit challenging considering the following goals: >>>>>> >>>>>> * We don't want to duplicate code, e.g. having the >>>>>> FileRepositoryAccessControl begin to inspect permissions for FileRemote >>>>>> directly, when FileRemoteAccessPolicy already does that >>>>>> * Currently permissions are granted at two levels: Model-level and >>>>>> File-level permissions and permissions are granted from either level. >>>>>> * We want to keep the policy in charge. If we start to bury the behavior >>>>>> in methods and functions then policy writers are no longer in control. >>>>>> >>>>>> All of ^ together tells me that I should work on creating two things >>>>>> next: >>>>>> 1) A way for policy writers to express which parameter refers to objects >>>>>> that also need their permissions checked. For example the policy should >>>>>> be able to say "remote is a parameter and it needs X permission". This >>>>>> is akin to the has_module_level_perms and has_obj_level_perms here >>>>>> except we also need to identify which parameter is being checked instead >>>>>> of the object the AccessPolicy itself governs. >>>>>> 2) A single way to check model-level and object-level permissions at >>>>>> once and allow if *either* passes. We would still allow policy writers >>>>>> to call either model-level or file-level checks also. >>>>>> >>>>>> I'll work on ^ next. Ideas and feedback are welcome. I pushed no new >>>>>> code today because it's a mess and not runnable at my stopping point. >>>>>> >>>>>> >>>>>> On Thu, Jun 25, 2020 at 6:18 PM Brian Bouterse <bmbou...@redhat.com> >>>>>> wrote: >>>>>>> >>>>>>> Here's another push to the branch (it includes the following >>>>>>> additions): >>>>>>> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC?expand=1 >>>>>>> >>>>>>> * A FileRepositoryAccessPolicy which provides RBAC for Repositories >>>>>>> (not yet sync) >>>>>>> * A new Mixin allowing the two policies to share some common components >>>>>>> >>>>>>> Next up: >>>>>>> * have the pup_file define the fileContentAdmin group programmatically >>>>>>> * Extend the FileRepositoryAccessPolicy to restrict sync operations >>>>>>> * Write up and organize the PoC into a clear, organized format >>>>>>> >>>>>>> Also of interest today @ttereshc and I had a great convo asking what to >>>>>>> do about potential problems when we use Django groups to be a "role". >>>>>>> My write up will address this in more detail than I can go into here. >>>>>>> We are also looking at what the django-role-permissions project could >>>>>>> offer us: >>>>>>> https://django-role-permissions.readthedocs.io/en/stable/utils.html >>>>>>> >>>>>>> I expect the PoC to be done by tomorrow and write-up by Monday, so I'm >>>>>>> going to schedule the public review meeting for next week towards the >>>>>>> end of the week. >>>>>>> >>>>>>> >>>>>>> On Wed, Jun 24, 2020 at 5:49 PM Brian Bouterse <bmbou...@redhat.com> >>>>>>> wrote: >>>>>>>> >>>>>>>> Moar progress! Today the following things got done: Today's changes >>>>>>>> are available here: >>>>>>>> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC?expand=1 >>>>>>>> >>>>>>>> * Got scoped querysets working! This restricts list views to only show >>>>>>>> objects a user has permissions to view. A db reset was all that was >>>>>>>> needed I think I didn't have all the changes in when I applied my >>>>>>>> earlier migrations >>>>>>>> * Added "detail view" restriction, and while it's in the policy and >>>>>>>> working DRF does a strange thing on "retrieve" where if it's not in >>>>>>>> the queryset (due to scoping ^) the user receives a 404, not a >>>>>>>> permission denied >>>>>>>> * Got permissions cleaning up on resource deletion now too >>>>>>>> >>>>>>>> Next up: >>>>>>>> * have the pup_file define the fileContentAdmin group programmatically >>>>>>>> * Make similar policies for FileRepository which governs itself and >>>>>>>> the "sync" action >>>>>>>> * Write up and organize the PoC into a clear, organized format >>>>>>>> >>>>>>>> Questions and feedback are welcome! >>>>>>>> >>>>>>>> On Tue, Jun 23, 2020 at 5:54 PM Brian Bouterse <bmbou...@redhat.com> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Lots of progress today! I have a mostly-complete policy for RBAC for >>>>>>>>> FileRemote. It's surprising how little code all of this ended up >>>>>>>>> being. >>>>>>>>> >>>>>>>>> Here's the actual RBAC stuff, it's all in pulp_file: >>>>>>>>> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC?expand=1 >>>>>>>>> Here's the parts that go in core. Note the LDAP stuff is all >>>>>>>>> optional, the only real requirement are two lines 1) enabling >>>>>>>>> guardian in INSTALLED_APPS and 2) adding it as an >>>>>>>>> AuthenticationBackend: >>>>>>>>> https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC >>>>>>>>> >>>>>>>>> I have some "how to use notes" here: >>>>>>>>> https://hackmd.io/DRqGFyRsSDmN7E4TtOPf-w The idea is that it >>>>>>>>> implements the FileRemote portions of this requirements docs: >>>>>>>>> https://hackmd.io/kZ1oYp8TTkeuC5KL_ffjGQ >>>>>>>>> >>>>>>>>> Here is the short list of things for FileRemote that still don't >>>>>>>>> work. This is mainly so I remember what to do next. :) >>>>>>>>> * The get_objects_for_user from DjangoGuardian I don't think it likes >>>>>>>>> Master/Detail or maybe it's how/where I'm using it. I haven't yet >>>>>>>>> debugged this. For this reason it doesn't provide list restriction >>>>>>>>> * It still needs "detail view" restriction. This is straightforward. >>>>>>>>> * The group should be programmatically defined, in this case it was >>>>>>>>> "defined in LDAP". It could *also* live in LDAP (or other external >>>>>>>>> group definition system) but the plugin builds permissions off of it >>>>>>>>> so it should also define it. This is easy. >>>>>>>>> >>>>>>>>> Feedback is welcome. I'm going to continue building this and then >>>>>>>>> schedule a public review of FileRemote, Content modification for file >>>>>>>>> repos, and sync restriction next week. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Jun 22, 2020 at 5:14 PM Brian Bouterse <bmbou...@redhat.com> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> # ldap PoC updates >>>>>>>>>> Now users, groups, and group membership are populating from ldap >>>>>>>>>> automatically on login (with auth backed by ldap also)! I'll be >>>>>>>>>> sharing my configs for both ldap and how to configure >>>>>>>>>> django-auth-ldap here soon in an organized way. This was done with >>>>>>>>>> django-auth-ldap and 0 customization to pulp code. It's 100% enabled >>>>>>>>>> through settings so this work is more of an approach we can document >>>>>>>>>> for users that they can enable and not a feature Pulp ships itself. >>>>>>>>>> >>>>>>>>>> # django-admin progress >>>>>>>>>> Thanks to @alikins existing PRs, I got django admin enabled and able >>>>>>>>>> to view/edit users, groups, group membership, and permissions at >>>>>>>>>> both the user and group levels. This is important because this will >>>>>>>>>> be the primary mechanism of administrators. This part is looking >>>>>>>>>> good. >>>>>>>>>> >>>>>>>>>> # new resources to help us out >>>>>>>>>> Through collaboration with @ttereshc and someone off list named >>>>>>>>>> @adelton (who actually authored this reference approach I referenced >>>>>>>>>> early on in this exploration), this very cool repository of testing >>>>>>>>>> tools was identified: https://github.com/adelton/webauthinfra It >>>>>>>>>> has a treasure trove of testing containers which Pulp devs in the >>>>>>>>>> future can test against. It keeps the user/group check in the apache >>>>>>>>>> which is fine alternative to the django-auth-ldap approach above. >>>>>>>>>> Pulp doesn't have to choose, it could work with either just >>>>>>>>>> configured differently. The pending PoC outline will go over these >>>>>>>>>> alternative approaches in detail. >>>>>>>>>> >>>>>>>>>> # Next Steps: back to the PoC itself >>>>>>>>>> Now that we have demonstrated good options of external >>>>>>>>>> users/groups/membership loading into Pulp we can confidently move >>>>>>>>>> back to finishing the RBAC PoC itself. I've started back into this. >>>>>>>>>> So the remaining work are the two steps below: >>>>>>>>>> >>>>>>>>>> 1. Finish the PoC that uses RBAC to restrict remotes, sync, and >>>>>>>>>> repository content modification. Currently I prototyped restriction >>>>>>>>>> of operations on Remotes, but I need to replicate the access >>>>>>>>>> policies to Repositories and Sync next. >>>>>>>>>> 2. Write it up and share it. >>>>>>>>>> 3. Schedule public meeting to review it (targeting next-week) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Jun 19, 2020 at 5:09 PM Brian Bouterse <bmbou...@redhat.com> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> I got the LDAP users both authenticating and importing into Pulp! >>>>>>>>>>> Next I'll do the groups and then I think the ldap parts will be >>>>>>>>>>> done. >>>>>>>>>>> >>>>>>>>>>> FYI: I'm going to write up the implementation design and have that >>>>>>>>>>> come with this proof of concept code . This will let us know what >>>>>>>>>>> choices it makes, why it makes them, and we can determine if these >>>>>>>>>>> are the right choices together. >>>>>>>>>>> >>>>>>>>>>> On Wed, Jun 17, 2020 at 4:57 PM Brian Bouterse >>>>>>>>>>> <bmbou...@redhat.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>> I got a lot further on this today. I have the test ldap setup with >>>>>>>>>>>> several test users and groups. I have django-auth-ldap configured >>>>>>>>>>>> mostly authenticating username/password against ldap instead of >>>>>>>>>>>> the internal database first. Once that is fully working the users >>>>>>>>>>>> will auto-populate into django and the groups should follow easily. >>>>>>>>>>>> >>>>>>>>>>>> Once that's done I'll be unblocked to finish the RBAC PoC. The >>>>>>>>>>>> rest of the parts are straightforward given the testing I've >>>>>>>>>>>> already done. More updates to come. >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Jun 15, 2020 at 5:03 PM Brian Bouterse >>>>>>>>>>>> <bmbou...@redhat.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> I got the ldap reference implementation performing auth really >>>>>>>>>>>>> nicely against a test ldap with this guide: >>>>>>>>>>>>> https://www.nginx.com/blog/nginx-plus-authenticate-users/ Now >>>>>>>>>>>>> there are some new challenges though: >>>>>>>>>>>>> >>>>>>>>>>>>> * Great that we can auth users, but we need nginx to >>>>>>>>>>>>> extract-and-forward the group information to Pulp itself. That >>>>>>>>>>>>> way a middleware can create the user AND group info in the >>>>>>>>>>>>> backend. >>>>>>>>>>>>> * we have to figure this out all again in Apache... >>>>>>>>>>>>> >>>>>>>>>>>>> Maybe we should be integrating Pulp directly against >>>>>>>>>>>>> django-auth-ldap [0]. I am going to try that next. The work I've >>>>>>>>>>>>> done isn't 100% reusable there, but most of it is because the >>>>>>>>>>>>> test server and configs I used can all be reused directly with >>>>>>>>>>>>> django-auth-ldap. The concern with this approach is that we would >>>>>>>>>>>>> be supporting LDAP (and transitively Active Directory) but are >>>>>>>>>>>>> there other directory services Pulp needs to support? >>>>>>>>>>>>> >>>>>>>>>>>>> I also emailed Bin Li asking for info on how their user and group >>>>>>>>>>>>> management works. >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Jun 9, 2020 at 11:48 AM Adrian Likins >>>>>>>>>>>>> <alik...@redhat.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Fri, Jun 5, 2020 at 8:23 PM Brian Bouterse >>>>>>>>>>>>>> <bmbou...@redhat.com> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 1) django admin (the built in django UI) will be the mechanism >>>>>>>>>>>>>>> administrators use to assign permissions to users and groups. >>>>>>>>>>>>>>> This means the use of django admin with pulp is very likely (to >>>>>>>>>>>>>>> me). >>>>>>>>>>>>>>> >>>>>>>>>>>>>> Hopefully https://github.com/pulp/pulpcore/pull/705 will be >>>>>>>>>>>>>> useful here. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 2) externally defined users and groups will need to be >>>>>>>>>>>>>>> "replicated" to django's db at login time, probably using >>>>>>>>>>>>>>> headers from the webserver This is consistent w/ the approach >>>>>>>>>>>>>>> recommended here: >>>>>>>>>>>>>>> https://www.adelton.com/django/external-authentication-for-django-projects >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> This is more or less what galaxy_ng ends up doing, at least for >>>>>>>>>>>>>> the scenarios where it runs hosted with external SSO. >>>>>>>>>>>>>> https://github.com/ansible/galaxy_ng/blob/master/galaxy_ng/app/auth/auth.py#L51 >>>>>>>>>>>>>> for example. >>>>>>>>>>>>>> > > _______________________________________________ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev _______________________________________________ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
