Re: [Freeipa-devel] Switching to pytest
On 10/06/2014 11:21 PM, John Dennis wrote: On 10/03/2014 09:24 AM, Petr Viktorin wrote: https://fedorahosted.org/freeipa/ticket/4610 Our test suite is currently not very maintainable. I want to dedicate some time to improve this. The biggest part of this effort will be switching to a different test framework, [pytest]. Compared to Nose, it makes complicated tests easier to write -- at the expense of using more magic. Just looking for information, why pytest? Nose is not enough, and knowing nothing clearly better (easier, more supportable, more popular*) than pytest, I went with what I know. * unittest/nose is more popular than pytest, of course Here is why I'm asking. After leaving IPA I went to another large Python based project, OpenStack. OpenStack is really a number of loosely affiliated autonomous projects, each of which gets to decide how they are going to write and run their unit tests. As might be expected it's a bit of a mess. But there has been some convergence lately using tox, testr and testtools in combination.Python like Java has no shortage of test frameworks and that diversity is actually a pain point. What happens is very few folks actually understand the intricacies of test construction and execution (because there are so many options, it can be convoluted, there are many layers and each project does things differently). The net result is a lot of wasted time, I groan every time tests get ported to the test framework de jour, which seems to happen with alarming frequency. I'd really like to see some convergence in the Python community on test frameworks so as a developer you only have to be an expert in one framework because debugging test failures (including simply reproducing the failure) is currently a time sink and just when you think you understand what the heck is going on someone announces we're moving to the next big thing. The proliferation of test frameworks is further compounded by weak documentation. If something goes boom it can literally take days to piece together enough information to diagnose the issue (often because there are so many layers). I sympathize. Do you have a solution, though? Staying with Nose is not one at this point. I'm not too convinced by convergence in OpenStack; it's still just one project. If you have some other arguments for testr/testtools, let me know; I don't see the advantages over pytest. So how does pytest fit in with everything else? Is stable, long term and widely adopted? What about mocking? Is there a consensus on fixtures vs. mocks (that isn't religious). Can pytest handle both? I'm not sure I understand correctly what you mean, but you can wrap mocks/monkeypatching in fixtures. So yes, pytest can handle both. I guess what I'm asking is there some convergence with Python test tools? Or with testing are we in the equivalent of the SCM wars of 5-7 years ago until most everyone figured out git was superior and had the most traction? Yeah, I guess you could say the testing war is raging on. But even the SCM situation is not clear-cut; e.g. Mozilla or Python use Mercurial, and given the availability of bridge tools, they aren't likely to switch any time soon. And the Linux package manager war is also raging on, with no clear winner. Picking the test framework other than the one that bubbles to the top and becomes the de facto standard has consequences. I clearly do not have enough information to make a judgment here and any additional information would be very much appreciated. Perhaps it's because I'm in Europe, but every major Python conference I go to has one or two pytest talks. Also, pytest was written by the same people as tox, which (I dare say) is the de-facto standard for test *orchestration* in Python. So in the test framework war I'm betting on this one. -- PetrĀ³ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 0159-0160 Support ID views in compat tree
On 10/01/2014 06:16 PM, Alexander Bokovoy wrote: Hi! Attached are patches to add support of FreeIPA ID views to Schema compatibility plugin (slapi-nis). There are two patches for FreeIPA and a separate patch for slapi-nis. Patches can be applied independently; if old slapi-nis is installed, it will simply work with new configuration but do nothing with respect to answering to requests using host-specific ID views. I included documentation on how slapi-nis ID views feature supposed to work, available in slapi-nis/doc/ipa/ipa-sch.txt. Any comments and fixes are welcome. There are no additional tests in slapi-nis to cover compat trees, we have multiple tests in FreeIPA for this purpose, will be run as part of FreeIPA CI effort. FreeIPA patches add ACIs for accessing ID view-applied entries over compat tree. They also include additional configuration; this configuration is needed to properly resolve ID view overrides when creating compat entries. A second FreeIPA patch adds support to override login shell. This part was missing from the original patchset by Tomas. For trusted AD users one needs patches to SSSD 1.12.2, made by Sumit Bose. There is also a regression (fixed by Sumit as well) that prevents authentication of AD users over PAM which affects authentication over compat tree. With the patch from Sumit authentication works again, both with ID view and without it. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel Hello Alexander, A question about backend_search_filter_has_cn_uid. It checks if a filter components contains (uid|uidNumber|gidNumber|memberUid) and in this case returns SLAPI_FILTER_SCAN_STOP. This value will interrupt the filter rewriting. In addition, for each component it calls idview_process_filter_cb to override an attribute that needs to be override in the view. So I wonder if it will work for filter like: ((attribute_to_override=xxx) (uid=yyy)) but will stop to early for ((uid=yyy)(attribute_to_override=xxx)) thanks thierry ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 0159-0160 Support ID views in compat tree
On Tue, 07 Oct 2014, thierry bordaz wrote: A question about backend_search_filter_has_cn_uid. It checks if a filter components contains (uid|uidNumber|gidNumber|memberUid) and in this case returns SLAPI_FILTER_SCAN_STOP. This value will interrupt the filter rewriting. In addition, for each component it calls idview_process_filter_cb to override an attribute that needs to be override in the view. So I wonder if it will work for filter like: ((attribute_to_override=xxx) (uid=yyy)) but will stop to early for ((uid=yyy)(attribute_to_override=xxx)) We handle the requests which come from various NSS clients here (nss-pam-ldapd, nss_ldap, sssd, and similar). They don't do reordering like you say. Since all these searches never base queries on attributes other the ones we check (uid|uidNumber|gidNumber|memberUid), we don't need to replace other attributes here. I can see there could be a case where we get something like (actual query): ((gidNumber=1878600500) (|(objectClass=groupOfNames) (objectClass=posixGroup)) (cn=*)((gidNumber=*)(!(gidNumber=0 and we need to ignore '*' and '0' if they come before explicit gidNumber=id, I'll fix this one. Do you think we need to traverse the filter tree until it ends in all cases? In all filter types I've seen we are interested only in a single name and once we've got attributes filled in, we don't need to continue. Maybe for the replacement case of idview_process_filter_cb() we can have some flag in the filter processing config that forces to go through the fliter no matter what? -- / Alexander Bokovoy ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 0159-0160 Support ID views in compat tree
Hi Alex, slapi_attr_set_valueset() calls slapi_valueset_set_valueset(), but this does not free the existing values, cf https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/9.0/html/Plug-in_Guide/Plugin_Programming_Guide-Function_Reference-slapi_valueset_set_valueset.html ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 0159-0160 Support ID views in compat tree
On Tue, 07 Oct 2014, Ludwig Krispenz wrote: Hi Alex, slapi_attr_set_valueset() calls slapi_valueset_set_valueset(), but this does not free the existing values, Here is the problem, I cannot free original values as slapi_attr_get_valueset() returns a copy and I never can get access to the original Slapi_ValueSet. I only can get access to the original Slapi_Value** array which means I could manipulate its content but that probably require resizing it and thus changing a pointer vs-va, to which I don't have access anyway. How can we fix this? -- / Alexander Bokovoy ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 0159-0160 Support ID views in compat tree
On 10/07/2014 12:12 PM, Alexander Bokovoy wrote: On Tue, 07 Oct 2014, Ludwig Krispenz wrote: Hi Alex, slapi_attr_set_valueset() calls slapi_valueset_set_valueset(), but this does not free the existing values, Here is the problem, I cannot free original values as slapi_attr_get_valueset() returns a copy and I never can get access to the original Slapi_ValueSet. good point. I only can get access to the original Slapi_Value** array which means I could manipulate its content but that probably require resizing it and thus changing a pointer vs-va, to which I don't have access anyway. you should never do this, a valueset contains more data than va How can we fix this? the best thing would probably be to fix slapi_attr_set_valueset or adding a new api function which frees teh old valueset, but this would add a dependency to specific DS version ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 0159-0160 Support ID views in compat tree
On Tue, 07 Oct 2014, Ludwig Krispenz wrote: On 10/07/2014 12:12 PM, Alexander Bokovoy wrote: On Tue, 07 Oct 2014, Ludwig Krispenz wrote: Hi Alex, slapi_attr_set_valueset() calls slapi_valueset_set_valueset(), but this does not free the existing values, Here is the problem, I cannot free original values as slapi_attr_get_valueset() returns a copy and I never can get access to the original Slapi_ValueSet. good point. I only can get access to the original Slapi_Value** array which means I could manipulate its content but that probably require resizing it and thus changing a pointer vs-va, to which I don't have access anyway. you should never do this, a valueset contains more data than va How can we fix this? the best thing would probably be to fix slapi_attr_set_valueset or adding a new api function which frees teh old valueset, but this would add a dependency to specific DS version What about something like this: void slapi_valueset_set_valueset(Slapi_ValueSet *vs1, const Slapi_ValueSet *vs2) { if ((vs1 != NULL) (vs1-num != 0)) { slapi_valueset_done(vs1); } slapi_valueset_init(vs1); valueset_set_valueset(vs1,vs2); } Another option is to fix slapi_attr_set_valueset(): int slapi_attr_set_valueset(Slapi_Attr *a, const Slapi_ValueSet *vs) { if (a-a_present_values-num != 0) { lapi_valueset_done(a-a_present_values); } slapi_valueset_set_valueset( a-a_present_values, vs); return 0; } Other uses of slapi_valueset_set_valueset() are operating on newly created Slapi_ValueSet instances. -- / Alexander Bokovoy ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 0159-0160 Support ID views in compat tree
On 10/07/2014 12:39 PM, Alexander Bokovoy wrote: On Tue, 07 Oct 2014, Ludwig Krispenz wrote: On 10/07/2014 12:12 PM, Alexander Bokovoy wrote: On Tue, 07 Oct 2014, Ludwig Krispenz wrote: Hi Alex, slapi_attr_set_valueset() calls slapi_valueset_set_valueset(), but this does not free the existing values, Here is the problem, I cannot free original values as slapi_attr_get_valueset() returns a copy and I never can get access to the original Slapi_ValueSet. good point. I only can get access to the original Slapi_Value** array which means I could manipulate its content but that probably require resizing it and thus changing a pointer vs-va, to which I don't have access anyway. you should never do this, a valueset contains more data than va How can we fix this? the best thing would probably be to fix slapi_attr_set_valueset or adding a new api function which frees teh old valueset, but this would add a dependency to specific DS version What about something like this: void slapi_valueset_set_valueset(Slapi_ValueSet *vs1, const Slapi_ValueSet *vs2) { if ((vs1 != NULL) (vs1-num != 0)) { slapi_valueset_done(vs1); } slapi_valueset_init(vs1); valueset_set_valueset(vs1,vs2); } Another option is to fix slapi_attr_set_valueset(): int slapi_attr_set_valueset(Slapi_Attr *a, const Slapi_ValueSet *vs) { if (a-a_present_values-num != 0) { lapi_valueset_done(a-a_present_values); } slapi_valueset_set_valueset( a-a_present_values, vs); return 0; } Other uses of slapi_valueset_set_valueset() are operating on newly created Slapi_ValueSet instances. both will be ok (for me), but we have to commit it first and you depend on that version. a more clumsy way would be to get the values and then use slapi_entry_attr_replace_sv() which would free the attr and recreate it ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Switching to pytest
On 10/07/2014 04:37 AM, Petr Viktorin wrote: So in the test framework war I'm betting on this one. That is the info I was looking for. Thanks!! -- John ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 0159-0160 Support ID views in compat tree
On Tue, 07 Oct 2014, Ludwig Krispenz wrote: On 10/07/2014 12:39 PM, Alexander Bokovoy wrote: On Tue, 07 Oct 2014, Ludwig Krispenz wrote: On 10/07/2014 12:12 PM, Alexander Bokovoy wrote: On Tue, 07 Oct 2014, Ludwig Krispenz wrote: Hi Alex, slapi_attr_set_valueset() calls slapi_valueset_set_valueset(), but this does not free the existing values, Here is the problem, I cannot free original values as slapi_attr_get_valueset() returns a copy and I never can get access to the original Slapi_ValueSet. good point. I only can get access to the original Slapi_Value** array which means I could manipulate its content but that probably require resizing it and thus changing a pointer vs-va, to which I don't have access anyway. you should never do this, a valueset contains more data than va How can we fix this? the best thing would probably be to fix slapi_attr_set_valueset or adding a new api function which frees teh old valueset, but this would add a dependency to specific DS version What about something like this: void slapi_valueset_set_valueset(Slapi_ValueSet *vs1, const Slapi_ValueSet *vs2) { if ((vs1 != NULL) (vs1-num != 0)) { slapi_valueset_done(vs1); } slapi_valueset_init(vs1); valueset_set_valueset(vs1,vs2); } Another option is to fix slapi_attr_set_valueset(): int slapi_attr_set_valueset(Slapi_Attr *a, const Slapi_ValueSet *vs) { if (a-a_present_values-num != 0) { lapi_valueset_done(a-a_present_values); } slapi_valueset_set_valueset( a-a_present_values, vs); return 0; } Other uses of slapi_valueset_set_valueset() are operating on newly created Slapi_ValueSet instances. both will be ok (for me), but we have to commit it first and you depend on that version. a more clumsy way would be to get the values and then use slapi_entry_attr_replace_sv() which would free the attr and recreate it Actually, you pointed me to a better solution: /* Replace the attribute's value with the override */ result = slapi_entry_attr_exists(entry, override_type); if (result == 0) { result = slapi_attr_get_values(override_attr, override_valueset); result = slapi_entry_attr_delete_sv(entry, override_type, NULL); result = slapi_entry_add_valueset(entry, override_type, override_valueset); } slapi_entry_delete_values_sv(entry, attr, NULL) will clear whole attribute and then I can add the valueset. This, unfortunately, works via iterating over the whole valueset and adding values one by one but in our case it would be most likely a single element anyway. -- / Alexander Bokovoy ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 0159-0160 Support ID views in compat tree
On 10/07/2014 11:43 AM, Alexander Bokovoy wrote: On Tue, 07 Oct 2014, thierry bordaz wrote: A question about backend_search_filter_has_cn_uid. It checks if a filter components contains (uid|uidNumber|gidNumber|memberUid) and in this case returns SLAPI_FILTER_SCAN_STOP. This value will interrupt the filter rewriting. In addition, for each component it calls idview_process_filter_cb to override an attribute that needs to be override in the view. So I wonder if it will work for filter like: ((attribute_to_override=xxx) (uid=yyy)) but will stop to early for ((uid=yyy)(attribute_to_override=xxx)) We handle the requests which come from various NSS clients here (nss-pam-ldapd, nss_ldap, sssd, and similar). They don't do reordering like you say. Since all these searches never base queries on attributes other the ones we check (uid|uidNumber|gidNumber|memberUid), we don't need to replace other attributes here. I can see there could be a case where we get something like (actual query): ((gidNumber=1878600500) (|(objectClass=groupOfNames) (objectClass=posixGroup)) (cn=*)((gidNumber=*)(!(gidNumber=0 and we need to ignore '*' and '0' if they come before explicit gidNumber=id, I'll fix this one. Do you think we need to traverse the filter tree until it ends in all cases? In all filter types I've seen we are interested only in a single name and once we've got attributes filled in, we don't need to continue. In idview_process_filter_cb, there is a override of an attribute name into ipaAnchorUUID. I do not know what attribute required this override, but my concern is that depending on the order of the attributes in the filter, this override may not occur. I agree that if no existing filter is in this case it is not required to go until the end of the filter. Now if new clients starts using such filter pattern, it will also require a change in the plugin. From the performance point of view, it requires more processing at the filter level. unless filters are very long I would not expect a significant impact. Maybe for the replacement case of idview_process_filter_cb() we can have some flag in the filter processing config that forces to go through the fliter no matter what? Yes that could be an accelerator, when idview_process_filter_cb() has override an attribute into ipaAnchorUUID a flag could be reported to backend_search_filter_has_cn_uid to decide to stop the filter processing. To be honest I would prefer a systematic traverse of the filter until its end, even at the cost of additional processing. It would be easier to understand what the rewrite is doing and make it compatible with client enhancements. thanks thierry ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 0159-0160 Support ID views in compat tree
On 10/07/2014 01:32 PM, Alexander Bokovoy wrote: On Tue, 07 Oct 2014, Alexander Bokovoy wrote: On Tue, 07 Oct 2014, Ludwig Krispenz wrote: On 10/07/2014 12:39 PM, Alexander Bokovoy wrote: On Tue, 07 Oct 2014, Ludwig Krispenz wrote: On 10/07/2014 12:12 PM, Alexander Bokovoy wrote: On Tue, 07 Oct 2014, Ludwig Krispenz wrote: Hi Alex, slapi_attr_set_valueset() calls slapi_valueset_set_valueset(), but this does not free the existing values, Here is the problem, I cannot free original values as slapi_attr_get_valueset() returns a copy and I never can get access to the original Slapi_ValueSet. good point. I only can get access to the original Slapi_Value** array which means I could manipulate its content but that probably require resizing it and thus changing a pointer vs-va, to which I don't have access anyway. you should never do this, a valueset contains more data than va How can we fix this? the best thing would probably be to fix slapi_attr_set_valueset or adding a new api function which frees teh old valueset, but this would add a dependency to specific DS version What about something like this: void slapi_valueset_set_valueset(Slapi_ValueSet *vs1, const Slapi_ValueSet *vs2) { if ((vs1 != NULL) (vs1-num != 0)) { slapi_valueset_done(vs1); } slapi_valueset_init(vs1); valueset_set_valueset(vs1,vs2); } Another option is to fix slapi_attr_set_valueset(): int slapi_attr_set_valueset(Slapi_Attr *a, const Slapi_ValueSet *vs) { if (a-a_present_values-num != 0) { lapi_valueset_done(a-a_present_values); } slapi_valueset_set_valueset( a-a_present_values, vs); return 0; } Other uses of slapi_valueset_set_valueset() are operating on newly created Slapi_ValueSet instances. both will be ok (for me), but we have to commit it first and you depend on that version. a more clumsy way would be to get the values and then use slapi_entry_attr_replace_sv() which would free the attr and recreate it Actually, you pointed me to a better solution: /* Replace the attribute's value with the override */ result = slapi_entry_attr_exists(entry, override_type); if (result == 0) { result = slapi_attr_get_values(override_attr, override_valueset); result = slapi_entry_attr_delete_sv(entry, override_type, NULL); result = slapi_entry_add_valueset(entry, override_type, override_valueset); } slapi_entry_delete_values_sv(entry, attr, NULL) will clear whole attribute and then I can add the valueset. This, unfortunately, works via iterating over the whole valueset and adding values one by one but in our case it would be most likely a single element anyway. Pasted a lot of typos but fixed it already: result = slapi_entry_attr_exists(entry, override_type); if (result == 0) { result = slapi_attr_get_valueset(override_attr, override_valueset); result = slapi_entry_attr_delete(entry, override_type); result = slapi_entry_add_valueset(entry, override_type, override_valueset); } ok ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Dogtag lightweight sub-CAs; updated design
On Tue, 07 Oct 2014 13:47:05 +0200 Martin Kosek mko...@redhat.com wrote: On 10/07/2014 05:31 AM, Fraser Tweedale wrote: Hi all, The Dogtag lightweight sub-CAs design has undergone major revision and expansion ahead of beginning the implementation (I plan to begin later this week). This feature will provide an API for admins to create sub-CAs for separate security domains and augment the existing API so that certificates requests can be directed to a particular sub-CA. This feature will be used in FreeIPA for issuing user or service certificates for particular purposes (that will be rejected when use for other purposes). Please review the document and provide feedback. http://pki.fedoraproject.org/wiki/Lightweight_sub-CAs Feedback/suggestions for the REST API (that FreeIPA will use) and ACI considerations (e.g. is it appropriate to use the existing agent credential or should a separate credential or more fine-grained ACIs be used) are particularly encouraged. Cheers, Fraser Thanks for sharing the design! Couple initial comments: Creating sub-CAs Creation of sub-CAs at any time after the initial spawning of an CA instance is a requirement. Preferably, restart would not be needed, however, if needed, it must be able to be performed without manual intervention. I am all for having the operation in effect without requiring restart, especially given the change is in replicated tree. What do you mean by restart without manual operation? That Dogtag would restart itself when it detects that subCA would be added? Key generation and storage Are we referring to http://www.freeipa.org/page/V4/PKCS11_in_LDAP http://www.freeipa.org/page/V4/PKCS11_in_LDAP/Schema ? Contact people: Jan Cholasta, Petr Spacek ACI considerations Agent credential is used by FreeIPA web interface, all authorization is then done on python framework level. We can add more agents and then switch the used certificate, but I wonder how to use it in authorization decisions. Apache service will need to to have access to all these agents anyway. We really need to move to a separate service for agent access, the framework is supposed to not have any more power than the user that connects to it. By giving the framework direct access to credentials we fundamentally change the proposition and erode the security properties of the separation. We have discussed before a proxy process that pass in commands as they come from the framework but assumes agent identity only after checking how the framework authenticated to it (via GSSAPI). First we need to think how fine grained authorization we want to do. We need to associate a user to an agent credential via a group, so that we can assign the rights via roles. I think we will want to be able to for example say that user Foo can generate certificates in specified subCA. I am not sure it is a good way to go, it would also make such private key distribution on IPA replicas + renewal a challenge. I do not think we need to start with very fine grained permissions initially. Right now, we only have Virtual Operations concept to authorize different operations with Dogtag CA, but it does not distinguish between different CAs. We could add a new Virtual Operation for every subCA, but it looks clumsy. But the ACI-based mechanism and our permission system would still be the easiest way to go, IMHO, compared to utilizing PKI agents. We need to have a different agent certificate per role, and then in the proxy process associate the right agent certificate based on what the framework asks and internal checking that the user is indeed allowed to do so. The framework will select the 'role' to use based on the operation to be performed. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Dogtag lightweight sub-CAs; updated design
Simo Sorce wrote: On Tue, 07 Oct 2014 13:47:05 +0200 Martin Kosek mko...@redhat.com wrote: On 10/07/2014 05:31 AM, Fraser Tweedale wrote: Hi all, The Dogtag lightweight sub-CAs design has undergone major revision and expansion ahead of beginning the implementation (I plan to begin later this week). This feature will provide an API for admins to create sub-CAs for separate security domains and augment the existing API so that certificates requests can be directed to a particular sub-CA. This feature will be used in FreeIPA for issuing user or service certificates for particular purposes (that will be rejected when use for other purposes). Please review the document and provide feedback. http://pki.fedoraproject.org/wiki/Lightweight_sub-CAs Feedback/suggestions for the REST API (that FreeIPA will use) and ACI considerations (e.g. is it appropriate to use the existing agent credential or should a separate credential or more fine-grained ACIs be used) are particularly encouraged. Cheers, Fraser Thanks for sharing the design! Couple initial comments: Creating sub-CAs Creation of sub-CAs at any time after the initial spawning of an CA instance is a requirement. Preferably, restart would not be needed, however, if needed, it must be able to be performed without manual intervention. I am all for having the operation in effect without requiring restart, especially given the change is in replicated tree. What do you mean by restart without manual operation? That Dogtag would restart itself when it detects that subCA would be added? Key generation and storage Are we referring to http://www.freeipa.org/page/V4/PKCS11_in_LDAP http://www.freeipa.org/page/V4/PKCS11_in_LDAP/Schema ? Contact people: Jan Cholasta, Petr Spacek ACI considerations Agent credential is used by FreeIPA web interface, all authorization is then done on python framework level. We can add more agents and then switch the used certificate, but I wonder how to use it in authorization decisions. Apache service will need to to have access to all these agents anyway. We really need to move to a separate service for agent access, the framework is supposed to not have any more power than the user that connects to it. By giving the framework direct access to credentials we fundamentally change the proposition and erode the security properties of the separation. We have discussed before a proxy process that pass in commands as they come from the framework but assumes agent identity only after checking how the framework authenticated to it (via GSSAPI). First we need to think how fine grained authorization we want to do. We need to associate a user to an agent credential via a group, so that we can assign the rights via roles. I think we will want to be able to for example say that user Foo can generate certificates in specified subCA. I am not sure it is a good way to go, it would also make such private key distribution on IPA replicas + renewal a challenge. I do not think we need to start with very fine grained permissions initially. Right now, we only have Virtual Operations concept to authorize different operations with Dogtag CA, but it does not distinguish between different CAs. We could add a new Virtual Operation for every subCA, but it looks clumsy. But the ACI-based mechanism and our permission system would still be the easiest way to go, IMHO, compared to utilizing PKI agents. We need to have a different agent certificate per role, and then in the proxy process associate the right agent certificate based on what the framework asks and internal checking that the user is indeed allowed to do so. The framework will select the 'role' to use based on the operation to be performed. I totally agree in principle but this will add significant complexity to replication and renewal. Each agent cert will need to be tracked by certmonger and renewed automatically. The basic framework for that is in place and IIRC fairly generalized so this should be relatively simple, but we've had a few bumps in the road to renewal. What I think will be more challenging is dealing with distribution of additional agent certs to other masters. We handle it now via ipa-replica-manage. Given that it is a requirement to be able to generate sub-CAs post-install there needs to be some mechanism to share this certificate amongst the other IPA masters. On the bright side Fraser has already considered some of this, at least for sub-CA key distribution, but there are no no details fleshed out yet. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Dogtag lightweight sub-CAs; updated design
On Tue, 07 Oct 2014 09:29:33 -0400 Rob Crittenden rcrit...@redhat.com wrote: Simo Sorce wrote: On Tue, 07 Oct 2014 13:47:05 +0200 Martin Kosek mko...@redhat.com wrote: On 10/07/2014 05:31 AM, Fraser Tweedale wrote: Hi all, The Dogtag lightweight sub-CAs design has undergone major revision and expansion ahead of beginning the implementation (I plan to begin later this week). This feature will provide an API for admins to create sub-CAs for separate security domains and augment the existing API so that certificates requests can be directed to a particular sub-CA. This feature will be used in FreeIPA for issuing user or service certificates for particular purposes (that will be rejected when use for other purposes). Please review the document and provide feedback. http://pki.fedoraproject.org/wiki/Lightweight_sub-CAs Feedback/suggestions for the REST API (that FreeIPA will use) and ACI considerations (e.g. is it appropriate to use the existing agent credential or should a separate credential or more fine-grained ACIs be used) are particularly encouraged. Cheers, Fraser Thanks for sharing the design! Couple initial comments: Creating sub-CAs Creation of sub-CAs at any time after the initial spawning of an CA instance is a requirement. Preferably, restart would not be needed, however, if needed, it must be able to be performed without manual intervention. I am all for having the operation in effect without requiring restart, especially given the change is in replicated tree. What do you mean by restart without manual operation? That Dogtag would restart itself when it detects that subCA would be added? Key generation and storage Are we referring to http://www.freeipa.org/page/V4/PKCS11_in_LDAP http://www.freeipa.org/page/V4/PKCS11_in_LDAP/Schema ? Contact people: Jan Cholasta, Petr Spacek ACI considerations Agent credential is used by FreeIPA web interface, all authorization is then done on python framework level. We can add more agents and then switch the used certificate, but I wonder how to use it in authorization decisions. Apache service will need to to have access to all these agents anyway. We really need to move to a separate service for agent access, the framework is supposed to not have any more power than the user that connects to it. By giving the framework direct access to credentials we fundamentally change the proposition and erode the security properties of the separation. We have discussed before a proxy process that pass in commands as they come from the framework but assumes agent identity only after checking how the framework authenticated to it (via GSSAPI). First we need to think how fine grained authorization we want to do. We need to associate a user to an agent credential via a group, so that we can assign the rights via roles. I think we will want to be able to for example say that user Foo can generate certificates in specified subCA. I am not sure it is a good way to go, it would also make such private key distribution on IPA replicas + renewal a challenge. I do not think we need to start with very fine grained permissions initially. Right now, we only have Virtual Operations concept to authorize different operations with Dogtag CA, but it does not distinguish between different CAs. We could add a new Virtual Operation for every subCA, but it looks clumsy. But the ACI-based mechanism and our permission system would still be the easiest way to go, IMHO, compared to utilizing PKI agents. We need to have a different agent certificate per role, and then in the proxy process associate the right agent certificate based on what the framework asks and internal checking that the user is indeed allowed to do so. The framework will select the 'role' to use based on the operation to be performed. I totally agree in principle but this will add significant complexity to replication and renewal. We already have this issue with DNSSEC, so I think we can solve it the same way (storing keys in LDAP encrypted with a master key). Each agent cert will need to be tracked by certmonger and renewed automatically. The basic framework for that is in place and IIRC fairly generalized so this should be relatively simple, but we've had a few bumps in the road to renewal. Alternatively I think we can avoid this by having the proxy process store the certs in LDAP (encrypted with the current main agent cert) and renew them by calling out to certmonger if the certs are close to expiration. We can make it simpler than it is now. What I think will be more challenging is dealing with distribution of additional agent certs to other masters. We handle it now via ipa-replica-manage. See above :) Given that it is a requirement to be able to generate sub-CAs post-install there needs to be some
Re: [Freeipa-devel] [PATCH 0034] Missing requires on python-dns
Forgot to add patch. On Tue, Oct 7, 2014 at 7:58 AM, Gabe Alford redhatri...@gmail.com wrote: Hello, Fix for https://fedorahosted.org/freeipa/ticket/4613 Thanks, Gabe From 2506c1ac5c9d92a1817bfc4fb321c206107222a0 Mon Sep 17 00:00:00 2001 From: Gabe redhatri...@gmail.com Date: Tue, 7 Oct 2014 07:35:45 -0600 Subject: [PATCH] Missing requires on python-dns in spec file - Updated to required python-dns version 1.11 --- freeipa.spec.in |6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/freeipa.spec.in b/freeipa.spec.in index b0d4b06a033e07fd0b4d80ddbde6dffc3d0ace85..cd43e47e21f2c48cb0310a70051f3e78ea857bb6 100644 --- a/freeipa.spec.in +++ b/freeipa.spec.in @@ -61,7 +61,7 @@ BuildRequires: sssd = 1.9.2 BuildRequires: python-lxml BuildRequires: python-pyasn1 = 0.0.9a BuildRequires: python-qrcode-core = 5.0.0 -BuildRequires: python-dns +BuildRequires: python-dns = 1.11 BuildRequires: m2crypto BuildRequires: check BuildRequires: libsss_idmap-devel @@ -120,7 +120,7 @@ Requires: subscription-manager %endif Requires(preun): python systemd-units Requires(postun): python systemd-units -Requires: python-dns +Requires: python-dns = 1.11 Requires: zip Requires: policycoreutils = %{POLICYCOREUTILSVER} Requires: tar @@ -204,7 +204,7 @@ Requires: nss-tools Requires: bind-utils Requires: oddjob-mkhomedir Requires: python-krbV -Requires: python-dns +Requires: python-dns = 1.11 Requires: libsss_autofs Requires: autofs Requires: libnfsidmap -- 1.7.1 ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
[Freeipa-devel] [PATCH 0034] Missing requires on python-dns
Hello, Fix for https://fedorahosted.org/freeipa/ticket/4613 Thanks, Gabe ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 0159-0160 Support ID views in compat tree
On 10/01/2014 06:16 PM, Alexander Bokovoy wrote: Hi! Attached are patches to add support of FreeIPA ID views to Schema compatibility plugin (slapi-nis). There are two patches for FreeIPA and a separate patch for slapi-nis. Patches can be applied independently; if old slapi-nis is installed, it will simply work with new configuration but do nothing with respect to answering to requests using host-specific ID views. I included documentation on how slapi-nis ID views feature supposed to work, available in slapi-nis/doc/ipa/ipa-sch.txt. Any comments and fixes are welcome. There are no additional tests in slapi-nis to cover compat trees, we have multiple tests in FreeIPA for this purpose, will be run as part of FreeIPA CI effort. FreeIPA patches add ACIs for accessing ID view-applied entries over compat tree. They also include additional configuration; this configuration is needed to properly resolve ID view overrides when creating compat entries. A second FreeIPA patch adds support to override login shell. This part was missing from the original patchset by Tomas. For trusted AD users one needs patches to SSSD 1.12.2, made by Sumit Bose. There is also a regression (fixed by Sumit as well) that prevents authentication of AD users over PAM which affects authentication over compat tree. With the patch from Sumit authentication works again, both with ID view and without it. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel Hi Alexander, Just a minor remark about idview_get_overrides. It does an internal search of the views and keep the returned views into cbdata-overrides. Later the entries will be freed. Internal search may also return referral arrays (usually freed by slapi_free_search_results_internal). In case cn=views container contains referral entries you may need to free them. Also, I have a concern about performance. My understanding is that for each search, backend_search_cb allocates a new cbdata with uninitialized 'overrides'. So it will trigger the internal search idview_get_overrides. Most of the time those overrides rules have not been updated, so idview_get_overrides will retrieve the same entries. Wouldn't it be possible to 'cache' those entries (to prevent the internal search), and just clear those entries if a mod/add/del happen one them ? thanks thierry ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
[Freeipa-devel] [PATCH] 0020 Set IPA CA for freeipa certificates
https://fedorahosted.org/freeipa/ticket/4618 -- David Kupka From ab15f67ee35d29cd30b6b6d703a000c3cfe3188b Mon Sep 17 00:00:00 2001 From: David Kupka dku...@redhat.com Date: Tue, 7 Oct 2014 10:19:09 -0400 Subject: [PATCH] Set IPA CA for freeipa certificates. In previous versions (before moving certmonger.py to DBus) it was set and some tools and modules depends on it. For example: ipa-getcert uses this to filter freeipa certificates. https://fedorahosted.org/freeipa/ticket/4618 --- ipapython/certmonger.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/ipapython/certmonger.py b/ipapython/certmonger.py index b46d65b2fb0149aceee0864774e2ab76623e7730..0291d01b42aa6701b24dcb6905dcffab68a9ba63 100644 --- a/ipapython/certmonger.py +++ b/ipapython/certmonger.py @@ -289,6 +289,9 @@ def start_tracking(nickname, secdir, password_file=None, command=None): params['key-nickname'] = nickname params['key-database'] = os.path.abspath(secdir) params['key-storage'] = 'NSSDB' +ca_path = cm.obj_if.find_ca_by_nickname('IPA') +if ca_path: +params['ca'] = ca_path if command: params['cert-postsave-command'] = command if password_file: -- 1.9.3 ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 0159-0160 Support ID views in compat tree
On Tue, 07 Oct 2014, thierry bordaz wrote: On 10/01/2014 06:16 PM, Alexander Bokovoy wrote: Hi! Attached are patches to add support of FreeIPA ID views to Schema compatibility plugin (slapi-nis). There are two patches for FreeIPA and a separate patch for slapi-nis. Patches can be applied independently; if old slapi-nis is installed, it will simply work with new configuration but do nothing with respect to answering to requests using host-specific ID views. I included documentation on how slapi-nis ID views feature supposed to work, available in slapi-nis/doc/ipa/ipa-sch.txt. Any comments and fixes are welcome. There are no additional tests in slapi-nis to cover compat trees, we have multiple tests in FreeIPA for this purpose, will be run as part of FreeIPA CI effort. FreeIPA patches add ACIs for accessing ID view-applied entries over compat tree. They also include additional configuration; this configuration is needed to properly resolve ID view overrides when creating compat entries. A second FreeIPA patch adds support to override login shell. This part was missing from the original patchset by Tomas. For trusted AD users one needs patches to SSSD 1.12.2, made by Sumit Bose. There is also a regression (fixed by Sumit as well) that prevents authentication of AD users over PAM which affects authentication over compat tree. With the patch from Sumit authentication works again, both with ID view and without it. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel Hi Alexander, Just a minor remark about idview_get_overrides. It does an internal search of the views and keep the returned views into cbdata-overrides. Later the entries will be freed. Internal search may also return referral arrays (usually freed by slapi_free_search_results_internal). In case cn=views container contains referral entries you may need to free them. What would be a reason to return these referrals in IPA case? This code is IPA-specific and we don't have any referrals for the tree or backend we are dealing with. I agree it would be good to do in general but how that is relevant to the specific use case? Also, I have a concern about performance. My understanding is that for each search, backend_search_cb allocates a new cbdata with uninitialized 'overrides'. So it will trigger the internal search idview_get_overrides. Most of the time those overrides rules have not been updated, so idview_get_overrides will retrieve the same entries. Wouldn't it be possible to 'cache' those entries (to prevent the internal search), and just clear those entries if a mod/add/del happen one them ? I was thinking about using this approach but rejected it for several reasons, primary due to lack of development time. We can do a lot of refactoring on how slapi-nis currently uses RAM to store all compat entries at the same time; the current approach with ID overrides tries to not complicate that logic. We agreed with Simo to do performance optimizations at a later stage. -- / Alexander Bokovoy ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 0159-0160 Support ID views in compat tree
On 10/07/2014 05:00 PM, Alexander Bokovoy wrote: On Tue, 07 Oct 2014, thierry bordaz wrote: On 10/01/2014 06:16 PM, Alexander Bokovoy wrote: Hi! Attached are patches to add support of FreeIPA ID views to Schema compatibility plugin (slapi-nis). There are two patches for FreeIPA and a separate patch for slapi-nis. Patches can be applied independently; if old slapi-nis is installed, it will simply work with new configuration but do nothing with respect to answering to requests using host-specific ID views. I included documentation on how slapi-nis ID views feature supposed to work, available in slapi-nis/doc/ipa/ipa-sch.txt. Any comments and fixes are welcome. There are no additional tests in slapi-nis to cover compat trees, we have multiple tests in FreeIPA for this purpose, will be run as part of FreeIPA CI effort. FreeIPA patches add ACIs for accessing ID view-applied entries over compat tree. They also include additional configuration; this configuration is needed to properly resolve ID view overrides when creating compat entries. A second FreeIPA patch adds support to override login shell. This part was missing from the original patchset by Tomas. For trusted AD users one needs patches to SSSD 1.12.2, made by Sumit Bose. There is also a regression (fixed by Sumit as well) that prevents authentication of AD users over PAM which affects authentication over compat tree. With the patch from Sumit authentication works again, both with ID view and without it. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel Hi Alexander, Just a minor remark about idview_get_overrides. It does an internal search of the views and keep the returned views into cbdata-overrides. Later the entries will be freed. Internal search may also return referral arrays (usually freed by slapi_free_search_results_internal). In case cn=views container contains referral entries you may need to free them. What would be a reason to return these referrals in IPA case? This code is IPA-specific and we don't have any referrals for the tree or backend we are dealing with. I agree it would be good to do in general but how that is relevant to the specific use case? You are right, this is IPA-specific and we control the fact that there is no referral entries there. Also, I have a concern about performance. My understanding is that for each search, backend_search_cb allocates a new cbdata with uninitialized 'overrides'. So it will trigger the internal search idview_get_overrides. Most of the time those overrides rules have not been updated, so idview_get_overrides will retrieve the same entries. Wouldn't it be possible to 'cache' those entries (to prevent the internal search), and just clear those entries if a mod/add/del happen one them ? I was thinking about using this approach but rejected it for several reasons, primary due to lack of development time. We can do a lot of refactoring on how slapi-nis currently uses RAM to store all compat entries at the same time; the current approach with ID overrides tries to not complicate that logic. We agreed with Simo to do performance optimizations at a later stage. Ok. I fully agree. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 0159-0160 Support ID views in compat tree
Hi Alex, I have a question regarding cbdata.target. It is/was a reference to the pblock used to generate a new dn, but now in idview_replace_target_dn(cbdata.target,...) it can be newly allocated and should be freed, so I think there should be a return code indicating if it was allocated or not. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH 0064] Create ipa-otp-decrement 389DS plugin
Attached is the latest patch. I believe this includes all of our discussions up until this point. However, a few bits of additional information are needed. First, I have renamed the plugin to ipa-otp-counter. I believe all replay prevention work can land inside this plugin, so the name is appropriate. Second, I uncovered a bug in 389 which prevents me from validating the non-replication request in bepre. This is the reason for the additional betxnpre callback. If the upstream 389 bug is fixed, we can merge this check back into bepre. https://fedorahosted.org/389/ticket/47919 Third, I believe we are now handling replication correct. An error is never returned. When a replication would cause the counter to decrease, we remove all counter/watermark related mods from the operation. This will allow the replication to apply without decrementing the value. There is also a new bepost method which check to see if the replication was discarded (via CSN) while having a higher counter value. If so, we apply the higher counter value. Here is the scenario. Server X receives two quick authentications; replications A and B are sent to server Y. Before server Y can process server X's replications, an authentication is performed on server Y; replication C is sent to server X. The following logic holds true: * csnA csnB csnC * valueA = valueC, valueB valueC When server X receives replication C, ipa-otp-counter will strip out all counter mod operations; applying the update but not the lower value. The value of replication B is retained. This is the correct behavior. When server Y processes replications A and B, ipa-otp-counter will detect that a higher value has a lower CSN and will manually set the higher value (in bepost). This initiates replication D, which is sent to server X. Here is the logic: * csnA csnB csnC csnD * valueA = valueC, valueB = valueD, valueD valueC Server X receives replication D. D has the highest CSN. It has the same value as replication B (server X's current value). Because the values are the same, ipa-otp-counter will strip all counter mod operations. This reduces counter write contention which might become a problem in N-way replication when N2. On Fri, 2014-10-03 at 19:52 +0200, thierry bordaz wrote: Hello Nathaniel, An additional comment about the patch. When the new value is detected to be invalid, it is fixed by a repair operation (trigger_replication). I did test it and it is fine to update, with an internal operation, the same entry that is currently updated. Now if you apply the repair operation into a be_preop or a betxn_preop, when it returns from preop the txn of the current operation will overwrite the repaired value. An option is to register a bepost that checks the value from the original entry (SLAPI_ENTRY_PRE_OP) and post entry (SLAPI_ENTRY_POST_OP). Then this postop checks the orginal/final value and can trigger the repair op. This time being outside of the main operation txn, the repair op will be applied. thanks thierry On 09/29/2014 08:30 PM, Nathaniel McCallum wrote: On Mon, 2014-09-22 at 09:32 -0400, Simo Sorce wrote: On Sun, 21 Sep 2014 22:33:47 -0400 Nathaniel McCallum npmccal...@redhat.com wrote: Comments inline. + +#define ch_malloc(type) \ +(type*) slapi_ch_malloc(sizeof(type)) +#define ch_calloc(count, type) \ +(type*) slapi_ch_calloc(count, sizeof(type)) +#define ch_free(p) \ +slapi_ch_free((void**) (p)) please do not redefine slapi functions, it just makes it harder to know what you used. +typedef struct { +bool exists; +long long value; +} counter; please no typedefs of structures, use struct counter { ... }; and reference it as struct counter in the code. Btw, FWIW you could use a value of -1 to indicate non-existence of the counter value, given counters can only be positive, this would avoid the need for a struct. +static int +send_error(Slapi_PBlock *pb, int rc, char *template, ...) +{ +va_list ap; +int res; + +va_start(ap, template); +res = vsnprintf(NULL, 0, template, ap); +va_end(ap); + +if (res 0) { +char str[res + 1]; + +va_start(ap, template); +res = vsnprintf(str, sizeof(str), template, ap); +va_end(ap); + +if (res 0) +slapi_send_ldap_result(pb, rc, NULL, str, 0, NULL); +} + +if (res = 0) +slapi_send_ldap_result(pb, rc, NULL, NULL, 0, NULL); + +slapi_pblock_set(pb, SLAPI_RESULT_CODE, rc); +return rc; +} This function seem not really useful, you use send_error() only at the end of one single function where you could
[Freeipa-devel] [PATCH] 347 Fix CA cert validity check for CA-less and external CA installer options
Hi, the attached patch fixes https://fedorahosted.org/freeipa/ticket/4612. Honza -- Jan Cholasta From a8c5e11b428ea53119c091f565ca4c1f4ec32672 Mon Sep 17 00:00:00 2001 From: Jan Cholasta jchol...@redhat.com Date: Tue, 7 Oct 2014 18:16:53 +0200 Subject: [PATCH] Fix CA cert validity check for CA-less and external CA installer options https://fedorahosted.org/freeipa/ticket/4612 --- ipaserver/install/installutils.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ipaserver/install/installutils.py b/ipaserver/install/installutils.py index 1e010ed..0fb0ca4 100644 --- a/ipaserver/install/installutils.py +++ b/ipaserver/install/installutils.py @@ -799,6 +799,7 @@ def load_pkcs12(cert_files, key_password, key_nickname, ca_cert_files, (, .join(cert_files))) for nickname in trust_chain[1:]: +nssdb.trust_root_cert(nickname, ',,') try: nssdb.verify_ca_cert_validity(nickname) except ValueError, e: @@ -958,6 +959,7 @@ def load_external_cert(files, subject_base): (, .join(files))) for nickname in trust_chain: +nssdb.trust_root_cert(nickname, ',,') try: nssdb.verify_ca_cert_validity(nickname) except ValueError, e: -- 1.9.3 ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
[Freeipa-devel] [PATCH] 348 Remove misleading authorization error message in cert-request with --add
Hi, the attached patch fixes https://fedorahosted.org/freeipa/ticket/4540. The error message is now the generic ACI error message, e.g. Insufficient access: Insufficient 'add' privilege to add the entry 'krbprincipalname=something/somehost.example@example.com,cn=services,cn=accounts,dc=example,dc=com'. Honza -- Jan Cholasta From 4b812d600e9352d3dc37d896b0d4714f07b8cdc6 Mon Sep 17 00:00:00 2001 From: Jan Cholasta jchol...@redhat.com Date: Tue, 7 Oct 2014 18:38:20 +0200 Subject: [PATCH] Remove misleading authorization error message in cert-request with --add https://fedorahosted.org/freeipa/ticket/4540 --- ipalib/plugins/cert.py | 6 +- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/ipalib/plugins/cert.py b/ipalib/plugins/cert.py index e4918a4..679ac14 100644 --- a/ipalib/plugins/cert.py +++ b/ipalib/plugins/cert.py @@ -365,11 +365,7 @@ class cert_request(VirtualCommand): if not add: raise errors.NotFound(reason=_(The service principal for this request doesn't exist.)) -try: -service = api.Command['service_add'](principal, force=True) -except errors.ACIError: -raise errors.ACIError(info=_('You need to be a member of ' -'the serviceadmin role to add services')) +service = api.Command['service_add'](principal, force=True) service = service['result'] dn = service['dn'] -- 1.9.3 ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH 0064] Create ipa-otp-decrement 389DS plugin
On 10/07/2014 06:00 PM, Nathaniel McCallum wrote: Attached is the latest patch. I believe this includes all of our discussions up until this point. However, a few bits of additional information are needed. First, I have renamed the plugin to ipa-otp-counter. I believe all replay prevention work can land inside this plugin, so the name is appropriate. Second, I uncovered a bug in 389 which prevents me from validating the non-replication request in bepre. This is the reason for the additional betxnpre callback. If the upstream 389 bug is fixed, we can merge this check back into bepre. https://fedorahosted.org/389/ticket/47919 Hi Nathaniel, Just a rapid question about that dependency on https://fedorahosted.org/389/ticket/47919. Using txnpreop_mod you manage to workaround the DS issue. Do you need a fix for the DS issue in 1.3.2 or can it be fixed in a later version ? thanks thierry Third, I believe we are now handling replication correct. An error is never returned. When a replication would cause the counter to decrease, we remove all counter/watermark related mods from the operation. This will allow the replication to apply without decrementing the value. There is also a new bepost method which check to see if the replication was discarded (via CSN) while having a higher counter value. If so, we apply the higher counter value. Here is the scenario. Server X receives two quick authentications; replications A and B are sent to server Y. Before server Y can process server X's replications, an authentication is performed on server Y; replication C is sent to server X. The following logic holds true: * csnA csnB csnC * valueA = valueC, valueB valueC When server X receives replication C, ipa-otp-counter will strip out all counter mod operations; applying the update but not the lower value. The value of replication B is retained. This is the correct behavior. When server Y processes replications A and B, ipa-otp-counter will detect that a higher value has a lower CSN and will manually set the higher value (in bepost). This initiates replication D, which is sent to server X. Here is the logic: * csnA csnB csnC csnD * valueA = valueC, valueB = valueD, valueD valueC Server X receives replication D. D has the highest CSN. It has the same value as replication B (server X's current value). Because the values are the same, ipa-otp-counter will strip all counter mod operations. This reduces counter write contention which might become a problem in N-way replication when N2. On Fri, 2014-10-03 at 19:52 +0200, thierry bordaz wrote: Hello Nathaniel, An additional comment about the patch. When the new value is detected to be invalid, it is fixed by a repair operation (trigger_replication). I did test it and it is fine to update, with an internal operation, the same entry that is currently updated. Now if you apply the repair operation into a be_preop or a betxn_preop, when it returns from preop the txn of the current operation will overwrite the repaired value. An option is to register a bepost that checks the value from the original entry (SLAPI_ENTRY_PRE_OP) and post entry (SLAPI_ENTRY_POST_OP). Then this postop checks the orginal/final value and can trigger the repair op. This time being outside of the main operation txn, the repair op will be applied. thanks thierry On 09/29/2014 08:30 PM, Nathaniel McCallum wrote: On Mon, 2014-09-22 at 09:32 -0400, Simo Sorce wrote: On Sun, 21 Sep 2014 22:33:47 -0400 Nathaniel McCallum npmccal...@redhat.com wrote: Comments inline. + +#define ch_malloc(type) \ +(type*) slapi_ch_malloc(sizeof(type)) +#define ch_calloc(count, type) \ +(type*) slapi_ch_calloc(count, sizeof(type)) +#define ch_free(p) \ +slapi_ch_free((void**) (p)) please do not redefine slapi functions, it just makes it harder to know what you used. +typedef struct { +bool exists; +long long value; +} counter; please no typedefs of structures, use struct counter { ... }; and reference it as struct counter in the code. Btw, FWIW you could use a value of -1 to indicate non-existence of the counter value, given counters can only be positive, this would avoid the need for a struct. +static int +send_error(Slapi_PBlock *pb, int rc, char *template, ...) +{ +va_list ap; +int res; + +va_start(ap, template); +res = vsnprintf(NULL, 0, template, ap); +va_end(ap); + +if (res 0) { +char str[res + 1]; + +va_start(ap, template); +res = vsnprintf(str, sizeof(str), template, ap); +va_end(ap); + +if (res 0) +slapi_send_ldap_result(pb, rc, NULL, str, 0, NULL); +} + +if (res = 0) +slapi_send_ldap_result(pb, rc, NULL, NULL, 0, NULL); + +slapi_pblock_set(pb, SLAPI_RESULT_CODE,
Re: [Freeipa-devel] [PATCH 0034] Missing requires on python-dns
On 07/10/14 15:58, Gabe Alford wrote: Forgot to add patch. On Tue, Oct 7, 2014 at 7:58 AM, Gabe Alford redhatri...@gmail.com mailto:redhatri...@gmail.com wrote: Hello, Fix for https://fedorahosted.org/freeipa/ticket/4613 Thanks, Gabe ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel Thank you! I prefer to use python-dns = 1.11.1, there are some DNSSEC fixes which we may use in tests. Could you send updated patch please? -- Martin Basti ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH 0034] Missing requires on python-dns
Done. Update patch to use python-dns = 1.11.1 On Tue, Oct 7, 2014 at 11:26 AM, Martin Basti mba...@redhat.com wrote: On 07/10/14 15:58, Gabe Alford wrote: Forgot to add patch. On Tue, Oct 7, 2014 at 7:58 AM, Gabe Alford redhatri...@gmail.com wrote: Hello, Fix for https://fedorahosted.org/freeipa/ticket/4613 Thanks, Gabe ___ Freeipa-devel mailing listFreeipa-devel@redhat.comhttps://www.redhat.com/mailman/listinfo/freeipa-devel Thank you! I prefer to use python-dns = 1.11.1, there are some DNSSEC fixes which we may use in tests. Could you send updated patch please? -- Martin Basti From 574fecb17cea0c265c3fce508061a583327f8c35 Mon Sep 17 00:00:00 2001 From: Gabe redhatri...@gmail.com Date: Tue, 7 Oct 2014 11:14:32 -0600 Subject: [PATCH] Missing requires on python-dns in spec file - Updated to required python-dns version 1.11.1 --- freeipa.spec.in |6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/freeipa.spec.in b/freeipa.spec.in index b0d4b06a033e07fd0b4d80ddbde6dffc3d0ace85..b2dcea66ddf5b26fc4b069c9203f547225afa437 100644 --- a/freeipa.spec.in +++ b/freeipa.spec.in @@ -61,7 +61,7 @@ BuildRequires: sssd = 1.9.2 BuildRequires: python-lxml BuildRequires: python-pyasn1 = 0.0.9a BuildRequires: python-qrcode-core = 5.0.0 -BuildRequires: python-dns +BuildRequires: python-dns = 1.11.1 BuildRequires: m2crypto BuildRequires: check BuildRequires: libsss_idmap-devel @@ -120,7 +120,7 @@ Requires: subscription-manager %endif Requires(preun): python systemd-units Requires(postun): python systemd-units -Requires: python-dns +Requires: python-dns = 1.11.1 Requires: zip Requires: policycoreutils = %{POLICYCOREUTILSVER} Requires: tar @@ -204,7 +204,7 @@ Requires: nss-tools Requires: bind-utils Requires: oddjob-mkhomedir Requires: python-krbV -Requires: python-dns +Requires: python-dns = 1.11.1 Requires: libsss_autofs Requires: autofs Requires: libnfsidmap -- 1.7.1 ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH 0064] Create ipa-otp-decrement 389DS plugin
On Tue, 2014-10-07 at 18:54 +0200, thierry bordaz wrote: On 10/07/2014 06:00 PM, Nathaniel McCallum wrote: Attached is the latest patch. I believe this includes all of our discussions up until this point. However, a few bits of additional information are needed. First, I have renamed the plugin to ipa-otp-counter. I believe all replay prevention work can land inside this plugin, so the name is appropriate. Second, I uncovered a bug in 389 which prevents me from validating the non-replication request in bepre. This is the reason for the additional betxnpre callback. If the upstream 389 bug is fixed, we can merge this check back into bepre. https://fedorahosted.org/389/ticket/47919 Hi Nathaniel, Just a rapid question about that dependency on https://fedorahosted.org/389/ticket/47919. Using txnpreop_mod you manage to workaround the DS issue. Do you need a fix for the DS issue in 1.3.2 or can it be fixed in a later version ? I would strongly prefer a fix ASAP. thanks thierry Third, I believe we are now handling replication correct. An error is never returned. When a replication would cause the counter to decrease, we remove all counter/watermark related mods from the operation. This will allow the replication to apply without decrementing the value. There is also a new bepost method which check to see if the replication was discarded (via CSN) while having a higher counter value. If so, we apply the higher counter value. Here is the scenario. Server X receives two quick authentications; replications A and B are sent to server Y. Before server Y can process server X's replications, an authentication is performed on server Y; replication C is sent to server X. The following logic holds true: * csnA csnB csnC * valueA = valueC, valueB valueC When server X receives replication C, ipa-otp-counter will strip out all counter mod operations; applying the update but not the lower value. The value of replication B is retained. This is the correct behavior. When server Y processes replications A and B, ipa-otp-counter will detect that a higher value has a lower CSN and will manually set the higher value (in bepost). This initiates replication D, which is sent to server X. Here is the logic: * csnA csnB csnC csnD * valueA = valueC, valueB = valueD, valueD valueC Server X receives replication D. D has the highest CSN. It has the same value as replication B (server X's current value). Because the values are the same, ipa-otp-counter will strip all counter mod operations. This reduces counter write contention which might become a problem in N-way replication when N2. On Fri, 2014-10-03 at 19:52 +0200, thierry bordaz wrote: Hello Nathaniel, An additional comment about the patch. When the new value is detected to be invalid, it is fixed by a repair operation (trigger_replication). I did test it and it is fine to update, with an internal operation, the same entry that is currently updated. Now if you apply the repair operation into a be_preop or a betxn_preop, when it returns from preop the txn of the current operation will overwrite the repaired value. An option is to register a bepost that checks the value from the original entry (SLAPI_ENTRY_PRE_OP) and post entry (SLAPI_ENTRY_POST_OP). Then this postop checks the orginal/final value and can trigger the repair op. This time being outside of the main operation txn, the repair op will be applied. thanks thierry On 09/29/2014 08:30 PM, Nathaniel McCallum wrote: On Mon, 2014-09-22 at 09:32 -0400, Simo Sorce wrote: On Sun, 21 Sep 2014 22:33:47 -0400 Nathaniel McCallum npmccal...@redhat.com wrote: Comments inline. + +#define ch_malloc(type) \ +(type*) slapi_ch_malloc(sizeof(type)) +#define ch_calloc(count, type) \ +(type*) slapi_ch_calloc(count, sizeof(type)) +#define ch_free(p) \ +slapi_ch_free((void**) (p)) please do not redefine slapi functions, it just makes it harder to know what you used. +typedef struct { +bool exists; +long long value; +} counter; please no typedefs of structures, use struct counter { ... }; and reference it as struct counter in the code. Btw, FWIW you could use a value of -1 to indicate non-existence of the counter value, given counters can only be positive, this would avoid the need for a struct. +static int +send_error(Slapi_PBlock *pb, int rc, char *template, ...) +{ +va_list ap; +int res; + +
Re: [Freeipa-devel] [PATCH 0064] Create ipa-otp-decrement 389DS plugin
On 2014/10/07 10:48, Nathaniel McCallum wrote: On Tue, 2014-10-07 at 18:54 +0200, thierry bordaz wrote: On 10/07/2014 06:00 PM, Nathaniel McCallum wrote: Attached is the latest patch. I believe this includes all of our discussions up until this point. However, a few bits of additional information are needed. First, I have renamed the plugin to ipa-otp-counter. I believe all replay prevention work can land inside this plugin, so the name is appropriate. Second, I uncovered a bug in 389 which prevents me from validating the non-replication request in bepre. This is the reason for the additional betxnpre callback. If the upstream 389 bug is fixed, we can merge this check back into bepre. https://fedorahosted.org/389/ticket/47919 Hi Nathaniel, Just a rapid question about that dependency on https://fedorahosted.org/389/ticket/47919. Using txnpreop_mod you manage to workaround the DS issue. Do you need a fix for the DS issue in 1.3.2 or can it be fixed in a later version ? I would strongly prefer a fix ASAP. Thanks, Nathaniel, Do you need the fix just in 389-ds-base-1.3.3.x on F21 and newer? Or any other versions, e.g., 1.3.2 on F20, 1.3.3.1-x on RHEL-7.1, etc... ? --noriko thanks thierry Third, I believe we are now handling replication correct. An error is never returned. When a replication would cause the counter to decrease, we remove all counter/watermark related mods from the operation. This will allow the replication to apply without decrementing the value. There is also a new bepost method which check to see if the replication was discarded (via CSN) while having a higher counter value. If so, we apply the higher counter value. Here is the scenario. Server X receives two quick authentications; replications A and B are sent to server Y. Before server Y can process server X's replications, an authentication is performed on server Y; replication C is sent to server X. The following logic holds true: * csnA csnB csnC * valueA = valueC, valueB valueC When server X receives replication C, ipa-otp-counter will strip out all counter mod operations; applying the update but not the lower value. The value of replication B is retained. This is the correct behavior. When server Y processes replications A and B, ipa-otp-counter will detect that a higher value has a lower CSN and will manually set the higher value (in bepost). This initiates replication D, which is sent to server X. Here is the logic: * csnA csnB csnC csnD * valueA = valueC, valueB = valueD, valueD valueC Server X receives replication D. D has the highest CSN. It has the same value as replication B (server X's current value). Because the values are the same, ipa-otp-counter will strip all counter mod operations. This reduces counter write contention which might become a problem in N-way replication when N2. On Fri, 2014-10-03 at 19:52 +0200, thierry bordaz wrote: Hello Nathaniel, An additional comment about the patch. When the new value is detected to be invalid, it is fixed by a repair operation (trigger_replication). I did test it and it is fine to update, with an internal operation, the same entry that is currently updated. Now if you apply the repair operation into a be_preop or a betxn_preop, when it returns from preop the txn of the current operation will overwrite the repaired value. An option is to register a bepost that checks the value from the original entry (SLAPI_ENTRY_PRE_OP) and post entry (SLAPI_ENTRY_POST_OP). Then this postop checks the orginal/final value and can trigger the repair op. This time being outside of the main operation txn, the repair op will be applied. thanks thierry On 09/29/2014 08:30 PM, Nathaniel McCallum wrote: On Mon, 2014-09-22 at 09:32 -0400, Simo Sorce wrote: On Sun, 21 Sep 2014 22:33:47 -0400 Nathaniel McCallum npmccal...@redhat.com wrote: Comments inline. + +#define ch_malloc(type) \ +(type*) slapi_ch_malloc(sizeof(type)) +#define ch_calloc(count, type) \ +(type*) slapi_ch_calloc(count, sizeof(type)) +#define ch_free(p) \ +slapi_ch_free((void**) (p)) please do not redefine slapi functions, it just makes it harder to know what you used. +typedef struct { +bool exists; +long long value; +} counter; please no typedefs of structures, use struct counter { ... }; and reference it as struct counter in the code. Btw, FWIW you could use a value of -1 to indicate non-existence of the counter value, given counters can only be positive, this would avoid the need for a struct. +static int +send_error(Slapi_PBlock *pb, int rc, char *template, ...) +{ +va_list ap; +int res; + +va_start(ap, template); +res = vsnprintf(NULL, 0, template, ap); +va_end(ap); + +if (res 0) { +
Re: [Freeipa-devel] [PATCH 0064] Create ipa-otp-decrement 389DS plugin
On Tue, 2014-10-07 at 10:52 -0700, Noriko Hosoi wrote: On 2014/10/07 10:48, Nathaniel McCallum wrote: On Tue, 2014-10-07 at 18:54 +0200, thierry bordaz wrote: On 10/07/2014 06:00 PM, Nathaniel McCallum wrote: Attached is the latest patch. I believe this includes all of our discussions up until this point. However, a few bits of additional information are needed. First, I have renamed the plugin to ipa-otp-counter. I believe all replay prevention work can land inside this plugin, so the name is appropriate. Second, I uncovered a bug in 389 which prevents me from validating the non-replication request in bepre. This is the reason for the additional betxnpre callback. If the upstream 389 bug is fixed, we can merge this check back into bepre. https://fedorahosted.org/389/ticket/47919 Hi Nathaniel, Just a rapid question about that dependency on https://fedorahosted.org/389/ticket/47919. Using txnpreop_mod you manage to workaround the DS issue. Do you need a fix for the DS issue in 1.3.2 or can it be fixed in a later version ? I would strongly prefer a fix ASAP. Thanks, Nathaniel, Do you need the fix just in 389-ds-base-1.3.3.x on F21 and newer? Or any other versions, e.g., 1.3.2 on F20, 1.3.3.1-x on RHEL-7.1, etc... ? I think we are just shipping 4.1 on F21. Someone please correct me if I'm wrong. --noriko thanks thierry Third, I believe we are now handling replication correct. An error is never returned. When a replication would cause the counter to decrease, we remove all counter/watermark related mods from the operation. This will allow the replication to apply without decrementing the value. There is also a new bepost method which check to see if the replication was discarded (via CSN) while having a higher counter value. If so, we apply the higher counter value. Here is the scenario. Server X receives two quick authentications; replications A and B are sent to server Y. Before server Y can process server X's replications, an authentication is performed on server Y; replication C is sent to server X. The following logic holds true: * csnA csnB csnC * valueA = valueC, valueB valueC When server X receives replication C, ipa-otp-counter will strip out all counter mod operations; applying the update but not the lower value. The value of replication B is retained. This is the correct behavior. When server Y processes replications A and B, ipa-otp-counter will detect that a higher value has a lower CSN and will manually set the higher value (in bepost). This initiates replication D, which is sent to server X. Here is the logic: * csnA csnB csnC csnD * valueA = valueC, valueB = valueD, valueD valueC Server X receives replication D. D has the highest CSN. It has the same value as replication B (server X's current value). Because the values are the same, ipa-otp-counter will strip all counter mod operations. This reduces counter write contention which might become a problem in N-way replication when N2. On Fri, 2014-10-03 at 19:52 +0200, thierry bordaz wrote: Hello Nathaniel, An additional comment about the patch. When the new value is detected to be invalid, it is fixed by a repair operation (trigger_replication). I did test it and it is fine to update, with an internal operation, the same entry that is currently updated. Now if you apply the repair operation into a be_preop or a betxn_preop, when it returns from preop the txn of the current operation will overwrite the repaired value. An option is to register a bepost that checks the value from the original entry (SLAPI_ENTRY_PRE_OP) and post entry (SLAPI_ENTRY_POST_OP). Then this postop checks the orginal/final value and can trigger the repair op. This time being outside of the main operation txn, the repair op will be applied. thanks thierry On 09/29/2014 08:30 PM, Nathaniel McCallum wrote: On Mon, 2014-09-22 at 09:32 -0400, Simo Sorce wrote: On Sun, 21 Sep 2014 22:33:47 -0400 Nathaniel McCallum npmccal...@redhat.com wrote: Comments inline. + +#define ch_malloc(type) \ +(type*) slapi_ch_malloc(sizeof(type)) +#define ch_calloc(count, type) \ +(type*) slapi_ch_calloc(count, sizeof(type)) +#define ch_free(p) \ +slapi_ch_free((void**) (p)) please do not redefine slapi functions, it just makes it harder to know what you used. +typedef struct { +bool exists; +long long value; +} counter; please no typedefs of structures, use struct counter { ... }; and reference it as struct counter in the code. Btw, FWIW you could use a value of -1 to indicate non-existence of the
[Freeipa-devel] [PATCH] 0021 Fix example usage in ipa man page.
https://fedorahosted.org/freeipa/ticket/4587 -- David Kupka From 883e90237fbde1075d00990568cde18773e80611 Mon Sep 17 00:00:00 2001 From: David Kupka dku...@redhat.com Date: Wed, 8 Oct 2014 01:43:47 -0400 Subject: [PATCH] Fix example usage in ipa man page. https://fedorahosted.org/freeipa/ticket/4587 --- ipa.1 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ipa.1 b/ipa.1 index fc39fceaae5aa4c614ccaaa7e608f2326d926755..6acfcfd9f7ab580c9b086fa249903cdf10544cdb 100644 --- a/ipa.1 +++ b/ipa.1 @@ -149,7 +149,7 @@ Create a new user with username foo, first name foo and last name bar. \fBipa group\-add bar \-\-desc this is an example group Create a new group with name bar and description this is an example group. .TP -\fBipa group\-add\-member bar \-\-users=admin,foo\fR +\fBipa group\-add\-member bar \-\-users={admin,foo}\fR Add users admin and foo to the group bar. .TP \fBipa user\-show foo \-\-raw\fR -- 1.9.3 ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel