Re: [Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18
I have not opened a certmonger bug, but here is a patch to fix the relevant code in certmonger. Nalin, please review and commit. I tested by renewing one of the dogtag system certs (the ocsp signing cert) Ade On Mon, 2012-08-27 at 09:40 -0400, Rob Crittenden wrote: > Petr Viktorin wrote: > > On 08/27/2012 02:39 PM, Dmitri Pal wrote: > >> On 08/17/2012 12:06 PM, Rob Crittenden wrote: > >>> Ade Lee wrote: > On Fri, 2012-08-17 at 09:34 -0400, Ade Lee wrote: > > On Thu, 2012-08-16 at 18:45 +0200, Martin Kosek wrote: > >> On 08/16/2012 01:28 PM, Ade Lee wrote: > >>> Patch attached this time. I should know better than to do this in > >>> the > >>> middle of the night .. > >>> > >>> On Thu, 2012-08-16 at 09:12 +0200, Martin Kosek wrote: > On 08/16/2012 07:53 AM, Ade Lee wrote: > > On Wed, 2012-08-15 at 23:41 -0400, Ade Lee wrote: > >> On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote: > >>> On 08/15/2012 03:54 PM, Ade Lee wrote: > On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote: > > On 08/08/2012 10:05 PM, Ade Lee wrote: > >> Hi, > >> > >> Dogtag 10 is being released on f18, and has a number of > >> changes that > >> will affect IPA. In particular, the following changes will > >> affect > >> current IPA code. > >> > >> * The directory layout of the dogtag instance has changed. > >> Instead of > >> using separate tomcat instances to host different > >> subsystems, the > >> standard dogtag installation will allow one to install a > >> CA. KRA, OCSP > >> and TKS within the same instance. There have been > >> corresponding changes > >> in the directory layout, as well as the default instance name > >> (pki-tomcat instead of pki-ca), and startup daemon > >> (pki-tomcatd, instead > >> of pki-cad, pki-krad etc.) > >> > >> * The default instance will use only four ports (HTTPS, > >> HTTP, AJP and > >> tomcat shutdown port) rather than the 6 previously used. > >> The default > >> ports will be changed to the standard tomcat ports. As > >> these ports are > >> local to the ipa server machine, this should not cause too > >> much > >> disruption. > >> > >> * There is a new single step installer written in python. > >> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove. > >> > >> * Dogtag 10 runs on tomcat7 - with a new corresponding > >> version of > >> tomcatjss. > >> > >> The attached patch integrates all the above changes in IPA > >> installation > >> and maintenance code. Once the patch is applied, users > >> will be able to: > >> > >> 1. run ipa-server-install to completion on f18 with dogtag > >> 10. > >> 2. install a new replica on f18 on dogtag 10. > >> 3. upgrade an f17 machine with an existing IPA instance to > >> f18/ dogtag > >> 10 - and have that old-style dogtag instance continue to > >> run correctly. > >> This will require the installation of the latest version of > >> tomcatjss as > >> well as the installation of tomcat6. The old-style > >> instance will > >> continue to use tomcat6. > >> 4. in addition, the new cert renewal code has been patched > >> and should > >> continue to work. > >> > >> What is not yet completed / supported: > >> > >> 1. Installation with an external CA is not yet completed in > >> the new > >> installer. We plan to complete this soon. > >> > >> 2. There is some IPA upgrade code that has not yet been > >> touched > >> (install/tools/ipa-upgradeconfig). > >> > >> 3. A script needs to be written to allow admins to convert > >> their > >> old-style dogtag instances to new style instances, as well > >> as code to > >> periodically prompt admins to do this. > >> > >> 4. Installation of old-style instances using > >> pkicreate/pkisilent on > >> dogtag 10 will no longer be supported, and will be disabled > >> soon. > >> > >> 5. The pki-selinux policy has been updated to reflect > >> these changes, > >> but is still in flux. In fact, it is our intention to > >> place the dogtag > >
[Freeipa-devel] [PATCH] 1049 validate MLS value
Validate that the MLS value in a SELinux user map user is in the range
c0..c1023.
Existing tests validate correct values, empty values, I'm just adding a
high value test.
rob
>From 94cbfc3aeaed521319a9da371a44f3873a46657a Mon Sep 17 00:00:00 2001
From: Rob Crittenden
Date: Tue, 28 Aug 2012 17:14:28 -0400
Subject: [PATCH] Restrict the SELinux user map user MLS value to 0-1023
https://fedorahosted.org/freeipa/ticket/3001
---
ipalib/plugins/selinuxusermap.py| 3 ++-
tests/test_xmlrpc/test_selinuxusermap_plugin.py | 11 +++
2 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/ipalib/plugins/selinuxusermap.py b/ipalib/plugins/selinuxusermap.py
index e4cebc1e41bc315e285899e4279bcac26143ab2e..d793987aaa17c38fa5d6d83cb56038a7a5ebcd23 100644
--- a/ipalib/plugins/selinuxusermap.py
+++ b/ipalib/plugins/selinuxusermap.py
@@ -97,7 +97,8 @@ def validate_selinuxuser(ugettext, user):
return _('Invalid SELinux user name, only a-Z and _ are allowed')
if not mls or not regex_mls.match(mls):
return _('Invalid MLS value, must match s[0-15](-s[0-15])')
-if mcs and not regex_mcs.match(mcs):
+m = regex_mcs.match(mcs)
+if mcs and (not m or (m.group(3) and (int(m.group(3)) > 1023))):
return _('Invalid MCS value, must match c[0-1023].c[0-1023] and/or c[0-1023]-c[0-c0123]')
return None
diff --git a/tests/test_xmlrpc/test_selinuxusermap_plugin.py b/tests/test_xmlrpc/test_selinuxusermap_plugin.py
index 83260e8ab982da59343d84eba63c21e135ce61d4..aa2d0cac92f0944653be87be5df1fbe96470b3bc 100644
--- a/tests/test_xmlrpc/test_selinuxusermap_plugin.py
+++ b/tests/test_xmlrpc/test_selinuxusermap_plugin.py
@@ -645,6 +645,17 @@ class test_selinuxusermap(Declarative):
dict(
+desc='Create rule with invalid MLS xguest_u:s0:c0.c1028',
+command=(
+'selinuxusermap_add', [rule1], dict(ipaselinuxuser=u'xguest_u:s0-s0:c0.c1028')
+),
+expected=errors.ValidationError(name='selinuxuser',
+error=u'Invalid MCS value, must match c[0-1023].c[0-1023] ' +
+u'and/or c[0-1023]-c[0-c0123]'),
+),
+
+
+dict(
desc='Create rule with invalid user via setattr',
command=(
'selinuxusermap_mod', [rule1], dict(setattr=u'ipaselinuxuser=deny')
--
1.7.11.4
___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Paging in Web UI
On 8/28/2012 11:23 AM, Petr Vobornik wrote: Your possible solution does not address how many results are fetched (unless I misunderstood). > If paging is enabled it doesn't, but it expects, that admin will disable it for larger setups.For smaller setups it isn't of much an issue. If paging is disabled, the limit is server 'search size limit' or --sizelimit option supplied by Web UI. I'm not sure how per-user preferences are handled in browsers, but don't forget we now have session support in the server. Server session data is available for use. I was thinking about using browser local storage (basically key-value DB). It has a benefit over session, that it survives a browser restart but it should contain only non-sensitive data (other users may see it). You mean browser cookie? If you don't want to fetch every record to implement paging smartly in conjunction with it's performance issues why not do the query on the server, cache the results in the session, and have the RPC return the total number of results plus a subset of the result. Another RPC could retrieve the next/previous subset of results from the cached result in the session. I think most software do paging like this. I don't know the reasons for not doing it that way first time. My solution was counting with that we still don't want to do it. Endi do you know the reasons? Earlier sessions didn't exists, but it is doable without them too. Yes, at the time the sessions didn't exist and we needed a quick/temporary solution. I agree ideally this should be done on the server side, but how would the server maintain the paging information? The server can keep the search result (either just the pkey list or the entire entries) in memcached, but the result might be large and there might be multiple users accessing multiple search pages, so the total memory requirement could be large. If we don't want the server to keep the search result, the server could rerun the search and return only the requested page and discard most of the results each time the user change page. This may affect server performance. We can also use Simple Paged Results, but if I understood correctly it requires the httpd to maintain an open connection to the LDAP server for each user and for each page. I'm not sure memcached can be used to move the connection object among forked httpd processes. Also Simple Paged Results can only go forward, so no Prev button unless somebody keeps the results. Another possibility is to use VLV, but it seems to require open connection too and only works with a predefined filter. Here's the page I wrote about this: http://freeipa.org/page/IPAv3_Directory_Browsing Currently we're using Option #2 but I'm not sure if we use SPR between httpd and the LDAP server. But even with SPR the result is still bound by some limits. It looks you have to connect as Directory Manager to get the complete list of pkey list/entries. See the table in this page: http://directory.fedoraproject.org/wiki/Simple_Paged_Results_Design The current implementation (keeping the pkey list in the UI) is conceptually similar to the front-end approach described in the above page. I don't think there any need in JSON formatted data for pretty printing with indentation. Is it an accident or oversight we're pretty printing the JSON data in an RPC. For large data sets compression would be a win, but most of our RPC communication is not large. Compression has an overhead. With small data you'll use more cycles compressing and uncompressing than you would sending verbose but small data blobs. Compression should be an RPC option specified by either side of the connection and the receiver should be prepared to conditionally uncompress based on a flag value in the RPC. If you use server session data to support paging you may not need to introduce compression since you would only be passing one page of data at a time. User specified page size (without limitation) is an absolute necessity. I am frequently annoyed by web sites which either do not allow me to specify page size or constrain it to ridiculous hard coded limits such as 10, 20 or 30. +1 Agreed, but if we implement the single continuous page I described in the earlier email the page size won't be much of an issue anymore. -- Endi S. Dewata ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Paging in Web UI
On 8/28/2012 8:30 AM, Petr Vobornik wrote: I would like to point out a problem in Web UI related to paging and suggest a possible solution: Current implementation of paging in Web UI is not serving it purpose which should be to do operations faster and be able to find all users. How current implementation should work: 1) get all primary keys (pkeys) of certain object type (users, ...) 2) load first 20 of them 3) from 1) we know record count so we can choose other page (different subset of 20) How it actually works: * get up to 2000 keys, with more items a LDAP limit is reached and result is truncated * we don't know how many are there, paging doesn't work as it should We have a 'search limit' which can be specified by the user as a search parameter. It has a default value of 100 which can configured in IPA Server -> Configuration. There's also something like 'max result size' (I don't remember the actual name) of 2000 entries which is a global limit imposed by the LDAP backend. I'm not sure if the 'max result size' is configurable, but if I remember correctly if we want to get the entire entries in LDAP we'd need to use Simple Paged Results or VLV. Problem: Let's say that paging is working (no LDAP limit) and consider this: Our page size is 20 records. With 400 users it is 20 pages. With 2000 users (current search limit) it is 200 pages and so on. Personally I wouldn't like to go through 20 pages. 200 or more pages sounds like a punishment. In the current UI we could jump to a particular page directly, so it's not necessary to go through all pages. There is also a question if we even need to do it. I don't see a benefit of paging here. For smaller setup an increase of page size may help admin to see more users on a page more quickly (rolling mouse wheel is faster than click on next page button). We probably can make it look like a single continuous page, but we would still use paging to retrieve the entries. So initially it will show the entries of the first page, when you scroll down it will get the next page, but it will show the entries one-by-one by scrolling up the page. If none of the entries in the previous page are visible anymore the page can be removed from cache, but we might want to keep several pages in the cache. If we implement this, the interface will look the same regardless of the page size or whether we enable/disable paging. For large setups, size of the pkey list must also be taken on mind. For each record there is a pkey and dn (dn is not required but it would need some hacking in python code to remove it). At the moment the list size raises drastically because JSON response includes indenting by spaces and is not compressed (easy to fix, but still there). With 10 000 or more users this pkey and dn list is pretty big (slow load on slower networks -vpns). This list is also loaded on each page change (record-wise, can be improved - cached). Yes, ideally the pkey list should be cached and reloaded on new search or cache expiration. Or the server can do the query as John described, so the UI/CLI will just need to specify the page number. IMO with hundreds or thousands of users the most common use case is search by login or name or something. Paging is not required this case. It may be required if the result count is greater than size limit. In such case an option to temporary increase size limit or enable paging would be beneficial. For large setup we might need to use simple paged results or VLV, but they have limitations and require server support. Here's a page I wrote sometime ago: http://freeipa.org/page/IPAv3_Directory_Browsing Apart from this case, paging should be off. It will speed up page load because it executes only one request. Possible solution: 1) I suggest to introduce configuration options for paging. They can be global (default for all search pages) or individual for pages or Web UI's users. Per-user configuration can be stored in browser (to not pollute LDAP with application stuff). Global configuration on server in config plugin (minor polluting). Those options should be (per page and global): * enable paging * page size Note: when paging is disabled page size is basically --sizelimit option. Agreed. The actual number of entries would be different for each deployment and for each entity too, so it makes sense to use different settings to optimize the performance. If we implement the single continuous page above this setting will not affect the visual, but it will affect the performance. 2) On search pages we should have controls to enable/disable paging and to change sizelimit for this particular moment. If we keep the current UI users might want to save the setting (either on the server or on the client as you described) so they don't have to reconfigure it every time they log in. But if we implement the single continuous page above this might not be necessary anymore because users won'
Re: [Freeipa-devel] Paging in Web UI
On 08/28/2012 04:17 PM, John Dennis wrote: On 08/28/2012 09:30 AM, Petr Vobornik wrote: I would like to point out a problem in Web UI related to paging and suggest a possible solution: Current implementation of paging in Web UI is not serving it purpose which should be to do operations faster and be able to find all users. How current implementation should work: 1) get all primary keys (pkeys) of certain object type (users, ...) 2) load first 20 of them 3) from 1) we know record count so we can choose other page (different subset of 20) How it actually works: * get up to 2000 keys, with more items a LDAP limit is reached and result is truncated * we don't know how many are there, paging doesn't work as it should Problem: Let's say that paging is working (no LDAP limit) and consider this: Our page size is 20 records. With 400 users it is 20 pages. With 2000 users (current search limit) it is 200 pages and so on. Personally I wouldn't like to go through 20 pages. 200 or more pages sounds like a punishment. There is also a question if we even need to do it. I don't see a benefit of paging here. For smaller setup an increase of page size may help admin to see more users on a page more quickly (rolling mouse wheel is faster than click on next page button). For large setups, size of the pkey list must also be taken on mind. For each record there is a pkey and dn (dn is not required but it would need some hacking in python code to remove it). At the moment the list size raises drastically because JSON response includes indenting by spaces and is not compressed (easy to fix, but still there). With 10 000 or more users this pkey and dn list is pretty big (slow load on slower networks -vpns). This list is also loaded on each page change (record-wise, can be improved - cached). IMO with hundreds or thousands of users the most common use case is search by login or name or something. Paging is not required this case. It may be required if the result count is greater than size limit. In such case an option to temporary increase size limit or enable paging would be beneficial. Apart from this case, paging should be off. It will speed up page load because it executes only one request. Possible solution: 1) I suggest to introduce configuration options for paging. They can be global (default for all search pages) or individual for pages or Web UI's users. Per-user configuration can be stored in browser (to not pollute LDAP with application stuff). Global configuration on server in config plugin (minor polluting). Those options should be (per page and global): * enable paging * page size Note: when paging is disabled page size is basically --sizelimit option. 2) On search pages we should have controls to enable/disable paging and to change sizelimit for this particular moment. 3) Compress JSON responses (on httpd level) This way admin can configure default's for his deployment and user's can adjust for certain situations. Btw always including member_xx attributes in find or show commands is not good for search pages either but that's another topic. Comments are welcome. Your possible solution does not address how many results are fetched (unless I misunderstood). If paging is enabled it doesn't, but it expects, that admin will disable it for larger setups. For smaller setups it isn't of much an issue. If paging is disabled, the limit is server 'search size limit' or --sizelimit option supplied by Web UI. I'm not sure how per-user preferences are handled in browsers, but don't forget we now have session support in the server. Server session data is available for use. I was thinking about using browser local storage (basically key-value DB). It has a benefit over session, that it survives a browser restart but it should contain only non-sensitive data (other users may see it). If you don't want to fetch every record to implement paging smartly in conjunction with it's performance issues why not do the query on the server, cache the results in the session, and have the RPC return the total number of results plus a subset of the result. Another RPC could retrieve the next/previous subset of results from the cached result in the session. I think most software do paging like this. I don't know the reasons for not doing it that way first time. My solution was counting with that we still don't want to do it. Endi do you know the reasons? Earlier sessions didn't exists, but it is doable without them too. I don't think there any need in JSON formatted data for pretty printing with indentation. Is it an accident or oversight we're pretty printing the JSON data in an RPC. For large data sets compression would be a win, but most of our RPC communication is not large. Compression has an overhead. With small data you'll use more cycles compressing and uncompressing than you would sending verbose but small data blobs. Compression should be an RPC option specified by either side of the
[Freeipa-devel] [PATCH] 297 Update Contributors.txt file
Update list of active developers working on IPA. --- I also tried to add people from community contributing on our wiki pages on freeipa.org. ACKed by Rob off-list. Pushed to master, ipa-3-0. Martin >From 1d72a448a3cff60e7dab3ec25ee73583bd52d0fa Mon Sep 17 00:00:00 2001 From: Martin Kosek Date: Tue, 14 Aug 2012 16:30:52 +0200 Subject: [PATCH] Update Contributors.txt file Update list of active developers working on IPA. --- Contributors.txt | 22 ++ 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/Contributors.txt b/Contributors.txt index 94cf047266656af4d6bcefe7ac01749d59fe7c04..bd1ad22c00db5563587f8a8062b999bda720b094 100644 --- a/Contributors.txt +++ b/Contributors.txt @@ -5,28 +5,25 @@ The following people have contributed to the FreeIPA project. Developers: Jr Aquino + Tomas Babej Alexander Bokovoy Jan Cholasta Rob Crittenden Nalin Dahyabhai John Dennis - Jason DeRose Endi Dewata Jakub Hrozek Martin Kosek Nathan Kinder - Ondrej Hamada Rich Megginson - Martin Nagy Simo Sorce Petr Viktorin + Petr Vobornik Andrew Wnuk Adam Young - Jan Zeleny - Pavel Zůna Documentation: - David O'Brien + Ella Deon Lackey Testing: Jenny Galipeau @@ -41,20 +38,24 @@ Translators: Yuri Chornoivan Teguh DC Piotr DrÄ g + Jérôme Fenal Gundachandru Jake Li Andrew Martynov Sankarshan Mukhopadhyay Wiki, Solution and Idea Contributors: + James Hogarth + Dale Macartney Viji V Nair Ryan Thompson David Zeuthen Graphic Design and User Interaction Design: + Kyle Baker MáirÃn Duffy -Managment: +Management: Scott Haines Bob Lord Dmitri Pal @@ -66,13 +67,18 @@ Past and Occasional Contributors: Yuri Chornoivan Frank Cusack Don Davis + Jason DeRose Gunther Deschner Stephen Gallagher + Ondrej Hamada Ian Kumlien Karl MacMillan Jon McCann Kevin McCarthy Jim Meyering + Martin Nagy + David O'Brien Pete Rowley Andreas Schneider - + Jan Zeleny + Pavel Zuna -- 1.7.11.4 ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 202 Password policy paging with proper sorting
ACK. Some comments below. On 8/27/2012 10:51 AM, Petr Vobornik wrote: This patch adds option to disable sorting when paging. It allowed to enable paging in password policy with order of items untouched (they are sorted on server side by priority). Is the sorting we see in the UI and CLI identical to the actual sorting used to resolve the policy? I notice that they are sorted lexicographically instead of numerically. Also fixing issue when paging is disabled and command summary = null. It displayed 'null' in facet footer. https://fedorahosted.org/freeipa/ticket/2677 Note: personally I would left paging disabled in password policy page. I don't think it is beneficial. It slows things down and there would be hardly more than 100 policies. This should be confirmed by someone more familiar with actual deployments. I'd have no objections. -- Endi S. Dewata ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 201 Successful action notification
On 8/27/2012 5:57 AM, Petr Vobornik wrote: User was not notified about success of actions executed from action list, action panel or facet control bar. This patch adds IPA.notify_success(message) call. It creates a yellow notification area with supplied message in Web UI header in the middle of the green area (empty space of first level navigation). This area is displayed for 3s and then it fades out (800ms). It also fades out when it is clicked. This call is used(directly or indirectly) in: * search facets: delete, disable, enable actions * details facets: delete action * user details facet: reset password action * host details facet: unprovision, set OTP actions * service details facet: unprovision action * host and service details facet: request, revoke, restore certificates actions * group details facet: change to POSIX/external actions * dns zone details facet: add/remove permission actions https://fedorahosted.org/freeipa/ticket/2977 ACK. What do you think about creating a console/log page to show the action history? This way if the user misses the notification he still can check the logs. The logs will be stored in memory only and have size limit. The page can also be used to show the CLI commands of those actions. -- Endi S. Dewata ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 200 Fix issue which broke setup of Web UI unit tests
On 8/27/2012 5:52 AM, Petr Vobornik wrote: Fix issue which broke setup of Web UI unit tests Web UI itself wasn't negatively affected. Issue introduced in be144da672e0634f7aaeff69d662cbc4d11aff0f (#2897). https://fedorahosted.org/freeipa/ticket/2897 ACK. -- Endi S. Dewata ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 199 Permissions: select only applicable options on type change
Found a couple of issues with Undo: 1. Using the scenario described in the ticket, if I undo the Type back to User Group the Attributes aren't updated, it still shows the Service attributes. 2. After that, if I undo the Attributes it will show the originally selected attribute (description) but the attribute will appear at the end of Service attributes (not User Group attributes) and the attributes are not sorted. I also have some comments below. On 8/22/2012 7:17 AM, Petr Vobornik wrote: Problem: When a permission is edited, and Type switched, the attributes selected for previous Type are still selected, and update fails, if they are invalid for the new Type. But it should get deselected or not even listed if Type changes. Fix: When Type is changed, attribute list is refreshed and still applicable attributes are chosen. If Type is reverted back, previously chosen attributes are back as chosen. If attributes are extended outside Web UI by not listed attr, this attr is listed at the list end. To my understanding the list of ACI attributes are obtained from the LDAP schema, so if a new attribute is added to the object class the UI will know about it and show it in the attribute list. However, if the attribute is added using the extensibleObject the UI may not know about it because there's no schema change, is this what you meant? In that case the UI won't show a checkbox for the attribute, so we'd probably have to use the Filter or Subtree permission target that accepts arbitrary attributes. Ideally the server should support a generic LDAP ACI target which would accept any combination of LDAP filter, subtree, and attributes, but that probably depends on the actual needs. Note: If user makes change in attribute list before type change, this change is forgotten. https://fedorahosted.org/freeipa/ticket/2617 -- Endi S. Dewata ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 198 Revert change causing failure in test automation
On 8/22/2012 2:50 AM, Petr Vobornik wrote: Move of click handler in patch for #2834 causes failure of automation tests. This patch reverts the problematic part. It should not affect function of fix for #2834. https://fedorahosted.org/freeipa/ticket/3014 ACK. -- Endi S. Dewata ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 197 Fixed search in HBAC test
On 8/21/2012 9:03 AM, Petr Vobornik wrote: Search in HBAC test wasn't working because expired flag wasn't set. https://fedorahosted.org/freeipa/ticket/2931 Notes: HBAC facets don't have refresh button. They can be refreshed by changing filter and searching. If one search with same filter, it sets expired flag but it doesn't refresh (search) because page state isn't changed. It refreshes when one go to different facet and returns back. Is this behavior acceptable? Or should we a) don't set expired flag when searching with unchanged filter b) force refresh when searching with unchanged filter c) add refresh button along with a) I prefer leave it as is or b) Hmm.. the behavior is consistent with the rest of the UI, but I think when the user hits Enter or clicks the Search icon in the filter box he'd expect the UI to run a new search and return the latest result even if it's the same filter. I agree with option (b). A Refresh button can be added too, but it's optional. So this patch is ACKed, but feel free to make a future improvement. Maybe instead of having an 'expired flag' we could store an 'expiration date'. If the user returns to the page before it expires, the UI can show the old data. Otherwise the UI will rerun the search. -- Endi S. Dewata ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH 0009] Improves deletion of PTR records in ipa host-del.
On 08/28/2012 02:13 PM, Tomas Babej wrote: > On 08/28/2012 02:11 PM, Tomas Babej wrote: >> Hi, >> >> Command ipa host-del with --updatedns now can deal both with hosts >> which zones are in FQDN form with or without a trailing dot. >> >> https://fedorahosted.org/freeipa/ticket/2809 >> >> Tomas >> >> ___ >> Freeipa-devel mailing list >> Freeipa-devel@redhat.com >> https://www.redhat.com/mailman/listinfo/freeipa-devel > > And the patch itself, once again, cries forgotten on my hard drive. > > Tomas > ACK. Pushed to master, ipa-3-0. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Paging in Web UI
On 08/28/2012 09:30 AM, Petr Vobornik wrote: I would like to point out a problem in Web UI related to paging and suggest a possible solution: Current implementation of paging in Web UI is not serving it purpose which should be to do operations faster and be able to find all users. How current implementation should work: 1) get all primary keys (pkeys) of certain object type (users, ...) 2) load first 20 of them 3) from 1) we know record count so we can choose other page (different subset of 20) How it actually works: * get up to 2000 keys, with more items a LDAP limit is reached and result is truncated * we don't know how many are there, paging doesn't work as it should Problem: Let's say that paging is working (no LDAP limit) and consider this: Our page size is 20 records. With 400 users it is 20 pages. With 2000 users (current search limit) it is 200 pages and so on. Personally I wouldn't like to go through 20 pages. 200 or more pages sounds like a punishment. There is also a question if we even need to do it. I don't see a benefit of paging here. For smaller setup an increase of page size may help admin to see more users on a page more quickly (rolling mouse wheel is faster than click on next page button). For large setups, size of the pkey list must also be taken on mind. For each record there is a pkey and dn (dn is not required but it would need some hacking in python code to remove it). At the moment the list size raises drastically because JSON response includes indenting by spaces and is not compressed (easy to fix, but still there). With 10 000 or more users this pkey and dn list is pretty big (slow load on slower networks -vpns). This list is also loaded on each page change (record-wise, can be improved - cached). IMO with hundreds or thousands of users the most common use case is search by login or name or something. Paging is not required this case. It may be required if the result count is greater than size limit. In such case an option to temporary increase size limit or enable paging would be beneficial. Apart from this case, paging should be off. It will speed up page load because it executes only one request. Possible solution: 1) I suggest to introduce configuration options for paging. They can be global (default for all search pages) or individual for pages or Web UI's users. Per-user configuration can be stored in browser (to not pollute LDAP with application stuff). Global configuration on server in config plugin (minor polluting). Those options should be (per page and global): * enable paging * page size Note: when paging is disabled page size is basically --sizelimit option. 2) On search pages we should have controls to enable/disable paging and to change sizelimit for this particular moment. 3) Compress JSON responses (on httpd level) This way admin can configure default's for his deployment and user's can adjust for certain situations. Btw always including member_xx attributes in find or show commands is not good for search pages either but that's another topic. Comments are welcome. Your possible solution does not address how many results are fetched (unless I misunderstood). I'm not sure how per-user preferences are handled in browsers, but don't forget we now have session support in the server. Server session data is available for use. If you don't want to fetch every record to implement paging smartly in conjunction with it's performance issues why not do the query on the server, cache the results in the session, and have the RPC return the total number of results plus a subset of the result. Another RPC could retrieve the next/previous subset of results from the cached result in the session. I don't think there any need in JSON formatted data for pretty printing with indentation. Is it an accident or oversight we're pretty printing the JSON data in an RPC. For large data sets compression would be a win, but most of our RPC communication is not large. Compression has an overhead. With small data you'll use more cycles compressing and uncompressing than you would sending verbose but small data blobs. Compression should be an RPC option specified by either side of the connection and the receiver should be prepared to conditionally uncompress based on a flag value in the RPC. If you use server session data to support paging you may not need to introduce compression since you would only be passing one page of data at a time. User specified page size (without limitation) is an absolute necessity. I am frequently annoyed by web sites which either do not allow me to specify page size or constrain it to ridiculous hard coded limits such as 10, 20 or 30. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
[Freeipa-devel] Paging in Web UI
I would like to point out a problem in Web UI related to paging and suggest a possible solution: Current implementation of paging in Web UI is not serving it purpose which should be to do operations faster and be able to find all users. How current implementation should work: 1) get all primary keys (pkeys) of certain object type (users, ...) 2) load first 20 of them 3) from 1) we know record count so we can choose other page (different subset of 20) How it actually works: * get up to 2000 keys, with more items a LDAP limit is reached and result is truncated * we don't know how many are there, paging doesn't work as it should Problem: Let's say that paging is working (no LDAP limit) and consider this: Our page size is 20 records. With 400 users it is 20 pages. With 2000 users (current search limit) it is 200 pages and so on. Personally I wouldn't like to go through 20 pages. 200 or more pages sounds like a punishment. There is also a question if we even need to do it. I don't see a benefit of paging here. For smaller setup an increase of page size may help admin to see more users on a page more quickly (rolling mouse wheel is faster than click on next page button). For large setups, size of the pkey list must also be taken on mind. For each record there is a pkey and dn (dn is not required but it would need some hacking in python code to remove it). At the moment the list size raises drastically because JSON response includes indenting by spaces and is not compressed (easy to fix, but still there). With 10 000 or more users this pkey and dn list is pretty big (slow load on slower networks -vpns). This list is also loaded on each page change (record-wise, can be improved - cached). IMO with hundreds or thousands of users the most common use case is search by login or name or something. Paging is not required this case. It may be required if the result count is greater than size limit. In such case an option to temporary increase size limit or enable paging would be beneficial. Apart from this case, paging should be off. It will speed up page load because it executes only one request. Possible solution: 1) I suggest to introduce configuration options for paging. They can be global (default for all search pages) or individual for pages or Web UI's users. Per-user configuration can be stored in browser (to not pollute LDAP with application stuff). Global configuration on server in config plugin (minor polluting). Those options should be (per page and global): * enable paging * page size Note: when paging is disabled page size is basically --sizelimit option. 2) On search pages we should have controls to enable/disable paging and to change sizelimit for this particular moment. 3) Compress JSON responses (on httpd level) This way admin can configure default's for his deployment and user's can adjust for certain situations. Btw always including member_xx attributes in find or show commands is not good for search pages either but that's another topic. Comments are welcome. Regards -- Petr Vobornik ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH 0009] Improves deletion of PTR records in ipa host-del.
On 08/28/2012 02:11 PM, Tomas Babej wrote:
Hi,
Command ipa host-del with --updatedns now can deal both with hosts
which zones are in FQDN form with or without a trailing dot.
https://fedorahosted.org/freeipa/ticket/2809
Tomas
___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel
And the patch itself, once again, cries forgotten on my hard drive.
Tomas
>From a86942a0c4253272678cdf50cacf36d20e654a80 Mon Sep 17 00:00:00 2001
From: Tomas Babej
Date: Tue, 28 Aug 2012 07:43:20 -0400
Subject: [PATCH] Improves deletion of PTR records in ipa host-del.
Command ipa host-del with --updatedns now can deal both with hosts
which zones are in FQDN form with or without a trailing dot.
https://fedorahosted.org/freeipa/ticket/2809
---
ipalib/plugins/host.py | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/ipalib/plugins/host.py b/ipalib/plugins/host.py
index a417ba0f7260522661c1045077af1462ff3c410e..43381e837b74bc68fe2d48c5e8757062407312a9 100644
--- a/ipalib/plugins/host.py
+++ b/ipalib/plugins/host.py
@@ -108,7 +108,12 @@ def remove_fwd_ptr(ipaddr, host, domain, recordtype):
api.log.debug('deleting ipaddr %s' % ipaddr)
try:
revzone, revname = get_reverse_zone(ipaddr)
-delkw = { 'ptrrecord' : "%s.%s." % (host, domain) }
+
+# in case domain is in FQDN form with a trailing dot, we needn't add
+# another one, in case it has no trailing dot, dnsrecord-del will
+# normalize the entry
+delkw = { 'ptrrecord' : "%s.%s" % (host, domain) }
+
api.Command['dnsrecord_del'](revzone, revname, **delkw)
except errors.NotFound:
pass
--
1.7.11.4
___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel
[Freeipa-devel] [PATCH 0009] Improves deletion of PTR records in ipa host-del.
Hi, Command ipa host-del with --updatedns now can deal both with hosts which zones are in FQDN form with or without a trailing dot. https://fedorahosted.org/freeipa/ticket/2809 Tomas ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
[Freeipa-devel] [PATCH 0054] Allow BIND to start if LDAP connection times out
Hello,
this patch allows BIND to start if LDAP connection times out. BIND will
reconnect in same way as after "connection refused" errors.
The patch closes https://fedorahosted.org/bind-dyndb-ldap/ticket/84 .
--
Petr^2 Spacek
From eaa35060fc47c1422ca7b577fe0096aadd2f8c0a Mon Sep 17 00:00:00 2001
From: Petr Spacek
Date: Tue, 28 Aug 2012 13:54:51 +0200
Subject: [PATCH] Allow BIND to start if LDAP connection times out.
https://fedorahosted.org/bind-dyndb-ldap/ticket/84
Signed-off-by: Petr Spacek
---
src/ldap_helper.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/ldap_helper.c b/src/ldap_helper.c
index da083d2e65032e650cfbbeb863262e0141403407..d533d1985c9fb8f56f333a99208d3f60f0b4c5bf 100644
--- a/src/ldap_helper.c
+++ b/src/ldap_helper.c
@@ -2797,7 +2797,7 @@ ldap_pool_connect(ldap_pool_t *pool, ldap_instance_t *ldap_inst)
ldap_conn = NULL;
CHECK(new_ldap_connection(pool, &ldap_conn));
result = ldap_connect(ldap_inst, ldap_conn, ISC_FALSE);
- if (result == ISC_R_NOTCONNECTED) {
+ if (result == ISC_R_NOTCONNECTED || result == ISC_R_TIMEDOUT) {
/* LDAP server is down which can happen, continue */
result = ISC_R_SUCCESS;
} else if (result != ISC_R_SUCCESS) {
--
1.7.11.2
___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH 0049] Fix two memory leaks in persistent search
On 08/22/2012 03:43 PM, Adam Tkac wrote: On Tue, Aug 14, 2012 at 02:32:55PM +0200, Petr Spacek wrote: Hello, This patch fixes two memory leaks in persistent search. Ack. From 892f1d5c59a97cdad7a2807ecd172488605ab181 Mon Sep 17 00:00:00 2001 From: Petr Spacek Date: Tue, 14 Aug 2012 12:38:43 +0200 Subject: [PATCH] Fix two memory leaks in persistent search. Signed-off-by: Petr Spacek --- src/ldap_helper.c | 45 - 1 file changed, 12 insertions(+), 33 deletions(-) Pushed to master: https://fedorahosted.org/bind-dyndb-ldap/changeset/6114ae200d8698dff421cf0936f8dd8262247710 -- Petr^2 Spacek ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH 0048] Lower minimal connection count for persistent search.
On 08/22/2012 03:37 PM, Adam Tkac wrote: On Mon, Aug 13, 2012 at 03:31:44PM +0200, Petr Spacek wrote: Hello, As result of better connection management it is possible to run persistent search with smaller connection pool. Attached patch updates built-in configuration checks with new lower bound. It closes patch series for https://fedorahosted.org/bind-dyndb-ldap/ticket/68: Avoid manual connection management outside ldap_query() Ack From 786531c5806331dc0486ed1d877563b00d9219da Mon Sep 17 00:00:00 2001 From: Petr Spacek Date: Mon, 13 Aug 2012 15:24:48 +0200 Subject: [PATCH] Lower minimal connection count for persistent search. As result of better connection management is possible to run persistent search with smaller connection pool. Signed-off-by: Petr Spacek --- src/ldap_helper.c | 14 -- 1 file changed, 4 insertions(+), 10 deletions(-) Pushed to master: https://fedorahosted.org/bind-dyndb-ldap/changeset/ea66d8daaa0feee14db3e5c8eefa6d9b6a45984c -- Petr^2 Spacek ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH 0047] Avoid manual connection management outside ldap_query()
On 08/28/2012 09:57 AM, Adam Tkac wrote: On Tue, Aug 28, 2012 at 08:51:31AM +0200, Petr Spacek wrote: On 08/22/2012 03:35 PM, Adam Tkac wrote: On Mon, Aug 13, 2012 at 03:15:52PM +0200, Petr Spacek wrote: Hello, this patch improves connection management in bind-dyndb-ldap and closes https://fedorahosted.org/bind-dyndb-ldap/ticket/68 . It should prevent all deadlocks on connection pool in future. Ack, just check my pedantic comments below, please. I partially disagree with one comment below. Amended patch is attached. ... Well, zone_dn and rdata checks are really redundant. First use of ldap_inst is in ldap_inst->mctx, so check is AFAIK necessary. I left REQUIRE(ldap_inst != NULL); in attached patch, other REQUIREs were deleted. You are right, sorry for false alarm. Ack. Regards, Adam Pushed to master: https://fedorahosted.org/bind-dyndb-ldap/changeset/8d77eac3ac3d601c043566c83ebaca581e7591fe -- Petr^2 Spacek ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH 0047] Avoid manual connection management outside ldap_query()
On Tue, Aug 28, 2012 at 08:51:31AM +0200, Petr Spacek wrote: > On 08/22/2012 03:35 PM, Adam Tkac wrote: > >On Mon, Aug 13, 2012 at 03:15:52PM +0200, Petr Spacek wrote: > >>Hello, > >> > >>this patch improves connection management in bind-dyndb-ldap and closes > >>https://fedorahosted.org/bind-dyndb-ldap/ticket/68 . > >> > >>It should prevent all deadlocks on connection pool in future. > > > >Ack, just check my pedantic comments below, please. > > I partially disagree with one comment below. Amended patch is attached. > ... > Well, zone_dn and rdata checks are really redundant. First use of > ldap_inst is in ldap_inst->mctx, so check is AFAIK necessary. > > I left REQUIRE(ldap_inst != NULL); in attached patch, other REQUIREs were > deleted. You are right, sorry for false alarm. Ack. Regards, Adam -- Adam Tkac, Red Hat, Inc. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
