Re: [Freeipa-devel] [PATCH] 884 migration context and logging
On Mon, 2011-10-03 at 16:44 -0400, Rob Crittenden wrote: > Martin Kosek wrote: > > On Mon, 2011-09-26 at 22:24 -0400, Rob Crittenden wrote: > >> We can't assume that there will be only one naming context. Look at each > >> one until we find an IPA one. > >> > >> Add logging so you can know that a migration attempt fails and why. > >> > >> rob > > > > Looks good, its just difficult to set up a proper environment for > > reproduction. So far, I found just this problem: > > > > [Tue Sep 27 10:30:39 2011] [error] [client 10.34.25.52] mod_wsgi > > (pid=32705): Exception occurred processing WSGI script > > '/usr/share/ipa/migration/migration.py'. > > [Tue Sep 27 10:30:40 2011] [error] [client 10.34.25.52] Traceback (most > > recent call last): > > [Tue Sep 27 10:30:40 2011] [error] [client 10.34.25.52] File > > "/usr/share/ipa/migration/migration.py", line 127, in application > > [Tue Sep 27 10:30:40 2011] [error] [client 10.34.25.52] > > bind(form_data['username'].value, form_data['password'].value) > > [Tue Sep 27 10:30:40 2011] [error] [client 10.34.25.52] File > > "/usr/share/ipa/migration/migration.py", line 107, in bind > > [Tue Sep 27 10:30:40 2011] [error] [client 10.34.25.52] > > logging.error('migration bind failed: %s' % convert_exception(e)) > > > > Martin > > > > Just missed saving the exception as a variable, should work now. > > rob Works fine, tested on multiple-suffix LDAP server. We should be also fine when anonymous access is not allowed (Simo was dealing with this in ipa-client-install in #1881) since migration.py binds via socket. I have just one suggestion - instead of searching for correct naming context on your own, you may want to use a function get_ipa_basedn() I implemented for ipa-client-install (#1868). This will do all the checks and return you just the IPA baseDN: https://fedorahosted.org/freeipa/changeset/00cffce6c2ba0121188326535d6c9cd244a4ae5b Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] #1794 - Speed up replica setup
On Oct 3, 2011, at 3:17 PM, Simo Sorce wrote: > On Mon, 2011-10-03 at 16:20 -0400, Simo Sorce wrote: >> Newer 389ds servers have a new option to have a different set of >> filtered attributes from normal replication. >> >> This has been added in order to allow DS to replicate memberof >> attributes only during a total update so that we do not need to run a >> fixup memberof task on a replica at install time. >> This task is quite inefficient for big database and can take a long >> time. By replicating memberof while the DB is locked we are guaranteed >> the memberof list is consistent so we do not need a fixup. >> >> This patch allows to enable this feature dynamically. If the server does >> not yet support the new option it falls back to the previous behavior. >> >> Fixes: https://fedorahosted.org/freeipa/ticket/1794 >> >> I am sending the patch but it has been jointly developed at various >> stages by Nathan, JR, and me. >> >> Simo. > > After some thinking I found out that we cannot commit this patch until > the memberof plugin is converted to use the new transaction interfaces > for plugins, as otherwise it is possible to run into race conditions > where the member/memberof relations are not settled if a new replica is > installed while member attributes are being changed. > > Granted the race is quite small and unlikely but real. > So please test and ack it, but we need to defer pushing to stable > branches until ds copes. > I think it is ok to push to master for testing, DS should have the > necessary support by the time we make another stable release from master > and in our test environments I am sure we will never hit the race. Do we know which 389-ds-base incorporates the new option? I would like to test and ack, but I'm not sure if I have a fixed 389-ds-base installed. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] #1794 - Speed up replica setup
On Mon, 2011-10-03 at 16:20 -0400, Simo Sorce wrote: > Newer 389ds servers have a new option to have a different set of > filtered attributes from normal replication. > > This has been added in order to allow DS to replicate memberof > attributes only during a total update so that we do not need to run a > fixup memberof task on a replica at install time. > This task is quite inefficient for big database and can take a long > time. By replicating memberof while the DB is locked we are guaranteed > the memberof list is consistent so we do not need a fixup. > > This patch allows to enable this feature dynamically. If the server does > not yet support the new option it falls back to the previous behavior. > > Fixes: https://fedorahosted.org/freeipa/ticket/1794 > > I am sending the patch but it has been jointly developed at various > stages by Nathan, JR, and me. > > Simo. After some thinking I found out that we cannot commit this patch until the memberof plugin is converted to use the new transaction interfaces for plugins, as otherwise it is possible to run into race conditions where the member/memberof relations are not settled if a new replica is installed while member attributes are being changed. Granted the race is quite small and unlikely but real. So please test and ack it, but we need to defer pushing to stable branches until ds copes. I think it is ok to push to master for testing, DS should have the necessary support by the time we make another stable release from master and in our test environments I am sure we will never hit the race. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] #1794 - Speed up replica setup
On 10/03/2011 04:39 PM, JR Aquino wrote: On Oct 3, 2011, at 3:17 PM, Simo Sorce wrote: On Mon, 2011-10-03 at 16:20 -0400, Simo Sorce wrote: Newer 389ds servers have a new option to have a different set of filtered attributes from normal replication. This has been added in order to allow DS to replicate memberof attributes only during a total update so that we do not need to run a fixup memberof task on a replica at install time. This task is quite inefficient for big database and can take a long time. By replicating memberof while the DB is locked we are guaranteed the memberof list is consistent so we do not need a fixup. This patch allows to enable this feature dynamically. If the server does not yet support the new option it falls back to the previous behavior. Fixes: https://fedorahosted.org/freeipa/ticket/1794 I am sending the patch but it has been jointly developed at various stages by Nathan, JR, and me. Simo. After some thinking I found out that we cannot commit this patch until the memberof plugin is converted to use the new transaction interfaces for plugins, as otherwise it is possible to run into race conditions where the member/memberof relations are not settled if a new replica is installed while member attributes are being changed. Granted the race is quite small and unlikely but real. So please test and ack it, but we need to defer pushing to stable branches until ds copes. I think it is ok to push to master for testing, DS should have the necessary support by the time we make another stable release from master and in our test environments I am sure we will never hit the race. Do we know which 389-ds-base incorporates the new option? I would like to test and ack, but I'm not sure if I have a fixed 389-ds-base installed. 1.2.10.a1 - in updates-testing ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 884 migration context and logging
Martin Kosek wrote: On Mon, 2011-09-26 at 22:24 -0400, Rob Crittenden wrote: We can't assume that there will be only one naming context. Look at each one until we find an IPA one. Add logging so you can know that a migration attempt fails and why. rob Looks good, its just difficult to set up a proper environment for reproduction. So far, I found just this problem: [Tue Sep 27 10:30:39 2011] [error] [client 10.34.25.52] mod_wsgi (pid=32705): Exception occurred processing WSGI script '/usr/share/ipa/migration/migration.py'. [Tue Sep 27 10:30:40 2011] [error] [client 10.34.25.52] Traceback (most recent call last): [Tue Sep 27 10:30:40 2011] [error] [client 10.34.25.52] File "/usr/share/ipa/migration/migration.py", line 127, in application [Tue Sep 27 10:30:40 2011] [error] [client 10.34.25.52] bind(form_data['username'].value, form_data['password'].value) [Tue Sep 27 10:30:40 2011] [error] [client 10.34.25.52] File "/usr/share/ipa/migration/migration.py", line 107, in bind [Tue Sep 27 10:30:40 2011] [error] [client 10.34.25.52] logging.error('migration bind failed: %s' % convert_exception(e)) Martin Just missed saving the exception as a variable, should work now. rob >From bf75398c01539299817de6208a3b9854c9178a6e Mon Sep 17 00:00:00 2001 From: Rob Crittenden Date: Mon, 26 Sep 2011 22:19:57 -0400 Subject: [PATCH] Migration: don't assume there is only one naming context, add logging. We can't assume that there will be only one naming context. Look at each one until we find an IPA one. Add logging so you can know that a migration attempt fails and why. https://fedorahosted.org/freeipa/ticket/1834 https://fedorahosted.org/freeipa/ticket/1835 --- install/migration/invalid.html |2 +- install/migration/migration.py | 56 2 files changed, 46 insertions(+), 12 deletions(-) diff --git a/install/migration/invalid.html b/install/migration/invalid.html index a641d1a..91de79f 100644 --- a/install/migration/invalid.html +++ b/install/migration/invalid.html @@ -35,7 +35,7 @@ If the problem persists, contact your administrator. - + Username: diff --git a/install/migration/migration.py b/install/migration/migration.py index ed6ade9..4ac891a 100644 --- a/install/migration/migration.py +++ b/install/migration/migration.py @@ -25,10 +25,24 @@ import errno import glob import ldap import wsgiref +import logging BASE_DN = '' LDAP_URI = 'ldaps://localhost:636' +def convert_exception(error): +""" +Convert an LDAP exception into something more readable. +""" +if not isinstance(error, ldap.TIMEOUT): +desc = error.args[0]['desc'].strip() +info = error.args[0].get('info', '').strip() +else: +desc = '' +info = '' + +return '%s (%s)' % (desc, info) + def wsgi_redirect(start_response, loc): start_response('302 Found', [('Location', loc)]) return [] @@ -44,6 +58,8 @@ def get_base_dn(): """ Retrieve LDAP server base DN. """ +global BASE_DN + if BASE_DN: return BASE_DN try: @@ -52,31 +68,50 @@ def get_base_dn(): entries = conn.search_ext_s( '', scope=ldap.SCOPE_BASE, attrlist=['namingcontexts'] ) -except ldap.LDAPError: -return '' -conn.unbind_s() -try: -return entries[0][1]['namingcontexts'][0] -except (IndexError, KeyError): +contexts = entries[0][1]['namingcontexts'] +for c in contexts: +try: +entry = conn.search_s(c, ldap.SCOPE_BASE, "(info=IPA*)") +if len(entry) == 0: +continue +if entry[0][1]['info'][0].lower() != 'ipa v2.0': +continue +BASE_DN = c +break +except ldap.LDAPError, e: +logging.error('migration context search failed: %s' % e) +conn.unbind_s() +return '' +except ldap.LDAPError, e: +logging.error('migration context search failed: %s' % e) return '' +finally: +conn.unbind_s() + +return BASE_DN def bind(username, password): base_dn = get_base_dn() if not base_dn: +logging.error('migration unable to get base dn') raise IOError(errno.EIO, 'Cannot get Base DN') bind_dn = 'uid=%s,cn=users,cn=accounts,%s' % (username, base_dn) try: conn = ldap.initialize(LDAP_URI) conn.simple_bind_s(bind_dn, password) except (ldap.INVALID_CREDENTIALS, ldap.UNWILLING_TO_PERFORM, -ldap.NO_SUCH_OBJECT): +ldap.NO_SUCH_OBJECT), e: +logging.error('migration invalid credentials for %s: %s' % (bind_dn, convert_exception(e))) raise IOError(errno.EPERM, 'Invalid LDAP credentials for user %s' % username) -except ldap.LDAPError: +except ld
[Freeipa-devel] [PATCH] #1794 - Speed up replica setup
Newer 389ds servers have a new option to have a different set of filtered attributes from normal replication. This has been added in order to allow DS to replicate memberof attributes only during a total update so that we do not need to run a fixup memberof task on a replica at install time. This task is quite inefficient for big database and can take a long time. By replicating memberof while the DB is locked we are guaranteed the memberof list is consistent so we do not need a fixup. This patch allows to enable this feature dynamically. If the server does not yet support the new option it falls back to the previous behavior. Fixes: https://fedorahosted.org/freeipa/ticket/1794 I am sending the patch but it has been jointly developed at various stages by Nathan, JR, and me. Simo. -- Simo Sorce * Red Hat, Inc * New York >From 1ce9bd802d0cc59057d89ef2a2a7f0605c582308 Mon Sep 17 00:00:00 2001 From: JR Aquino Date: Fri, 16 Sep 2011 10:23:02 -0700 Subject: [PATCH] Replication: Adjust replica installation to omit processing memberof computations https://fedorahosted.org/freeipa/ticket/1794 If the master does not yet support the total update list feature we still run the memberof fixup task and not fail to replicate due to the new attribute not being settable. Jointly-developed-with: Simo Sorce Jointly-developed-with: Nathank Kinder --- install/tools/ipa-replica-install |1 - install/ui/test/data/json_metadata.json |1 + ipaserver/install/dsinstance.py |6 ++ ipaserver/install/replication.py| 28 +++- 4 files changed, 30 insertions(+), 6 deletions(-) diff --git a/install/tools/ipa-replica-install b/install/tools/ipa-replica-install index af317cde5bfbcfccde8d768b35c369207221e636..c77c21327b22326230c1587b932da5dd1bab3a76 100755 --- a/install/tools/ipa-replica-install +++ b/install/tools/ipa-replica-install @@ -466,7 +466,6 @@ def main(): raise RuntimeError("Failed to configure the client") ds.replica_populate() -ds.init_memberof() #Everything installed properly, activate ipa service. ipaservices.knownservices.ipa.enable() diff --git a/install/ui/test/data/json_metadata.json b/install/ui/test/data/json_metadata.json index 482eb6c3b8e502ee81714fe611822f2593bfd8f9..f87ab114c48330d6af791214d462fc9fc6562ffa 100644 --- a/install/ui/test/data/json_metadata.json +++ b/install/ui/test/data/json_metadata.json @@ -3273,6 +3273,7 @@ "nsds5replicaroot", "nsds5replicasessionpausetime", "nsds5replicatedattributelist", +"nsds5replicatedattributelisttotal", "nsds5replicatimeout", "nsds5replicatombstonepurgeinterval", "nsds5replicatransportinfo", diff --git a/ipaserver/install/dsinstance.py b/ipaserver/install/dsinstance.py index 790b560b92a28897a8f7e4ad076669241b168c3f..858bb892cb39fc36c7512cfb5e6c74b68fafb37f 100644 --- a/ipaserver/install/dsinstance.py +++ b/ipaserver/install/dsinstance.py @@ -169,6 +169,7 @@ class DsInstance(service.Service): self.idmax = None self.subject_base = None self.open_ports = [] +self.run_init_memberof = True if realm_name: self.suffix = util.realm_to_suffix(self.realm_name) self.__setup_sub_dict() @@ -295,6 +296,7 @@ class DsInstance(service.Service): repl.setup_replication(self.master_fqdn, r_binddn="cn=Directory Manager", r_bindpw=self.dm_password) +self.run_init_memberof = repl.needs_memberof_fixup() def __enable(self): self.backup_state("enabled", self.is_enabled()) @@ -433,6 +435,10 @@ class DsInstance(service.Service): self._ldap_mod("memberof-conf.ldif") def init_memberof(self): + +if not self.run_init_memberof: +return + self._ldap_mod("memberof-task.ldif", self.sub_dict) # Note, keep dn in sync with dn in install/share/memberof-task.ldif dn = "cn=IPA install %s,cn=memberof task,cn=tasks,cn=config" % self.sub_dict["TIME"] diff --git a/ipaserver/install/replication.py b/ipaserver/install/replication.py index 986fb5e83e9296ad415f54fa106170391904b3c0..a048cac4d070110b2071ec04b3880f812eb129c4 100644 --- a/ipaserver/install/replication.py +++ b/ipaserver/install/replication.py @@ -107,6 +107,7 @@ class ReplicationManager(object): self.starttls = starttls tmp = util.realm_to_suffix(realm) self.suffix = str(DN(tmp)).lower() +self.need_memberof_fixup = False # If we are passed a password we'll use it as the DM password # otherwise we'll do a GSSAPI bind. @@ -433,6 +434,7 @@ class ReplicationManager(object): which use a different name on each side. If master is None then isn't a dogtag replication agreement. """ + cn, dn = self.agreem
Re: [Freeipa-devel] [PATCH] 877 prompt for current password
Martin Kosek wrote: On Mon, 2011-09-19 at 09:03 -0400, Rob Crittenden wrote: Jan Cholasta wrote: On 16.9.2011 21:16, Rob Crittenden wrote: Prompt for the current password when changing your own password using ipa passwd. I had to jump through several hoops with this: - Added a new sortorder option so the Current password is prompted first IMO something like "before='password'" would be more readable and probably less error-prone than "sortorder=-1". The params are sorted numerically based on whether they are required, have a default, etc. A negative value means it will appear first. This is intended to be generic enough without having to worry about nested resolution (A before B, B before C, C before A). - Pass a magic value for current_password if changing someone else's password NOTE: This breaks the API for passwd. There is no way around it. I have this as a minor update as it won't cause older clients to blow up too badly, but their passwd command won't work. rob Honza Generally, it works fine except for the case when user passes its own user name. Do we want to support the following way? # klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: f...@idm.lab.bos.redhat.com Valid starting ExpiresService principal 09/23/11 09:48:05 09/24/11 09:48:05 krbtgt/idm.lab.bos.redhat@idm.lab.bos.redhat.com # ipa passwd fbar New Password: Enter New Password again to verify: ipa: ERROR: Insufficient access: Invalid credentials Maybe we could throw an error when user passes its own principal to ipa passwd command. After all, this argument is for changing _other_ user passwords. Martin Fixed. The username wasn't being normalized into a principal until after the default was set (where we determine whether to prompt for current password). rob >From fc5325be152c394a15ca79d5abda07b06c021e3d Mon Sep 17 00:00:00 2001 From: Rob Crittenden Date: Mon, 26 Sep 2011 22:19:57 -0400 Subject: [PATCH] Migration: don't assume there is only one naming context, add logging. We can't assume that there will be only one naming context. Look at each one until we find an IPA one. Add logging so you can know that a migration attempt fails and why. https://fedorahosted.org/freeipa/ticket/1834 https://fedorahosted.org/freeipa/ticket/1835 --- install/migration/invalid.html |2 +- install/migration/migration.py | 54 --- 2 files changed, 45 insertions(+), 11 deletions(-) diff --git a/install/migration/invalid.html b/install/migration/invalid.html index a641d1a..91de79f 100644 --- a/install/migration/invalid.html +++ b/install/migration/invalid.html @@ -35,7 +35,7 @@ If the problem persists, contact your administrator. - + Username: diff --git a/install/migration/migration.py b/install/migration/migration.py index ed6ade9..e8100ef 100644 --- a/install/migration/migration.py +++ b/install/migration/migration.py @@ -25,10 +25,24 @@ import errno import glob import ldap import wsgiref +import logging BASE_DN = '' LDAP_URI = 'ldaps://localhost:636' +def convert_exception(error): +""" +Convert an LDAP exception into something more readable. +""" +if not isinstance(error, ldap.TIMEOUT): +desc = error.args[0]['desc'].strip() +info = error.args[0].get('info', '').strip() +else: +desc = '' +info = '' + +return '%s (%s)' % (desc, info) + def wsgi_redirect(start_response, loc): start_response('302 Found', [('Location', loc)]) return [] @@ -44,6 +58,8 @@ def get_base_dn(): """ Retrieve LDAP server base DN. """ +global BASE_DN + if BASE_DN: return BASE_DN try: @@ -52,31 +68,50 @@ def get_base_dn(): entries = conn.search_ext_s( '', scope=ldap.SCOPE_BASE, attrlist=['namingcontexts'] ) -except ldap.LDAPError: -return '' -conn.unbind_s() -try: -return entries[0][1]['namingcontexts'][0] -except (IndexError, KeyError): +contexts = entries[0][1]['namingcontexts'] +for c in contexts: +try: +entry = conn.search_s(c, ldap.SCOPE_BASE, "(info=IPA*)") +if len(entry) == 0: +continue +if entry[0][1]['info'][0].lower() != 'ipa v2.0': +continue +BASE_DN = c +break +except ldap.LDAPError, e: +logging.error('migration context search failed: %s' % e) +conn.unbind_s() +return '' +except ldap.LDAPError, e: +logging.error('migration context search failed: %s' % e) return '' +finally: +conn.unbind_s() + +return BASE_DN def bind(username, password): base_dn = get_base_dn() if not base_dn: +logging.error('migration unable to get
[Freeipa-devel] HBAC Authorization Alternative to SSSD
Attached is a pam_python module that can be used to perform FreeIPA HBAC authorization in conjunction with pam_python.so (http://ace-host.stuart.id.au/russell/files/pam_python/) I have been working on this for a while as an alternative to sssd on systems that cannot support the sssd installation. There is no caching provided by this code, and is intended as a proof of concept or interim fix on a small scale. I have been craving a more formal c code approach to this general method, but am not adept in the c language. If anyone is feeling savoy, assistance in creating a more formal pam module would be very appreciated! #!/usr/bin/env python # # pam_pyauth.py (Python LDAP RBAC) # # Requires Python 2.4 or Greater # # Copyright (c) 2010 Jr Aquino # # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted (subject to the limitations in the # disclaimer below) provided that the following conditions are met: # #* Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # #* Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the # distribution. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT # HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED # WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF # MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE # LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR # CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF # SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR # BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, # WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE # OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN # IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. import re import os import socket import syslog import ldap class LDAP(object): "This class is used for defining ldap.conf values and searching ldap" def __init__(self): # Initial Setup # Read in ldap.conf conf = open('/etc/ldap.conf', 'r').readlines() # Setup base variables self.binddn = None self.bindpw = None self.baseDN = None self.ugroupDN = None self.pwgroupDN = None self.sysgroupDN = None self.hostgroupDN = None self.ignore_users = [] self.ldap_servers = [] # Regex Definitions uri_check = uri_check = re.compile(r'uri ((ldap|ldaps)://(.*))') binddn_check = re.compile(r'binddn (.*)') bindpw_check = re.compile(r'bindpw (.*)') basedn_check = re.compile(r'base (.*)') ignore_check = re.compile(r'nss_initgroups_ignoreusers (.*)') ugroup_check = re.compile(r'nss_base_group (.*)') pwgroup_check = re.compile(r'nss_base_passwd (.*)') sysgroup_check = re.compile(r'nss_base_systemgroup (.*)') ldaphostgroup_check = re.compile(r'nss_base_hostgroup (.*)') rolegroup_check = re.compile(r'nss_base_rolegroup (.*)') ignore_users = [] ldap_servers = [] # Anonymously bind if no auth data present self.binddn = '' self.bindpw = '' # Parse ldap.conf for line in conf: binddn_match = binddn_check.search(line) bindpw_match = bindpw_check.search(line) basedn_match = basedn_check.search(line) uri_match = uri_check.search(line) ignore_match = ignore_check.search(line) ugroup_match = ugroup_check.search(line) pwgroup_match = pwgroup_check.search(line) sysgroup_match = sysgroup_check.search(line) hostgroup_match = ldaphostgroup_check.search(line) rolegroup_match = rolegroup_check.search(line) if binddn_match: self.binddn = binddn_match.group(1) if bindpw_match: self.bindpw = bindpw_match.group(1) if basedn_match: self.baseDN = basedn_match.group(1) if uri_match: self.ldap_servers = uri_match.group(1).split() if ignore_match: self.ignore_users = ignore_match.group(1).split(',') if ugroup_match: self.ugroupDN = ugroup_match.group(1) if pwgroup_match: self.pwgroupDN = pwgroup_match.group(1) if sysgroup_match: self.sysgroupDN = sysgroup_match.group(1) if hostgroup_match: self.hostgroupDN = hostgroup_match.group(1) if ro
[Freeipa-devel] [PATCH] 134 Improve handling of GIDs when migrating groups
Since IPA v2 server already contain predefined groups that may collide with groups in migrated (IPA v1) server (for example admins, ipausers), users having colliding group as their primary group may happen to belong to an unknown group on new IPA v2 server. Implement --group-overwrite-gid option to overwrite GID of already existing groups to prevent this issue. https://fedorahosted.org/freeipa/ticket/1866 >From 7be44806e8f5f2862d1033a7c1726bbb0d7573ae Mon Sep 17 00:00:00 2001 From: Martin Kosek Date: Mon, 3 Oct 2011 16:01:01 +0200 Subject: [PATCH] Improve handling of GIDs when migrating groups Since IPA v2 server already contain predefined groups that may collide with groups in migrated (IPA v1) server (for example admins, ipausers), users having colliding group as their primary group may happen to belong to an unknown group on new IPA v2 server. Implement --group-overwrite-gid option to overwrite GID of already existing groups to prevent this issue. https://fedorahosted.org/freeipa/ticket/1866 --- ipalib/plugins/migration.py | 65 +++--- 1 files changed, 54 insertions(+), 11 deletions(-) diff --git a/ipalib/plugins/migration.py b/ipalib/plugins/migration.py index 93ac114d8f30ec9f97dcb4bc59ee9ac39f50f4c4..3321171062cf766a4629dc736b4965d997c7c6e1 100644 --- a/ipalib/plugins/migration.py +++ b/ipalib/plugins/migration.py @@ -71,6 +71,15 @@ EXAMPLES: Specify the user and group container. This can be used to migrate user and group data from an IPA v1 server: ipa migrate-ds --user-container='cn=users,cn=accounts' --group-container='cn=groups,cn=accounts' ldap://ds.example.com:389 + + Since IPA v2 server already contain predefined groups that may collide with + groups in migrated (IPA v1) server (for example admins, ipausers), users having + colliding group as their primary group may happen to belong to an unknown group + on new IPA v2 server. + Use --group-overwrite-gid option to overwrite GID of already existing groups + to prevent this issue: +ipa migrate-ds --group-overwrite-gid --user-container='cn=users,cn=accounts' --group-container='cn=groups,cn=accounts' ldap://ds.example.com:389 + """) # USER MIGRATION CALLBACKS AND VARS @@ -228,6 +237,21 @@ def _pre_migrate_group(ldap, pkey, dn, entry_attrs, failed, config, ctx, **kwarg return dn +def _group_exc_callback(ldap, dn, entry_attrs, exc, options): +if isinstance(exc, errors.DuplicateEntry): +if options.get('groupoverwritegid', False) and \ + entry_attrs.get('gidnumber') is not None: +try: +new_entry_attrs = {'gidnumber':entry_attrs['gidnumber']} +ldap.update_entry(dn, new_entry_attrs) +except errors.EmptyModlist: +# no change to the GID +pass +# mark as success +return + +raise exc + # DS MIGRATION PLUGIN def construct_filter(template, oc_list): @@ -252,6 +276,7 @@ class migrate_ds(Command): # pre_callback - is called for each object just after it was #retrieved from DS and before being added to IPA # post_callback - is called for each object after it was added to IPA +# exc_callback - is called when adding entry to IPA raises an exception # # {pre, post}_callback parameters: # ldap - ldap2 instance connected to IPA @@ -270,7 +295,8 @@ class migrate_ds(Command): 'oc_blacklist_option' : 'userignoreobjectclass', 'attr_blacklist_option' : 'userignoreattribute', 'pre_callback' : _pre_migrate_user, -'post_callback' : _post_migrate_user +'post_callback' : _post_migrate_user, +'exc_callback' : None }, 'group': { 'filter_template' : '(&(|%s)(cn=*))', @@ -278,7 +304,8 @@ class migrate_ds(Command): 'oc_blacklist_option' : 'groupignoreobjectclass', 'attr_blacklist_option' : 'groupignoreattribute', 'pre_callback' : _pre_migrate_group, -'post_callback' : None +'post_callback' : None, +'exc_callback' : _group_exc_callback, }, } migrate_order = ('user', 'group') @@ -359,6 +386,12 @@ class migrate_ds(Command): default=tuple(), autofill=True, ), +Flag('groupoverwritegid', +cli_name='group_overwrite_gid', +label=_('Overwrite GID'), +doc=_('When migrating a group already existing in IPA domain overwrite the '\ + 'group GID and report as success'), +), StrEnum('schema?', cli_name='schema', label=_('LDAP schema'), @@ -539,16 +572,26 @@ can use their Kerberos accounts.''') try: ldap.add_entry(dn, entry_attrs) except errors.ExecutionError, e: -failed[ldap_obj_name][pkey] = unicode(e) -
[Freeipa-devel] Fedora 16 support (systemd)
Hi, I was working for a month on systemd support for FreeIPA and now there is something you may try -- http://koji.fedoraproject.org/koji/taskinfo?taskID=3399157 Provided you have Fedora 16 + updates-testing install, provided that you get FreeIPA packages from the Koji task above, FreeIPA can be installed and configured on Fedora 16. Note that systemd support *will not work* for anything before Fedora 16 + current updates-testing as it requires very recent systemd version (at least 36-3 build in F16) and depends on very recent dogtag, 389ds, and tomcat6 packages. Attached is also current patch to introduce systemd/fedora16 support. The patch is against ipa-2-1 branch, I have not checked how it applies to master yet. -- / Alexander Bokovoy diff --git a/Makefile b/Makefile index 9d88025..3cd08e2 100644 --- a/Makefile +++ b/Makefile @@ -8,7 +8,7 @@ PRJ_PREFIX=freeipa RPMBUILD ?= $(PWD)/rpmbuild TARGET ?= master -SUPPORTED_PLATFORM=redhat +SUPPORTED_PLATFORM ?= redhat # After updating the version in VERSION you should run the version-update # target. diff --git a/freeipa.spec.in b/freeipa.spec.in index 8ebe189..2458eaa 100644 --- a/freeipa.spec.in +++ b/freeipa.spec.in @@ -28,6 +28,9 @@ BuildRequires: 389-ds-base-devel >= 1.2.9 BuildRequires: svrcore-devel BuildRequires: /usr/share/selinux/devel/Makefile BuildRequires: policycoreutils >= %{POLICYCOREUTILSVER} +%if 0%{?fedora} >= 16 +BuildRequires: systemd-units +%endif %endif BuildRequires: nspr-devel BuildRequires: nss-devel @@ -89,7 +92,11 @@ Requires(pre): 389-ds-base >= 1.2.9.7-1 Requires: openldap-clients Requires: nss Requires: nss-tools +%if 0%{?fedora} >= 16 +Requires: krb5-server >= 1.9.1-15 +%else Requires: krb5-server +%endif Requires: krb5-server-ldap Requires: krb5-pkinit-openssl Requires: cyrus-sasl-gssapi%{?_isa} @@ -102,6 +109,11 @@ Requires: python-ldap Requires: python-krbV Requires: acl Requires: python-pyasn1 >= 0.0.9a +%if 0%{?fedora} >= 16 +Requires: systemd-units >= 36-3 +Requires(pre): systemd-units +Requires(post): systemd-units +%endif %if 0%{?fedora} >= 15 Requires: selinux-policy >= 3.9.16-18 %else @@ -109,6 +121,12 @@ Requires: selinux-policy >= 3.9.7-27 %endif Requires(post): selinux-policy-base Requires: slapi-nis >= 0.21 +%if 0%{?fedora} >= 16 +Requires: pki-ca >= 9.0.14-1 +Requires: pki-silent >= 9.0.14-1 +# Only tomcat6 greater than this version provides proper systemd support +Requires: tomcat6 >= 6.0.32-17 +%else %if 0%{?fedora} >= 15 Requires: pki-ca >= 9.0.12 Requires: pki-silent >= 9.0.12 @@ -116,13 +134,19 @@ Requires: pki-silent >= 9.0.12 Requires: pki-ca >= 9.0.5 Requires: pki-silent >= 9.0.5 %endif +%endif Requires: dogtag-pki-common-theme Requires: dogtag-pki-ca-theme %if 0%{?rhel} Requires: subscription-manager %endif +%if 0%{?fedora} >= 16 +Requires(preun): python systemd-units +Requires(postun): python systemd-units +%else Requires(preun): python initscripts chkconfig Requires(postun): python initscripts chkconfig +%endif # We have a soft-requires on bind. It is an optional part of # IPA but if it is configured we need a way to require versions @@ -251,6 +275,9 @@ package. %build export CFLAGS="$CFLAGS %{optflags}" export CPPFLAGS="$CPPFLAGS %{optflags}" +%if 0%{?fedora} >= 16 +export SUPPORTED_PLATFORM=fedora16 +%endif make version-update cd ipa-client; ../autogen.sh --prefix=%{_usr} --sysconfdir=%{_sysconfdir} --localstatedir=%{_localstatedir} --libdir=%{_libdir} --mandir=%{_mandir}; cd .. %if ! %{ONLY_CLIENT} @@ -312,7 +339,16 @@ mkdir -p %{buildroot}%{_sysconfdir}/httpd/conf.d/ /bin/touch %{buildroot}%{_sysconfdir}/httpd/conf.d/ipa.conf /bin/touch %{buildroot}%{_sysconfdir}/httpd/conf.d/ipa-pki-proxy.conf /bin/touch %{buildroot}%{_sysconfdir}/httpd/conf.d/ipa-rewrite.conf -install -m755 ipa.init %{buildroot}%{_initrddir}/ipa +%if 0%{?fedora} >= 16 +# Default to systemd initscripts for F16 and above +mkdir -p %{buildroot}%{_unitdir} +for i in ipa.service ipa_kpasswd.service ; do + install -m 644 init/systemd/$i %{buildroot}%{_unitdir}/$i +done +rm -f %{buildroot}%{_initrddir}/ipa_kpasswd +%else +install -m755 init/SystemV/ipa.init %{buildroot}%{_initrddir}/ipa +%endif %endif mkdir -p %{buildroot}%{_sysconfdir}/ipa/ @@ -332,8 +368,14 @@ rm -rf %{buildroot} %if ! %{ONLY_CLIENT} %post server if [ $1 = 1 ]; then +%if 0%{?fedora} >= 16 +# Use systemd scheme +/bin/systemctl --system daemon-reload 2>&1 || : +%else +# Use SystemV scheme only before F16 /sbin/chkconfig --add ipa /sbin/chkconfig --add ipa_kpasswd +%endif fi if [ $1 -gt 1 ] ; then /usr/sbin/ipa-upgradeconfig || : @@ -342,14 +384,28 @@ fi %preun server if [ $1 = 0 ]; then +%if 0%{?fedora} >= 16 +# Use systemd scheme +/bin/systemctl --quiet stop ipa.service || : +/bin/systemctl --quiet disable ipa.service || : +%else +# Use SystemV scheme only before F16 /sbin/chkconfig --del ipa /sbin/chkconfig --del ipa_kpasswd /sbin
Re: [Freeipa-devel] [PATCH] 291 I18n clean-up.
On Mon, 2011-10-03 at 09:48 +0200, Petr Vobornik wrote: > On 10/01/2011 12:09 AM, Endi Sukma Dewata wrote: > > The hard-coded 'undo' and 'undo all' labels have been moved into > > internal.py to allow translation. > > > > Ticket #1897 > > ACK > Pushed to master, ipa-2-1. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 292 Disable sudo options Delete button if nothing selected.
On Mon, 2011-10-03 at 10:33 +0200, Petr Vobornik wrote: > On 10/01/2011 12:10 AM, Endi Sukma Dewata wrote: > > The Delete button for sudo options in sudo rule details page now > > will only work if there is at least one row selected. > > > > Ticket #1896 > > ACK > Pushed to master, ipa-2-1. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
[Freeipa-devel] [PATCH] 133 Be more clear about selfsign option
Installing IPA server --selfsign option is currently a one-way ticket to server with limited certificate capabilities. Make sure that user really want to install it by implementing the following steps: - moving the option to the bottom of certificate options section - adding a warning to ipa-server-install man page - adding a warning to ipa-server-install help - adding a warning to ipa-server-install configuration summary when one runs ipa-server-install https://fedorahosted.org/freeipa/ticket/1908 >From 2dc4882c3173c2b18c2958f39a93fda73c73a634 Mon Sep 17 00:00:00 2001 From: Martin Kosek Date: Mon, 3 Oct 2011 12:30:34 +0200 Subject: [PATCH] Be more clear about selfsign option Installing IPA server --selfsign option is currently a one-way ticket to server with limited certificate capabilities. Make sure that user really want to install it by implementing the following steps: - moving the option to the bottom of certificate options section - adding a warning to ipa-server-install man page - adding a warning to ipa-server-install help - adding a warning to ipa-server-install configuration summary when one runs ipa-server-install https://fedorahosted.org/freeipa/ticket/1908 --- install/tools/ipa-server-install | 10 -- install/tools/man/ipa-server-install.1 |8 +--- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/install/tools/ipa-server-install b/install/tools/ipa-server-install index 504d6af50f70278864dacf44cac9e4bbc832e069..7d961cb872efa6ce65cbb737871e000497a852b4 100755 --- a/install/tools/ipa-server-install +++ b/install/tools/ipa-server-install @@ -141,8 +141,6 @@ def parse_options(): parser.add_option_group(basic_group) cert_group = OptionGroup(parser, "certificate system options") -cert_group.add_option("", "--selfsign", dest="selfsign", action="store_true", - default=False, help="Configure a self-signed CA instance rather than a dogtag CA") cert_group.add_option("", "--external-ca", dest="external_ca", action="store_true", default=False, help="Generate a CSR to be signed by an external CA") cert_group.add_option("", "--external_cert_file", dest="external_cert_file", @@ -166,6 +164,9 @@ def parse_options(): cert_group.add_option("--subject", action="callback", callback=subject_callback, type="string", help="The certificate subject base (default O=)") +cert_group.add_option("", "--selfsign", dest="selfsign", action="store_true", + default=False, help="Configure a self-signed CA instance rather than a dogtag CA. " \ + "WARNING: Certificate management capabilities will be limited") parser.add_option_group(cert_group) dns_group = OptionGroup(parser, "DNS options") @@ -667,6 +668,11 @@ def main(): print "This program will set up the FreeIPA Server." print "" print "This includes:" +if options.selfsign: +print " * Configure NSS to handle a self-signed CA" +print "WARNING: certificate management capabilities will be limited" +else: +print " * Configure a stand-alone CA (dogtag) for certificate management" if options.conf_ntp: print " * Configure the Network Time Daemon (ntpd)" print " * Create and configure an instance of Directory Server" diff --git a/install/tools/man/ipa-server-install.1 b/install/tools/man/ipa-server-install.1 index 306fceb190c8af261bd9f580c2043f8a28fe86ba..f305723b1926851c007d0fd177e52baa51d927d6 100644 --- a/install/tools/man/ipa-server-install.1 +++ b/install/tools/man/ipa-server-install.1 @@ -71,9 +71,6 @@ An unattended installation that will never prompt for user input .SS "CERTIFICATE SYSTEM OPTIONS" .TP -\fB\-\-selfsign\fR -Configure a self\-signed CA instance for issuing server certificates instead of using dogtag for certificates -.TP \fB\-\-external\-ca\fR Generate a CSR to be signed by an external CA .TP @@ -106,6 +103,11 @@ The password of the Kerberos KDC PKCS#12 file .TP \fB\-\-subject\fR=\fISUBJECT\fR The certificate subject base (default O=REALM.NAME) +.TP +\fB\-\-selfsign\fR +Configure a self\-signed CA instance for issuing server certificates instead of using dogtag for certificates. + +WARNING: Using this option will restrain the server certificate management capabilities. Please, keep in mind that there is no way to change this setting later. .SS "DNS OPTIONS" .TP -- 1.7.6.2 ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 129 migrate process cannot handle multivalued pkey attribute
On Mon, 2011-10-03 at 10:10 +0200, Jan Cholasta wrote: > On 3.10.2011 09:54, Martin Kosek wrote: > > On Sat, 2011-10-01 at 19:45 +0200, Jan Cholasta wrote: > >> On 29.9.2011 12:01, Martin Kosek wrote: > >>> When group/user is migrated, the attribute used for RDN may be > >>> multivalued. Make sure that we pick the value used in the RDN > >>> which should be the unique one and not just the first one. > >>> > >>> https://fedorahosted.org/freeipa/ticket/1892 > >>> > >> > >> Every time you do "import *", god kills a kitten. Also, it pollutes the > >> module namespace with unnecessary symbols and decreases code readability. > > > > World is not just black and white. In this case I think its OK since > > ipalib/dn.py has a nice maintained __all__ list with all 3 DN related > > classes. Thus. I see no namespace pollution. > > IMO it still somewhat decreases code readability. But, whatever, it's no > showstopper. > > > > >> > >> I'm a bit puzzled why do you do this: > >> +try: > >> +pkey = dn[ldap_obj.primary_key.name].lower() > >> +except KeyError: > >> +failed[ldap_obj_name][str(dn)] = > >> unicode(_rdn_err_msg) > >> +continue > >> > >> and not just this: > >> +pkey = ava.value.lower() > > > > Good point. Updated patch attached. > > > > Martin > > > >> > >> Besides that, the issue seems to be fixed. > >> > >> Honza > >> > > > > ACK. > > Honza > Pushed to master, ipa-2-1. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 292 Disable sudo options Delete button if nothing selected.
On 10/01/2011 12:10 AM, Endi Sukma Dewata wrote: The Delete button for sudo options in sudo rule details page now will only work if there is at least one row selected. Ticket #1896 ACK -- Petr Vobornik ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 129 migrate process cannot handle multivalued pkey attribute
On 3.10.2011 09:54, Martin Kosek wrote: On Sat, 2011-10-01 at 19:45 +0200, Jan Cholasta wrote: On 29.9.2011 12:01, Martin Kosek wrote: When group/user is migrated, the attribute used for RDN may be multivalued. Make sure that we pick the value used in the RDN which should be the unique one and not just the first one. https://fedorahosted.org/freeipa/ticket/1892 Every time you do "import *", god kills a kitten. Also, it pollutes the module namespace with unnecessary symbols and decreases code readability. World is not just black and white. In this case I think its OK since ipalib/dn.py has a nice maintained __all__ list with all 3 DN related classes. Thus. I see no namespace pollution. IMO it still somewhat decreases code readability. But, whatever, it's no showstopper. I'm a bit puzzled why do you do this: +try: +pkey = dn[ldap_obj.primary_key.name].lower() +except KeyError: +failed[ldap_obj_name][str(dn)] = unicode(_rdn_err_msg) +continue and not just this: +pkey = ava.value.lower() Good point. Updated patch attached. Martin Besides that, the issue seems to be fixed. Honza ACK. Honza -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 129 migrate process cannot handle multivalued pkey attribute
On Sat, 2011-10-01 at 19:45 +0200, Jan Cholasta wrote: > On 29.9.2011 12:01, Martin Kosek wrote: > > When group/user is migrated, the attribute used for RDN may be > > multivalued. Make sure that we pick the value used in the RDN > > which should be the unique one and not just the first one. > > > > https://fedorahosted.org/freeipa/ticket/1892 > > > > Every time you do "import *", god kills a kitten. Also, it pollutes the > module namespace with unnecessary symbols and decreases code readability. World is not just black and white. In this case I think its OK since ipalib/dn.py has a nice maintained __all__ list with all 3 DN related classes. Thus. I see no namespace pollution. > > I'm a bit puzzled why do you do this: > +try: > +pkey = dn[ldap_obj.primary_key.name].lower() > +except KeyError: > +failed[ldap_obj_name][str(dn)] = > unicode(_rdn_err_msg) > +continue > > and not just this: > +pkey = ava.value.lower() Good point. Updated patch attached. Martin > > Besides that, the issue seems to be fixed. > > Honza > >From a3cf06058a32ebf03aa85d00a43fe832270f2701 Mon Sep 17 00:00:00 2001 From: Martin Kosek Date: Thu, 29 Sep 2011 11:55:13 +0200 Subject: [PATCH] migrate process cannot handle multivalued pkey attribute When group/user is migrated, the attribute used for RDN may be multivalued. Make sure that we pick the value used in the RDN which should be the unique one and not just the first one. https://fedorahosted.org/freeipa/ticket/1892 --- ipalib/plugins/migration.py | 18 +- 1 files changed, 17 insertions(+), 1 deletions(-) diff --git a/ipalib/plugins/migration.py b/ipalib/plugins/migration.py index f75612cef07eb06f4747d7989ee2d5d41c3bf8bd..93ac114d8f30ec9f97dcb4bc59ee9ac39f50f4c4 100644 --- a/ipalib/plugins/migration.py +++ b/ipalib/plugins/migration.py @@ -24,6 +24,7 @@ import ldap as _ldap from ipalib import api, errors, output from ipalib import Command, List, Password, Str, Flag, StrEnum from ipalib.cli import to_cli +from ipalib.dn import * if api.env.in_server and api.env.context in ['lite', 'server']: try: from ipaserver.plugins.ldap2 import ldap2 @@ -77,6 +78,7 @@ EXAMPLES: _krb_err_msg = _('Kerberos principal %s already exists. Use \'ipa user-mod\' to set it manually.') _grp_err_msg = _('Failed to add user to the default group. Use \'ipa group-add-member\' to add manually.') _ref_err_msg = _('Migration of LDAP search reference is not supported.') +_dn_err_msg = _('Malformed DN') _supported_schemas = (u'RFC2307bis', u'RFC2307') @@ -496,7 +498,21 @@ can use their Kerberos accounts.''') failed[ldap_obj_name][entry_attrs[0]] = unicode(_ref_err_msg) continue -pkey = entry_attrs[ldap_obj.primary_key.name][0].lower() +try: +dn = DN(dn) +except ValueError: +failed[ldap_obj_name][dn] = unicode(_dn_err_msg) +continue + +ava = dn[0][0] +if ava.attr == ldap_obj.primary_key.name: +# In case if pkey attribute is in the migrated object DN +# and the original LDAP is multivalued, make sure that +# we pick the correct value (the unique one stored in DN) +pkey = ava.value.lower() +else: +pkey = entry_attrs[ldap_obj.primary_key.name][0].lower() + if pkey in exclude: continue -- 1.7.6.2 ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 291 I18n clean-up.
On 10/01/2011 12:09 AM, Endi Sukma Dewata wrote: The hard-coded 'undo' and 'undo all' labels have been moved into internal.py to allow translation. Ticket #1897 ACK -- Petr Vobornik ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel