Re: [Freeipa-devel] [QE] Test categorization into tiers and acceptance testing - tagging proposals
On 03/31/2015 02:47 PM, Martin Koci wrote:
Hi all,
I'd like to open discussion on test categorization into tiers and
acceptance testing, respectively test tagging which should help us to
accomplish following goals:
1) Acceptance test - other FreeIPA partner projects (389/DS/PKI) should
be able to have an Acceptance test that would run basic *stable* test
suite that would check if anything significant broke. It should be fast
enough so that the projects can run it in a Jenkins CI after commits.
If we also have tags @dogtag or @sssd, the projects could simply run
just the tests affecting the projects - faster execution.
2) FreeIPA test run optimization. Currently, all FreeIPA tests are
running when new commit is pushed. This takes lot of resources. It would
be nice to at least be able to NOT run Tier 2 tests if Tier1 tests are
failing. Or it would be nice to not run some very expensive tests after
each commit, but maybe once per day/week.
*TIERS*
So after discussions with couple of developers and QE's we have created
and summarized following proposal for sorting current IPA tests into
tiers.
Currently used tests reside in freeipa/ipatests. From these the only
unit tests (tier 0 candidate) are test_{ipalib,ipapython} with the
exception of test_ipalib/test_rpc.py which requires kerberos.
The rest of the tests either require ipa/lite-server or are an
integration test. The rest of the tests (majority XML RPC, UI
tests, ...) then fall under the definition of Tier 1 test, as they
require at least running IPA instance and admin TGT.
As for the tagging of the test cases, pytest's capabilities can be used
[2]. Though pytest.mark currently does not work with declarative tests
(it marks all of them), when the test is an ordinary function/method the
marking works as expected. The declarative tests could be rewritten in
the future to more pytest specific form, e.g.
test_xmlrpc/test_host_plugin.py
Official guideline for this categorization will be created on the
upstream wiki once we agree on that.
*ACCEPTANCE TESTING*
As for the acceptance testing Similar to `Test categorization into
tiers` (1) proposal, there is a need to define a subset of freeipa tests
that could be run by other projects or users to find out whether or not
their changes (e.g. new build, feature) works with IPA.
This run could be composed from tier {0,1} execution followed by a
subset of integration tests test cases. The proposed mechanism for this
is the same as in [4], using pytest.mark to select the classes/tests to
run in this context.
What I'd like to ask you here is to share any ideas on the form of the
acceptance run as well as to help me identify the areas (and tests) that
are considered important and should be a part of this test set.
*TAGGING*
Tagging the actual tests classes with pytest decorator
(http://pytest.org/latest/mark.html). would be better than let
developers manually maintain lists of tests for different projects. The
benefit for pytest mark kept in the code is that whatever we do with the
test class (rename, move, merge), the tag goes with it, not extra list
needs to be maintained.
As for tagging itself, the original idea which Martin Kosek was
proposing was to use just the acceptance tag for marking the base T2
tests that would be part of FreeIPA acceptance tests.
However, it seems there is a value in tagging the tests that exercise
also certain sub-component of FreeIPA - SSSD, Dogtag. As long as we do
not get too wild with the tags, it should be OK.
So we could agreed on followings tags:
- tier0, tier1, tier2
- acceptance
- sssd
- dogtag
This would lead to e.g.
@pytest.mark.dogtag
@pytest.mark.acceptance
@pytest.mark.tier2
class TestExternalCA(IntegrationTest):
...
or simpler
@dogtag
@acceptance
@tier2
class TestExternalCA(IntegrationTest):
Hope it's not too long and that it makes sense.
It makes a lot of sense to me (it should, since I contributed to this proposal
too). So I will be looking forward to other developers thought on this.
If there are no objections, we could start with the actual patches and have
them properly reviewed.
Can I get your thoughts on this, please?
Thank you.
Regards,
/koca
*[1] - https://fedorahosted.org/freeipa/ticket/4922
*[2] - http://pytest.org/latest/mark.html
--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [RFC] COPR drop support for old distribution
ehlo, CentOS 7.1 was finally released[1]. Yupi. Fedora 21 was rewleased[2] few months ago. People can use FreeIPA 4.1 without any problem. So there's no more reason to maintain COPR repositories for older distributions. It will significantly reduce extra dependencies in repositories. It would be better to focus on backporting FreeIPA 4.2 in COPR. I know it has not been released yet. LS [1] http://lists.centos.org/pipermail/centos-announce/2015-April/021010.html [2] https://fedoraproject.org/wiki/Releases/21/Schedule -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
Re: [Freeipa-devel] [PATCH] webui: use no_members option in entity select search
On 03/31/2015 04:16 PM, Petr Vobornik wrote: Obtaining member information for entity selects is not needed and it causes unwanted performance hit, especially with larger groups. This patch removes it. https://fedorahosted.org/freeipa/ticket/4948 Works as expected and the speedup is substantial (ca 10x faster lookup of default group in user group rules for 16 groups with 100 members each). ACK. -- Martin^3 Babinsky -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
Re: [Freeipa-devel] One-way trust design
Thank you, the design page reads well to me. I had a short chat with Alexander where we cleared up some confusion. On Mon, Feb 23, 2015 at 06:02:53PM +0200, Alexander Bokovoy wrote: == New design == In order to support one-way trust to Active Directory, we need to switch SSSD in IPA master mode to use TDO credentials when resolving AD users and groups. This is a high level description of the design, and majority of work to allow the switch will be done by SSSD team. Corresponding ticket tracker on SSSD side is [https://fedorahosted.org/sssd/ticket/2579 ticket 2579], the text below is an overview of the design. On each IPA master SSSD runs in IPA master mode. This mode means that in case of existing trust to AD forest, SSSD will directly resolve AD users and groups against Active Directory Domain Controllers. To perform user/group resolution, SSSD needs to authenticate against AD LDAP servers and it does so using Kerberos authentication based on a host/ipa.master@IPA.REALM service ticket. The ticket towards AD LDAP services is issued by FreeIPA KDC with the help of cross-realm trust credentials. For one-way trust SSSD cannot use this approach because Active Directory Domain Controllers do not trust FreeIPA realm and, therefore, no cross-realm trust credentials exist in AD for FreeIPA realm. However, SSSD can use TDO object which always exists in AD for the trusting domain (cross-forest trust is done by forest root domains' trust). This means the ticket SSSD would need to request belongs to a different realm (AD forest root realm) rather than to FreeIPA realm. As FreeIPA supports multiple trusts to separate Active Directory forests, a support for multiple separate tickets is required. SSSD will need to gain ability to use different credentials caches to store TDO tickets and use different keytabs with TDO credentials to obtain the ticket from an Active Directory Domain Controllers. In order to separate privilege access, FreeIPA masters have to provide keytabs for SSSD running on IPA masters, one keytab per trusted AD forest, so that SSSD could request the keys when required. I will experiment with retrieving keytabs manually for now to simulate this part, then I'll write up a more detailed design on how to handle the one-way trusts. Additionally, FreeIPA management framework will need to change its defaults from producing a two-way trust to a one-way trust. Two-way trust will be added back when support for Global Catalog service will be added so that Active Directory resources could be properly accessed and access to them discretionally granted to FreeIPA users and groups. -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [PATCH 0026] ipa-server-install: deprecate manual setting of master KDC password
https://fedorahosted.org/freeipa/ticket/4516 -- Martin^3 Babinsky From 4237d0d11ab6fd34d066dba3f3d72bfa8c8a52d8 Mon Sep 17 00:00:00 2001 From: Martin Babinsky mbabi...@redhat.com Date: Tue, 31 Mar 2015 10:02:52 +0200 Subject: [PATCH] ipa-server-install: deprecate manual setting of master KDC password Option '-P' was used in older version of FreeIPA to set up KDC master password during server install. This is no longer neccessary or desirable since the password of sufficient strength can be generated automatically during installation. https://fedorahosted.org/freeipa/ticket/4516 --- install/tools/ipa-server-install | 8 +++- install/tools/man/ipa-server-install.1 | 8 +--- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/install/tools/ipa-server-install b/install/tools/ipa-server-install index 56a43770d95387762bce09634bd1056ba7f20576..9f237b8fcd9d21604b3ef4e0ada0e5427cd0e162 100755 --- a/install/tools/ipa-server-install +++ b/install/tools/ipa-server-install @@ -167,7 +167,7 @@ def parse_options(): sensitive=True, help=Directory Manager password) basic_group.add_option(-P, --master-password, dest=master_password, sensitive=True, - help=kerberos master password (normally autogenerated)) + help=SUPPRESS_HELP) basic_group.add_option(-a, --admin-password, sensitive=True, dest=admin_password, help=admin user kerberos password) @@ -697,6 +697,12 @@ def main(): signal.signal(signal.SIGTERM, signal_handler) signal.signal(signal.SIGINT, signal_handler) +if options.master_password: +msg = (WARNING:\noption '-P/--master-password' is deprecated. + KDC master password of sufficient strength is autogenerated + during IPA server installation and should not be set + manually.) +print textwrap.fill(msg, width=79, replace_whitespace=False) if options.uninstall: uninstalling = True standard_logging_setup(paths.IPASERVER_UNINSTALL_LOG, debug=options.debug) diff --git a/install/tools/man/ipa-server-install.1 b/install/tools/man/ipa-server-install.1 index e5224b110b136cbf56bf82887709a46880f22e89..1eaed72119a9cd2f9876d3dc3c4a662782c18a36 100644 --- a/install/tools/man/ipa-server-install.1 +++ b/install/tools/man/ipa-server-install.1 @@ -36,9 +36,6 @@ Your DNS domain name \fB\-p\fR \fIDM_PASSWORD\fR, \fB\-\-ds\-password\fR=\fIDM_PASSWORD\fR The password to be used by the Directory Server for the Directory Manager user .TP -\fB\-P\fR \fIMASTER_PASSWORD\fR, \fB\-\-master\-password\fR=\fIMASTER_PASSWORD\fR -The kerberos master password (normally autogenerated) -.TP \fB\-a\fR \fIADMIN_PASSWORD\fR, \fB\-\-admin\-password\fR=\fIADMIN_PASSWORD\fR The password for the IPA admin user .TP @@ -176,6 +173,11 @@ Uninstall an existing IPA installation \fB\-U\fR, \fB\-\-unattended\fR An unattended uninstallation that will never prompt for user input +.SH DEPRECATED OPTIONS +.TP +\fB\-P\fR \fIMASTER_PASSWORD\fR, \fB\-\-master\-password\fR=\fIMASTER_PASSWORD\fR +The kerberos master password (normally autogenerated). + .SH EXIT STATUS 0 if the (un)installation was successful -- 2.1.0 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [PATCH] 0004 User life cycle: support of MODRDN to a new superior
Hello, In user life cycle, Active entries are moved to Delete container and Delete entries can be moved back to Staging container. This requires a LDAP modrdn with new superior that is not supported in ldap2. thanks thierry From 7206c9dd84402c15d7a6a0a64eb404426c5385b5 Mon Sep 17 00:00:00 2001 From: Thierry bordaz (tbordaz) tbor...@redhat.com Date: Wed, 1 Apr 2015 16:42:43 +0200 Subject: [PATCH 7/7] User life cycle: allows MODRDN from ldap2 MODRDN allows to move an entry to a new superior. This function is needed from ldap2 class Reviewed By: https://fedorahosted.org/freeipa/ticket/3813 --- ipapython/ipaldap.py | 26 ++ 1 file changed, 26 insertions(+) diff --git a/ipapython/ipaldap.py b/ipapython/ipaldap.py index ce07006eb790c80fd42bd6eb611732ce9000db13..a16d0dc839c9e4720cb2b88d2e056be8a7fb9c70 100644 --- a/ipapython/ipaldap.py +++ b/ipapython/ipaldap.py @@ -581,6 +581,9 @@ class IPASimpleLDAPObject(object): dn = str(dn) assert isinstance(newrdn, (DN, RDN)) newrdn = str(newrdn) +if newsuperior: +assert isinstance(newsuperior, DN) +newsuperior = str(newsuperior) return self.conn.rename_s(dn, newrdn, newsuperior, delold) def result(self, msgid=ldap.RES_ANY, all=1, timeout=None): @@ -1610,6 +1613,29 @@ class LDAPClient(object): self.conn.rename_s(dn, new_rdn, delold=int(del_old)) time.sleep(.3) # Give memberOf plugin a chance to work +def move_entry_newsuperior(self, dn, new_rdn, new_superior=None, del_old=True): + +Move entry to a new superior and update entry's relative distinguished name. + +Keyword arguments: +new_superior -- superior where the entry is moved +del_old -- delete old RDN value (default True) + +:raises: +errors.NotFound if new_superior doesn't exist +errors.EmptyModlist if no new_superior and RDN is not changed + +assert isinstance(dn, DN) +assert isinstance(new_rdn, RDN) +if new_superior: +assert isinstance(new_superior, DN) +self.find_entries(filter=None, attrs_list=['dn'], base_dn=new_superior, scope=self.SCOPE_BASE) +with self.error_handler(): +self.conn.rename_s(dn, new_rdn, newsuperior=new_superior, delold=int(del_old)) +time.sleep(.3) # Give memberOf plugin a chance to work +else: +self.update_entry_rdn(dn, new_rdn, del_old=del_old) + def update_entry(self, entry, entry_attrs=None): Update entry's attributes. -- 1.7.11.7 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
Re: [Freeipa-devel] [PATCH] 811 performance: faster DN implementation
On 31.3.2015 12:11, Petr Vobornik wrote: The major change is that DN is no longer internally composed of RDNs and AVAs but it rather keeps the data in open ldap format - the same as output of str2dn function. Therefore, for immutable DNs, no other transformations are required on instantiation. Note: I guess that this is an python-ldap format rather than OpenLDAP format. It would be handy to fix commands for further generations to save them some banging with their heads against a wall of confusion. -- Petr^2 Spacek -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [PATCH 0223] Fix ldap2 do not create shared instance by default
Since API is not singleton anymore, ldap2 instance should not be shared
between all APIs.
Patch attached.
--
Martin Basti
From 5add879b420b6d73a1de63a933073c3659efc9aa Mon Sep 17 00:00:00 2001
From: Martin Basti mba...@redhat.com
Date: Wed, 25 Mar 2015 15:34:16 +0100
Subject: [PATCH] Fix ldap2 shared instance
Since API is not singleton anymore, ldap2 connections should not be
shared by default.
---
ipalib/backend.py| 2 +-
ipaserver/plugins/ldap2.py | 2 +-
ipatests/test_ipalib/test_backend.py | 12 ++--
3 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/ipalib/backend.py b/ipalib/backend.py
index 4c1001d4d47613537b64c314a2d22769a27f4c69..fcbbd254afc797019e9ea63214b1ee034b8c13f8 100644
--- a/ipalib/backend.py
+++ b/ipalib/backend.py
@@ -46,7 +46,7 @@ class Connectible(Backend):
`request.destroy_context()` can properly close all open connections.
-def __init__(self, shared_instance=True):
+def __init__(self, shared_instance=False):
Backend.__init__(self)
if shared_instance:
self.id = self.name
diff --git a/ipaserver/plugins/ldap2.py b/ipaserver/plugins/ldap2.py
index 3211b3390fb979f090467445905513d33e537e17..fd4ed29903fb2f3afe0f4b74467bf53df49654fa 100644
--- a/ipaserver/plugins/ldap2.py
+++ b/ipaserver/plugins/ldap2.py
@@ -61,7 +61,7 @@ class ldap2(LDAPClient, CrudBackend):
LDAP Backend Take 2.
-def __init__(self, shared_instance=True, ldap_uri=None, base_dn=None,
+def __init__(self, shared_instance=False, ldap_uri=None, base_dn=None,
schema=None):
self.__ldap_uri = None
diff --git a/ipatests/test_ipalib/test_backend.py b/ipatests/test_ipalib/test_backend.py
index c69757cb3d68ebc12f9c91572d37603738357c4e..121c4745bd1dfebfbeed75ba1b46b4420064fe63 100644
--- a/ipatests/test_ipalib/test_backend.py
+++ b/ipatests/test_ipalib/test_backend.py
@@ -76,7 +76,7 @@ class test_Connectible(ClassChecker):
object.__setattr__(self, 'args', args)
object.__setattr__(self, 'kw', kw)
return 'The connection.'
-o = example()
+o = example(shared_instance=True)
args = ('Arg1', 'Arg2', 'Arg3')
kw = dict(key1='Val1', key2='Val2', key3='Val3')
assert not hasattr(context, 'example')
@@ -104,7 +104,7 @@ class test_Connectible(ClassChecker):
class example(self.cls):
pass
for klass in (self.cls, example):
-o = klass()
+o = klass(shared_instance=True)
e = raises(NotImplementedError, o.create_connection)
assert str(e) == '%s.create_connection()' % klass.__name__
@@ -114,7 +114,7 @@ class test_Connectible(ClassChecker):
class example(self.cls):
destroy_connection = Disconnect()
-o = example()
+o = example(shared_instance=True)
m = disconnect: 'context.%s' does not exist in thread %r
e = raises(StandardError, o.disconnect)
@@ -131,7 +131,7 @@ class test_Connectible(ClassChecker):
class example(self.cls):
pass
for klass in (self.cls, example):
-o = klass()
+o = klass(shared_instance=True)
e = raises(NotImplementedError, o.destroy_connection)
assert str(e) == '%s.destroy_connection()' % klass.__name__
@@ -142,7 +142,7 @@ class test_Connectible(ClassChecker):
class example(self.cls):
pass
for klass in (self.cls, example):
-o = klass()
+o = klass(shared_instance=True)
assert o.isconnected() is False
conn = 'whatever'
setattr(context, klass.__name__, conn)
@@ -157,7 +157,7 @@ class test_Connectible(ClassChecker):
class example(self.cls):
pass
for klass in (self.cls, example):
-o = klass()
+o = klass(shared_instance=True)
e = raises(AttributeError, getattr, o, 'conn')
assert str(e) == msg % (
klass.__name__, threading.currentThread().getName()
--
2.1.0
--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
