Re: [Freeipa-devel] [PATCH] dogtag clean-up
Patch looks great, thanks for including such detailed documentation. But it's not applying to the current master: Falling back to patching base and 3-way merge... Auto-merging ipalib/plugins/service.py CONFLICT (content): Merge conflict in ipalib/plugins/service.py Auto-merging ipalib/x509.py Failed to merge in the changes. Patch failed at 0001 dogtag clean-up Can you take a look? On Wed, 2009-12-02 at 19:01 -0500, John Dennis wrote: > The essence of this patch is to return the correct types from > certificate plugins and avoid scraping Javascript from dogtag (CMS) > html responses with better error handling. Instead we ask CMS to > always return our data as XML documents which can be much more > robustly parsed (including properly handling issues such as character > encoding, escapes, etc.). > > Fundamentally the process is split into two parts. A parsing routine > which returns a dict with all the values from CMS in the correct > Python types for IPA. The possible values returned from CMS are fully > documented and can easily be read via the documentation link in HTML > posted at the bottom (plus in the code of course). The command plugin > invokes the parsing routine and picks out from the parse result dict > the values it wants to return (and may optionaly convert the type as > needed for XMLRPC, this is fully documented, in particular serial > numbers need special handling in XMLRPC). This model allows us to use > different parsing methods without disturbing the logic in the command > plugin should that ever be necessary (i.e. clear separation of > responsibilities). > > Status results are never returned in the command result. Instead we > use the defined exception handling logic for IPA XMLRPC. If the > command fails in some fashion we return a CertificateOperationError > exception. On the receiving end if no exception has been thrown it > knows the values returned are valid. > > Careful attention has been paid to the types being used. Strings are > always unicode, integral values are represented as either int or long > objects. No longer are integral values represented as strings with > confusion as to thier radix representation (with the notable exception > of serial numbers which must be passed through XMLRPC as decimal > strings, the rules for this are fully documented). > > The logic in the selfsign and dogtag plugins have been brought into > alignment. > > Much more extensive error checking has been added to selfsign to > handle issues concering serial number operations. > > A new error exception has been added (CertificateOperationError). > > Error messages have been localized. > > The check_ra.py test was updated (unfortunately this test requires a > configured server so I used my test server). > > Extensive documentation has been added to many of the routines. > > Easy to browse HTML documentation for the dogtag plugin can be found > here (for the time being) > > http://jdennis.fedorapeople.org/ipa/dogtag > > I've noticed we have a bit of code duplication going on with CMS > interactions. In the future we shold consolodate all CMS code in one > library (module). > > This patch has been lingering in my private repo too long. I no longer > want to keep merging as others modify the same code :-) So here it > is. Other components of the fixes have already been posted as patches. > > > ___ > Freeipa-devel mailing list > Freeipa-devel@redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-devel ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 326 bump IPA install version
On Wed, 2009-12-02 at 16:26 -0500, Rob Crittenden wrote: > We store a rough version of IPA at install time in the base object, bump > this up to V2.0 > > rob ack. pushed to master. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 325 test for cert plugin
On Wed, 2009-12-02 at 13:11 -0500, Rob Crittenden wrote: > John Dennis wrote: > > On 12/01/2009 11:19 PM, Rob Crittenden wrote: > >> An extremely basic test for the cert plugin. Only tests the cert-request > >> command but it's a start. > > > > I think the test should also check for the correct return type. For > > instance shouldn't > > > > assert res['subject'] == 'CN=ipatestcert.greyoak.com' > > > > by followed (or preceded by) > > > > assert type(res['subject']) is unicode > > > > Also, is this going to deprecate checks/check_ra.py? > > > > Ah, excellent point. What it probably really should do is call > xmlrpc_test.assert_attr_equal() which should do the unicode enforcement. > If we need additional types we can add an expected type argument, > defaulting to unicode. > > I'm reluctant to tackle this just yet with Jason's big patch looming. It > contains a bunch of changes to the test infrastructure to handle the new > return values. Perhaps I'll shelve this patch for a few days until we > can get his patch committed, then rebase this and add in the type > enforcement and resubmit. > > rob ack. pushed to master. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] jderose 028 Lossless datetime round-trip
On Thu, 2009-12-03 at 11:56 -0500, Rob Crittenden wrote: > Jason Gerard DeRose wrote: > > As per John's request, this patch allows lossless round-tripping of > > Python datetime.datetime objects. > > > > Unfortunately, the xmlrpclib dumps() and loads() functions use funny > > wrapper objects like xmlrpclib.DateTime rather than directly serializing > > to/from standard Python types like datetime.datetime. This makes > > lossless round-tripping pretty cumbersome to implement. > > > > Doing a loads(foo, use_datetime=True) would work, but the `use_datetime` > > kwarg is only available in Python2.5 and newer, so I instead extended my > > xml_wrap() and xml_unwrap() functions. > > > > What should this do it if the incoming DateTime value is not parsed > correctly by datetime.datetime()? > > rob I don't believe this can happen... DateTime and datetime are both stored in a time.struct_time, so if the XML contains an invalid date, things will have already blown-up when the DateTime was created. I image xmlrpclib will raise a ProtocolError error, but I can add a test for this. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] service record conundrum
Rob Crittenden wrote: > Dmitri Pal wrote: >> Rob Crittenden wrote: >>> Here is sort of a tricky problem, need some advice (LONG). >>> >>> When we bootstrap an IPA server we create a number of principals for >>> the server itself. We create a host/, HTTP/ and ldap/ principal using >>> kadmin.local. By using kadmin.local this entry is put into >>> cn=kerberos,dc=example,dc=com. >>> >>> This has the nice side effect of making these records not appear as >>> service entries so they are unmodifiable by anyone, meaning an admin >>> will have a really hard time hosing their server. >>> >>> The downside is that these records do not appear as service entries, >>> so if you search for services on the IPA server you'll get nothing. >>> >> >> How do we search? What base DN we use? One of the solutions might be to >> install these principals as is and only later apply ipaService object >> class to them so that the search for services would find them. Would be >> a bit ugly since as far as I understand these services are in a >> different location in the tree but this approach might be less painfull >> than LDIF and delete and add. >> I hope that we will get the RDN renames pretty soon so that this would >> not be an issue but it might not be soon enough for v2. >> > > We search in the baseDN of the type of object is is, so cn=services, > cn=computers, cn=users, etc. > > We also filter on the objectclasses that should be in that object. > > Searching in 2 places is possible just not something we currently do. > > I'm leaning towards moving the entries, more so since I haven't gotten > any "that is the dumbest idea I've heard all week" responses :-) > > We store a list of the IPA masters in the DIT somewhere, I'll have to > see if I can find a way to maintain protection of the principals using > that. > Will it affect expectations of the KDC on where it searches for its entries? Would we have to also update KDC DAL configuration to search the records in a different place? > rob -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. --- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] jderose 028 Lossless datetime round-trip
On 12/03/2009 11:56 AM, Rob Crittenden wrote: What should this do it if the incoming DateTime value is not parsed correctly by datetime.datetime()? Well in theory this should never happen if the parameter value complies with the specification which demands it be in iso8601 format. I would imagine it would raise a ValueError just like any other ill formed parameter. -- John Dennis Looking to carve out IT costs? www.redhat.com/carveoutcosts/ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] jderose 028 Lossless datetime round-trip
Jason Gerard DeRose wrote: As per John's request, this patch allows lossless round-tripping of Python datetime.datetime objects. Unfortunately, the xmlrpclib dumps() and loads() functions use funny wrapper objects like xmlrpclib.DateTime rather than directly serializing to/from standard Python types like datetime.datetime. This makes lossless round-tripping pretty cumbersome to implement. Doing a loads(foo, use_datetime=True) would work, but the `use_datetime` kwarg is only available in Python2.5 and newer, so I instead extended my xml_wrap() and xml_unwrap() functions. What should this do it if the incoming DateTime value is not parsed correctly by datetime.datetime()? rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] service record conundrum
Dmitri Pal wrote: Rob Crittenden wrote: Here is sort of a tricky problem, need some advice (LONG). When we bootstrap an IPA server we create a number of principals for the server itself. We create a host/, HTTP/ and ldap/ principal using kadmin.local. By using kadmin.local this entry is put into cn=kerberos,dc=example,dc=com. This has the nice side effect of making these records not appear as service entries so they are unmodifiable by anyone, meaning an admin will have a really hard time hosing their server. The downside is that these records do not appear as service entries, so if you search for services on the IPA server you'll get nothing. How do we search? What base DN we use? One of the solutions might be to install these principals as is and only later apply ipaService object class to them so that the search for services would find them. Would be a bit ugly since as far as I understand these services are in a different location in the tree but this approach might be less painfull than LDIF and delete and add. I hope that we will get the RDN renames pretty soon so that this would not be an issue but it might not be soon enough for v2. We search in the baseDN of the type of object is is, so cn=services, cn=computers, cn=users, etc. We also filter on the objectclasses that should be in that object. Searching in 2 places is possible just not something we currently do. I'm leaning towards moving the entries, more so since I haven't gotten any "that is the dumbest idea I've heard all week" responses :-) We store a list of the IPA masters in the DIT somewhere, I'll have to see if I can find a way to maintain protection of the principals using that. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] service record conundrum
Rob Crittenden wrote: > Here is sort of a tricky problem, need some advice (LONG). > > When we bootstrap an IPA server we create a number of principals for > the server itself. We create a host/, HTTP/ and ldap/ principal using > kadmin.local. By using kadmin.local this entry is put into > cn=kerberos,dc=example,dc=com. > > This has the nice side effect of making these records not appear as > service entries so they are unmodifiable by anyone, meaning an admin > will have a really hard time hosing their server. > > The downside is that these records do not appear as service entries, > so if you search for services on the IPA server you'll get nothing. > How do we search? What base DN we use? One of the solutions might be to install these principals as is and only later apply ipaService object class to them so that the search for services would find them. Would be a bit ugly since as far as I understand these services are in a different location in the tree but this approach might be less painfull than LDIF and delete and add. I hope that we will get the RDN renames pretty soon so that this would not be an issue but it might not be soon enough for v2. -- Thank you, Dmitri Pal Engineering Manager IPA project, Red Hat Inc. --- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
[Freeipa-devel] [PATCHES] Use the dns plugin during installation
Hi,
these three patches should make sure that we add dns records the right
way. It will also serve for the ipa-dns-install command that's almost
ready, patch will be coming soon.
Thanks
Martin
>From edbd834dab63b86c04d9b1bb736c64cda3e38b2a Mon Sep 17 00:00:00 2001
From: Martin Nagy
Date: Thu, 3 Dec 2009 16:32:56 +0100
Subject: [PATCH 1/3] Move api finalization in ipa-server-install after writing default.conf
We will need to have ipalib correctly configured before we start
installing DNS entries with api.Command.dns.
---
install/tools/ipa-server-install | 45 ++---
1 files changed, 22 insertions(+), 23 deletions(-)
diff --git a/install/tools/ipa-server-install b/install/tools/ipa-server-install
index 0694d6e..21bead1 100755
--- a/install/tools/ipa-server-install
+++ b/install/tools/ipa-server-install
@@ -481,18 +481,12 @@ def main():
fstore = sysrestore.FileStore('/var/lib/ipa/sysrestore')
+# Configuration for ipalib, we will bootstrap and finalize later, after
+# we are sure we have the configuration file ready.
cfg = dict(
in_server=True,
-webui_assets_dir=ASSETS_DIR,
debug=options.debug
)
-if not options.uninstall:
-if options.ca:
-cfg['ra_plugin'] = 'dogtag'
-else:
-cfg['ra_plugin'] = 'selfsign'
-api.bootstrap(**cfg)
-api.finalize()
if options.uninstall:
if not options.unattended:
@@ -502,6 +496,8 @@ def main():
print "Aborting uninstall operation."
sys.exit(1)
+api.bootstrap(**cfg)
+api.finalize()
return uninstall(not certs.ipa_self_signed())
# This will override any settings passed in on the cmdline
@@ -662,6 +658,24 @@ def main():
else:
dns_forwarders = ()
+# Create the management framework config file and finalize api
+fstore.backup_file("/etc/ipa/default.conf")
+fd = open("/etc/ipa/default.conf", "w")
+fd.write("[global]\n")
+fd.write("basedn=" + util.realm_to_suffix(realm_name) + "\n")
+fd.write("realm=" + realm_name + "\n")
+fd.write("domain=" + domain_name + "\n")
+fd.write("xmlrpc_uri=https://%s/ipa/xml\n"; % host_name)
+fd.write("ldap_uri=ldapi://%%2fvar%%2frun%%2fslapd-%s.socket\n" % dsinstance.realm_to_serverid(realm_name))
+fd.write("enable_ra=True\n")
+if options.ca:
+fd.write("ra_plugin=dogtag\n")
+fd.write('webui_assets_dir=' + ASSETS_DIR + '\n')
+fd.close()
+
+api.bootstrap(**cfg)
+api.finalize()
+
if not options.unattended:
print ""
print "The following operations may take some minutes to complete."
@@ -749,21 +763,6 @@ def main():
http.create_instance(realm_name, host_name, domain_name, autoconfig=True, self_signed_ca=not options.ca)
ipautil.run(["/sbin/restorecon", "/var/cache/ipa/sessions"])
-# Create the management framework config file
-fstore.backup_file("/etc/ipa/default.conf")
-fd = open("/etc/ipa/default.conf", "w")
-fd.write("[global]\n")
-fd.write("basedn=" + util.realm_to_suffix(realm_name) + "\n")
-fd.write("realm=" + realm_name + "\n")
-fd.write("domain=" + domain_name + "\n")
-fd.write("xmlrpc_uri=https://%s/ipa/xml\n"; % host_name)
-fd.write("ldap_uri=ldapi://%%2fvar%%2frun%%2fslapd-%s.socket\n" % dsinstance.realm_to_serverid(realm_name))
-fd.write("enable_ra=True\n")
-if options.ca:
-fd.write("ra_plugin=dogtag\n")
-fd.write('webui_assets_dir=' + ASSETS_DIR + '\n')
-fd.close()
-
# Apply any LDAP updates. Needs to be done after the configuration file
# is created
service.print_msg("Applying LDAP updates")
--
1.6.2.5
>From c601970aff98645e5c9d82b490d5b90acb26313f Mon Sep 17 00:00:00 2001
From: Martin Nagy
Date: Tue, 10 Nov 2009 13:21:09 +0100
Subject: [PATCH 2/3] Use the dns plug-in for addition of records during installation
Fixes #528943
---
install/share/Makefile.am |1 -
install/share/dns.ldif| 88
install/share/dns_reverse.ldif| 24
ipaserver/install/bindinstance.py | 113 ++--
4 files changed, 81 insertions(+), 145 deletions(-)
delete mode 100644 install/share/dns_reverse.ldif
diff --git a/install/share/Makefile.am b/install/share/Makefile.am
index b74f990..e3e7cf6 100644
--- a/install/share/Makefile.am
+++ b/install/share/Makefile.am
@@ -16,7 +16,6 @@ app_DATA =\
default-keytypes.ldif \
delegation.ldif \
dns.ldif \
- dns_reverse.ldif \
kerberos.ldif \
indices.ldif \
bind.named.conf.template \
diff --git a/install/share/dns.ldif b/install/share/dns.ldif
index 8ce9d69..cb783b8 100644
--- a/install/share/dns.ldif
+++ b/install/share/dns.ldif
@@ -4,91 +4,3 @@ objectClass: nsContainer
objectClass: top
cn: dns
aci: (targetfilter = "(objectClass=idnsRecord)")(targetattr != "aci")(version 3.0; acl "DNS Servers Updates"
Re: [Freeipa-devel] service record conundrum
Rob Crittenden wrote: Here is sort of a tricky problem, need some advice (LONG). When we bootstrap an IPA server we create a number of principals for the server itself. We create a host/, HTTP/ and ldap/ principal using kadmin.local. By using kadmin.local this entry is put into cn=kerberos,dc=example,dc=com. This has the nice side effect of making these records not appear as service entries so they are unmodifiable by anyone, meaning an admin will have a really hard time hosing their server. The downside is that these records do not appear as service entries, so if you search for services on the IPA server you'll get nothing. Even worse it means you can't request certificates for these services, because they don't exist. Not that one really should since we also generate certificates for these at bootstrap, but we don't store them anywhere because there isn't any place to put them. This also means that we can't track expiration of these. To make things even more fun we have the DS uniqueness plugin configured so there can be no duplication in principal names. Since this is in the RDN of service records we can't even create a bit of a bogus entry to still protect the principals and yet be able to store certificates. Remember too that these records are creating during installation, effectively bootstrapping the real services (httpd, dirsrv), so we have limited options for how to generate them to begin with. One idea I had is to continue to use kadmin.local to create the principals and then move them out of cn=kerberos into cn=services, adding whatever additional data we need. This way we would maintain the principalkeys. Then we'd need to insert the certificates we generate. Unfortunately 389-DS doesn't seem to support newsuperior so I guess we'd have to move it ourselves via delete and re-add. 389 will soon support new superior. But yes, delete then add is the only way to do that at present. If you want to preserve the uuid of the entry, you could dump to ldif, edit the DN to "move" it, then ldif2db back - kind of painful, but would preserve the uuid of the entries. So I'm basically stuck right now. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel smime.p7s Description: S/MIME Cryptographic Signature ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Problem with ipa installation: certutil
On Wed, 2009-12-02 at 09:38 -0500, Rob Crittenden wrote: > Martin Nagy wrote: > > Hi, > > I'm trying to install ipa and am getting a python traceback (attached). > > It seems that running certutil didn't succeed so I added a debugging > > print before it's execution and tried to run it manually. This is what I > > get: > > > > # /usr/bin/certutil -d /etc/httpd/alias -S -n 'CA certificate' -s > > 'cn=IPA Test Certificate Authority' -x -t 'CT,,C' -1 -2 -5 -m 1056 -v > > 120 -z /etc/httpd/alias/noise.txt -f /etc/httpd/alias/pwdfile.txt > > certutil -o: unable to open "tempcertreq" for writing (-5950, 2) > > Exit 255 > > > > (The "Exit 255" is from my shell saying that certutil exited returning > > 255). I did a git grep tempcertreq in freeipa git tree but didn't find > > anything, so I'm assuming we weren't creating it or anything. Does > > anyone know what might be causing this error? > > > > Martin > > This message comes directly from certutil itself. It tries to open the > file "tempcertreq" in the cwd. > > Odd since you are installing this as root, right? Perhaps you are in a > directory that no longer exists? Correct. I was in my freeipa git directory when I executed ipa-server-install but had to delete it and clone again in other terminal. > I seem to recall running into this in v1 as well and though we did a > chdir(). Maybe we do that in some places and not others. Should we make a patch to prevent any future problems like this (even if they are rare)? Maybe at the beginning we could chdir() to our current directory to make sure, and abort if that fails. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
