Re: [Freeipa-devel] [PATCH] jderose 019 remove some cruft
On Wed, 2009-10-14 at 17:21 -0400, Rob Crittenden wrote: Jason Gerard DeRose wrote: On Tue, 2009-10-13 at 22:45 -0600, Jason Gerard DeRose wrote: This removes the util.add_global_options() function and the frontend.Application class, neither of which are now needed. And *this* actually attaches the patch. ;) ack pushed to master. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 296 work with newer schema layout of 389-DS
On Wed, 2009-10-14 at 11:53 -0400, Rob Crittenden wrote: The HEAD branch of upstream of 389-DS has lots new schema stuff. We have to work around some incompatibilities with the DNS schema in 05rfc2247.ldif but this isn't required in the HEAD, so don't fail if we can't replace this file. It isn't needed in newer versions of DS. Ack Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 294 sleep before doing a task
Rob Crittenden wrote: One of the last steps of an install is to run through any updates. This change adds a sleep() prior to calling tasks to ensure postop writes are done We were seeing a rare deadlock of DS when creating the memberOf task because one thread was adding memberOf in a postop while another was trying to create an index and this was causing a PRLock deadlock. rob sleep might not be the best synchronization mechanism out there, but I think that in this case it is pretty much the only one available and it gets the job done, so ack. Pavel ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 294 sleep before doing a task
On Thu, 2009-10-15 at 15:28 +0200, Pavel Zuna wrote: Rob Crittenden wrote: One of the last steps of an install is to run through any updates. This change adds a sleep() prior to calling tasks to ensure postop writes are done We were seeing a rare deadlock of DS when creating the memberOf task because one thread was adding memberOf in a postop while another was trying to create an index and this was causing a PRLock deadlock. rob sleep might not be the best synchronization mechanism out there, but I think that in this case it is pretty much the only one available and it gets the job done, so ack. So are we covering a DS bug here ? Or are we doing an asynchronous ldap request when we should do a synchronous one and wait for it to finish (I've fixed another place where we were doing that and racing against our own requests) ? Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 294 sleep before doing a task
On 10/15/2009 06:40 AM, Simo Sorce wrote: On Thu, 2009-10-15 at 15:28 +0200, Pavel Zuna wrote: Rob Crittenden wrote: One of the last steps of an install is to run through any updates. This change adds a sleep() prior to calling tasks to ensure postop writes are done We were seeing a rare deadlock of DS when creating the memberOf task because one thread was adding memberOf in a postop while another was trying to create an index and this was causing a PRLock deadlock. rob sleep might not be the best synchronization mechanism out there, but I think that in this case it is pretty much the only one available and it gets the job done, so ack. So are we covering a DS bug here ? Or are we doing an asynchronous ldap request when we should do a synchronous one and wait for it to finish (I've fixed another place where we were doing that and racing against our own requests) ? It has nothing to do with the way you are performing your LDAP operations. The issue really stems from what I consider to be a bug in NSPR's implementation of reader-writer locks. It is documented that a single thread can hold multiple reader locks safely, but I've found that to not exactly be the case. The NSPR implementation favors writers, so a thread waiting for the writer lock will block attempts by other threads to get a reader lock. The problem is that we use reader locks in a re-entrant fashion, so a thread that already has a reader lock can be blocked when attempting to get a second reader lock due to a waiting writer. This thread in turn blocks the writer thread since it already holds a reader lock. I have proposed a solution to the NSPR developers that would allow an attempt to get a reader lock to go through even if a writer is waiting if the requesting thread already has another reader lock. I'm hoping that this can be resolved in NSPR, otherwise we may have to change DS to use the pthread_rwlock_* interfaces instead. The sleep is a temporary workaround. This issue should not arise in normal operation since the lock in question is around the backend struct, which is only modified when there is some sort of database maintenance operation (such as the reindexing task that Rob triggered it with). Simo. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 294 sleep before doing a task
On Thu, 2009-10-15 at 08:15 -0700, Nathan Kinder wrote: On 10/15/2009 06:40 AM, Simo Sorce wrote: On Thu, 2009-10-15 at 15:28 +0200, Pavel Zuna wrote: Rob Crittenden wrote: One of the last steps of an install is to run through any updates. This change adds a sleep() prior to calling tasks to ensure postop writes are done We were seeing a rare deadlock of DS when creating the memberOf task because one thread was adding memberOf in a postop while another was trying to create an index and this was causing a PRLock deadlock. rob sleep might not be the best synchronization mechanism out there, but I think that in this case it is pretty much the only one available and it gets the job done, so ack. So are we covering a DS bug here ? Or are we doing an asynchronous ldap request when we should do a synchronous one and wait for it to finish (I've fixed another place where we were doing that and racing against our own requests) ? It has nothing to do with the way you are performing your LDAP operations. The issue really stems from what I consider to be a bug in NSPR's implementation of reader-writer locks. It is documented that a single thread can hold multiple reader locks safely, but I've found that to not exactly be the case. The NSPR implementation favors writers, so a thread waiting for the writer lock will block attempts by other threads to get a reader lock. The problem is that we use reader locks in a re-entrant fashion, so a thread that already has a reader lock can be blocked when attempting to get a second reader lock due to a waiting writer. This thread in turn blocks the writer thread since it already holds a reader lock. I have proposed a solution to the NSPR developers that would allow an attempt to get a reader lock to go through even if a writer is waiting if the requesting thread already has another reader lock. I'm hoping that this can be resolved in NSPR, otherwise we may have to change DS to use the pthread_rwlock_* interfaces instead. The sleep is a temporary workaround. This issue should not arise in normal operation since the lock in question is around the backend struct, which is only modified when there is some sort of database maintenance operation (such as the reindexing task that Rob triggered it with). Nathan, thanks for the explanation, very much appreciated. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
[Freeipa-devel] [PATCH] jderose 021 Fixed try/except/finally for Python 2.4 compatability
This should fix the build failure in the daily build. From 5fad455ff41c7ab8acb8b41ea1c9c752830ce1ea Mon Sep 17 00:00:00 2001 From: Jason Gerard DeRose jder...@redhat.com Date: Thu, 15 Oct 2009 15:00:57 -0600 Subject: [PATCH] Fixed try/except/finally for Python 2.4 compatability --- ipaserver/rpcserver.py | 39 --- 1 files changed, 20 insertions(+), 19 deletions(-) diff --git a/ipaserver/rpcserver.py b/ipaserver/rpcserver.py index 06fb5ae..72f2219 100644 --- a/ipaserver/rpcserver.py +++ b/ipaserver/rpcserver.py @@ -103,25 +103,26 @@ class WSGIExecutioner(Executioner): error = None _id = None try: -self.create_context(ccache=environ.get('KRB5CCNAME')) -if ( -environ.get('CONTENT_TYPE', '').startswith(self.content_type) -and environ['REQUEST_METHOD'] == 'POST' -): -data = read_input(environ) -(name, args, options, _id) = self.unmarshal(data) -else: -(name, args, options, _id) = self.simple_unmarshal(environ) -if name not in self.Command: -raise CommandError(name=name) -result = self.Command[name](*args, **options) -except PublicError, e: -error = e -except StandardError, e: -self.exception( -'non-public: %s: %s', e.__class__.__name__, str(e) -) -error = InternalError() +try: +self.create_context(ccache=environ.get('KRB5CCNAME')) +if ( +environ.get('CONTENT_TYPE', '').startswith(self.content_type) +and environ['REQUEST_METHOD'] == 'POST' +): +data = read_input(environ) +(name, args, options, _id) = self.unmarshal(data) +else: +(name, args, options, _id) = self.simple_unmarshal(environ) +if name not in self.Command: +raise CommandError(name=name) +result = self.Command[name](*args, **options) +except PublicError, e: +error = e +except StandardError, e: +self.exception( +'non-public: %s: %s', e.__class__.__name__, str(e) +) +error = InternalError() finally: destroy_context() return self.marshal(result, error, _id) -- 1.6.3.3 ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel