On 10/12/2012 06:21 PM, Rob Crittenden wrote: > John Dennis wrote: >> On 10/12/2012 11:20 AM, Martin Kosek wrote: >>> Hello, >>> >>> I was investigating global unit test failure on Fedora 18 for most of >>> today, I >>> would like to share results I found so far. >>> >>> Unit test and its related scripts on F18 now reports NSS BUSY >>> exception, just >>> like this one: >>> >>> # ./make-testcert >>> Traceback (most recent call last): >>> File "./make-testcert", line 134, in <module> >>> sys.exit(makecert(reqdir)) >>> File "./make-testcert", line 111, in makecert >>> add=True) >>> File "./make-testcert", line 68, in run >>> result = self.execute(method, *args, **options) >>> File "/root/freeipa-master2/ipalib/backend.py", line 146, in execute >>> raise error #pylint: disable=E0702 >>> ipalib.errors.NetworkError: cannot connect to >>> 'http://vm-042.idm.lab.bos.redhat.com/ipa/session/xml': [Errno -8053] >>> (SEC_ERROR_BUSY) NSS could not shutdown. Objects are still in use. >>> >>> Something In F18 must have changed, this worked before... But leaked >>> NSSConnection objects without proper close() now ends with the >>> exception above. >>> >>> In case of make-testcert script, the exception is raised because the >>> script >>> does the following procedure: >>> >>> 1) connect, do one command >>> 2) disconnect >>> 3) connect, do second command >>> >>> However, during disconnect, NSSConnection is leaked which makes NSS very >>> uncomfortable during second connection atempt (and nss_shutdown()). I >>> managed >>> to fix this issue with attached patch. ./make-testcert or "./make-test >>> tests/test_xmlrpc/test_group_plugin.py" works fine now. >>> >>> But global "./make-test" still fails, I think there is some remaining >>> NSSConnection leak, I suspect there is something wrong with how we use >>> our >>> context (threading.local object). It looses a connection or some other >>> thread >>> invoked in ldap2 module may be kicking in, here is my debug output: >>> >>> CONTEXT[xmlclient] = <ipalib.request.Connection object at 0x9a1f5ec> >>> >>> Test a simple LDAP bind using ldap2 ... SKIP: No directory manager >>> password in >>> /root/.ipa/.dmpw >>> Test the `ipaserver.rpcserver.jsonserver.unmarshal` method. ... ok >>> tests.test_ipaserver.test_rpcserver.test_session.test_mount ... CONTEXT >>> 150714476: GET languages >>> >>> CONTEXT[xmlclient] = None >>> >>> The connection is in the context, but then something happens and it is >>> gone. >>> Then, unit tests try to connect again and NSS fails. >>> >>> I would be really glad if somebody with a knowledge of NSS or how >>> threads in >>> Python/IPA work could give me some advice... >> >> O.K. I'll take a look at it. I seem to recall Rob looked into something >> similar a couple of days ago. Rob, do you have any additional >> information to share? >> >> > > Nothing to add. I looked at this briefly yesterday but nothing jumped out at > me. I think this may be unrelated to the previous problem I investigated. It > very well could be we are not closing a connection somewhere. > > rob
Yes, we are not. I fixed it in one place (patch in my original e-mail). With the patch, one could run at least some parts of the unit test. I assume we are simply not treating initializing and shutting down of NSS correctly in some places (also with regards to threads), but I have not found a fix yet. Martin _______________________________________________ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel