Martin Kosek wrote:
On 10/12/2012 06:21 PM, Rob Crittenden wrote:
John Dennis wrote:
On 10/12/2012 11:20 AM, Martin Kosek wrote:
Hello,
I was investigating global unit test failure on Fedora 18 for most of
today, I
would like to share results I found so far.
Unit test and its related scripts on F18 now reports NSS BUSY
exception, just
like this one:
# ./make-testcert
Traceback (most recent call last):
File "./make-testcert", line 134, in <module>
sys.exit(makecert(reqdir))
File "./make-testcert", line 111, in makecert
add=True)
File "./make-testcert", line 68, in run
result = self.execute(method, *args, **options)
File "/root/freeipa-master2/ipalib/backend.py", line 146, in execute
raise error #pylint: disable=E0702
ipalib.errors.NetworkError: cannot connect to
'http://vm-042.idm.lab.bos.redhat.com/ipa/session/xml': [Errno -8053]
(SEC_ERROR_BUSY) NSS could not shutdown. Objects are still in use.
Something In F18 must have changed, this worked before... But leaked
NSSConnection objects without proper close() now ends with the
exception above.
In case of make-testcert script, the exception is raised because the
script
does the following procedure:
1) connect, do one command
2) disconnect
3) connect, do second command
However, during disconnect, NSSConnection is leaked which makes NSS very
uncomfortable during second connection atempt (and nss_shutdown()). I
managed
to fix this issue with attached patch. ./make-testcert or "./make-test
tests/test_xmlrpc/test_group_plugin.py" works fine now.
But global "./make-test" still fails, I think there is some remaining
NSSConnection leak, I suspect there is something wrong with how we use
our
context (threading.local object). It looses a connection or some other
thread
invoked in ldap2 module may be kicking in, here is my debug output:
CONTEXT[xmlclient] = <ipalib.request.Connection object at 0x9a1f5ec>
Test a simple LDAP bind using ldap2 ... SKIP: No directory manager
password in
/root/.ipa/.dmpw
Test the `ipaserver.rpcserver.jsonserver.unmarshal` method. ... ok
tests.test_ipaserver.test_rpcserver.test_session.test_mount ... CONTEXT
150714476: GET languages
CONTEXT[xmlclient] = None
The connection is in the context, but then something happens and it is
gone.
Then, unit tests try to connect again and NSS fails.
I would be really glad if somebody with a knowledge of NSS or how
threads in
Python/IPA work could give me some advice...
O.K. I'll take a look at it. I seem to recall Rob looked into something
similar a couple of days ago. Rob, do you have any additional
information to share?
Nothing to add. I looked at this briefly yesterday but nothing jumped out at
me. I think this may be unrelated to the previous problem I investigated. It
very well could be we are not closing a connection somewhere.
rob
Yes, we are not. I fixed it in one place (patch in my original e-mail). With
the patch, one could run at least some parts of the unit test. I assume we are
simply not treating initializing and shutting down of NSS correctly in some
places (also with regards to threads), but I have not found a fix yet.
If you modify tests/test_xmlrpc/xmlrpc_test.py and remove
request.destroy_context() from tearDown then the tests will pass. But
this shouldn't be necessary, we really want to clean up the context
between requests.
I'm guessing that we're being affected by an external change. It could
be in a number of places: httplib, xmlrpclib, NSS, etc.
rob
_______________________________________________
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel
