Klaas wrote:
> It seems clear that the import lock does not include fully-executing
> the module contents. To fix this, just import cookielib before the
What is the exact meaning of "not include fully-executing" - regarding the
examples "import cookielib" ?
Do you really mean the import statement can return without having executed the
cookielib module code fully?
(As said, a simple deadlock is not at all my problem)
> threads are spawned. Better yet, use your own locks around the
> acquisition of the opener instance (this code seems fraughtfully
> thread-unsafe--fix that and you solve other problems besides this one).
>
thanks. I will probably have to do the costly pre-import of things in main
thread and spread locks as I have also no other real idea so far.
Yet this costs the smoothness of app startup and corrupts my believe in Python
capabs of "lazy execution on demand".
I'd like to get a more fundamental understanding of the real problems than just
a general "stay away and lock and lock everything without real understanding".
* I have no real explanation why the import of a module like cookielib is not
thread-safe. And in no way I can really explain the real OS-level crashes on
dual cores/fast CPU's. Python may throw this and that, Python variable states
maybe wrong, but how can it crash on OS-level when no extension libs are
(hopefully) responsible?
* The Import Lock should be a very hard lock: As soon as any thread imports
something, all other threads are guaranteed to be out of any imports. A dead
lock is not the problem here.
* cookielib module execution code consists only of definitions and of
re.compile's. re.compile's should be thread safe?
* the things in my code patter are function local code except "opener =
urlcookie_openers.get(user)" and "urlcookie_openers[user] = opener" : Simple
dictionary accesses which are atomic from all my knowledge and experience. I
think, I have thought about enough, what could be not thread safe. The only
questionable things have to do with rare change of some globals, but this has
not at all to do with the severe problems here and could only affect e.g wrong
url2_proxy or double/unecessary re-creation of an opener, which is uncritical
in my app.
I'm still puzzled and suspect there is a major problem in Python, maybe in
win32ui or - no idea ... ?
-robert
==================================================================================
def f():
...
opener = urlcookie_openers.get(user)
if not opener:
import cookielib #<----1
cj=cookielib.CookieJar() #<----2
build_opener = urllib2.build_opener
httpCookieProcessor = urllib2.HTTPCookieProcessor(cj)
if url2_proxy:
opener = build_opener(url2_proxy,httpCookieProcessor)
else:
opener = build_opener(httpCookieProcessor)
opener.addheaders #$pycheck_no
opener.addheaders= app_addheaders
urlcookie_openers[user] = opener
ufile = opener.open(urllib2.Request(url,data,dict(headers)))
...
thread.start_new(f,())
--
http://mail.python.org/mailman/listinfo/python-list
