I get python crashes and (in better cases) strange Python exceptions when (in
most cases) importing and using cookielib lazy on demand in a thread.
It is mainly with cookielib, but remember the problem also with other imports
(e.g. urllib2 etc.).
And again very often in all these cases where I get weired Python exceptions,
the problem is around re-functions - usually during re.compile calls during
import (see some of the exceptions below). But not only.
Very strange: The errors occur almost only on machines with dual core/multi
processors - and very very rarely on very fast single core machines (>3GHz).
I'm using Python2.3.5 on Win with win32ui (build 210) - the cookielib taken
from Python 2.5.
I took care that I'm not starting off thread things or main application loop
etc. during an import (which would cause a simple & explainable deadlock freeze
on the import lock)
With real OS-level crashes I know from user reports (packaged app), that these
errors occur very likely early after app start - thus when lazy imports are
likely to do real execution.
I researched this bug for some time. I think I can meanwhile exclude
(ref-count, mem.leak) problems in win32ui (the only complex extension lib I
use) as cause for this. All statistics point towards import problems.
Any ideas?
Are there problems known with the import lock (Python 2.3.5) ?
(I cannot easily change from Python 2.3 and it takes weeks to get significant
feedback after random improvements)
-robert
PS:
The basic pattern of usage is:
==================
def f():
...
opener = urlcookie_openers.get(user)
if not opener:
import cookielib #<----1
cj=cookielib.CookieJar() #<----2
build_opener = urllib2.build_opener
httpCookieProcessor = urllib2.HTTPCookieProcessor(cj)
if url2_proxy:
opener = build_opener(url2_proxy,httpCookieProcessor)
else:
opener = build_opener(httpCookieProcessor)
opener.addheaders #$pycheck_no
opener.addheaders= app_addheaders
urlcookie_openers[user] = opener
ufile = opener.open(urllib2.Request(url,data,dict(headers)))
...
thread.start_new(f,())
=========================
Symptoms:
__________
sometimes ufile is None and other weired invalid states.
typical Python exceptions when in better cases there is no OS-level crash:
---------
# Attributes randomly missing like:
#<----2
"AttributeError: \'module\' object has no attribute \'CookieJar\'\\n"]
---------
# weired invalid states during computation like:
#<----1
... File "cookielib.pyo", line 184, in ?\\n\', \' File
"sre.pyo", line 179, in compile\\n\', \' File "sre.pyo", line 228, in
_compile\\n\', \' File
"sre_compile.pyo", line 467, in compile\\n\', \' File "sre_parse.pyo", line
624, in parse\\n\', \'
File "sre_parse.pyo", line 317, in _parse_sub\\n\', \' File "sre_parse.pyo",
line 588, in
_parse\\n\', \' File "sre_parse.pyo", line 92, in closegroup\\n\',
\'ValueError: list.remove(x): x
not in list\\n\']
...
'windows', "(5, 1, 2600, 2, 'Service Pack 2')/NP=2")
---------
#<----1
File "cookielib.pyo", line 116, in ?\\n\', \' File "sre.pyo", line 179, in
compile\\n\', \' File "sre.pyo", line 228, in _compile\\n\', \' File
"sre_compile.pyo", line 467, in compile\\n\', \' File "sre_parse.pyo", line
624, in parse\\n\', \' File "sre_parse.pyo", line 317, in _parse_sub\\n\', \'
File "sre_parse.pyo", line 494, in _parse\\n\', \' File "sre_parse.pyo", line
140, in __setitem__\\n\', \'IndexError: list assignment index out of range\\n\']
('windows', "(5, 1, 2600, 2, 'Service Pack 2')/NP=2"
---------
# weired errors in other threads:
# after dlg.DoModal() in main thread
File "wintools.pyo", line 115, in PreTranslateMessage\\n\', \'TypeError: an
integer is required\\n\']
('windows', "(5, 1, 2600, 2, 'Service Pack 2')/NP=2")
---------
# after win32ui.PumpWaitingMessages(wc.WM_PAINT, wc.WM_MOUSELAST) in main thread
\'TypeError: argument list must be a tuple\\n\'
...
--
http://mail.python.org/mailman/listinfo/python-list
