Re: [PyKDE] Threading problem with PyQt4 and Macosx

Kevin Cureton Wed, 21 Feb 2007 16:58:45 -0800

This may or may not be related. Alas, it was a few months ago, so the information may be irrelevant.

I converted a multi-threaded application from Qt3 to Qt4. The application uses QThreads to talk to a database and then tells the main GUI thread to populate a table widget (QListView back in Qt3). Back in Qt3, I was lazy and locked the GUI thread to let my thread do its update of the GUI. Qt4 effectively removed that option.

My first attempt at a rewrite in Qt4 experienced a deadlock situation. I've attached a program trace (stack.txt). This version used the event loop on the QThreads so they could support receiving events and signals (done by calling exec_ in the run method of the QThread). The application would run fine for a while, but then I'd open a dialog that used threads and things would lock up. It appeared to be a deadlock situation between the Qt library lock (which is called during QCoreApplication.postEvent) and Python GIL (at least that is the best I can come up with given the stack trace).

After a few days of no progress, I reworked the code to avoid the QThread event loop and handled passing data to the worker inside the thread using a thread-safe queue. While that solved the deadlock problem (since I turned off the QThread event loop), I'm now experience repeated crashes. I'm in process of trying to scope a working test case, but it does seem very sensitive to the number of threads. The crashes always end up being in QApplication.notify somewhere.

On my Mac (10.4.8 PPC), this is a typical trace from the crash:

Thread 0 Crashed:

0 QtCore 0x012f987c QObject::thread() const + 4 (icplusplus.c:28)

1 QtGui 0x0290cba8 QApplication::notify(QObject*, QEvent*) + 116 (qsystemtrayicon_mac.mm:346)

2 QtGui.so 0x0233c4a8 sipQApplication::notify(QObject*, QEvent*) + 112 (bundle1.s:283)

3 QtGui 0x02947430 QApplicationPrivate::globalEventProcessor(OpaqueEventHandlerCallRef*, OpaqueEventRef*, void*) + 1296 (qsystemtrayicon_mac.mm:346)

4 com.apple.HIToolbox 0x93207554 DispatchEventToHandlers(EventTargetRec*, OpaqueEventRef*, HandlerCallRec*) + 692

5 com.apple.HIToolbox 0x93206cac SendEventToEventTargetInternal(OpaqueEventRef*, OpaqueEventTargetRef*, HandlerCallRec*) + 372

6 com.apple.HIToolbox 0x93206b28 SendEventToEventTargetWithOptions + 40

I'm using Qt 4.2.1 and PyQt 4.1.1 (with sip 4.5.2). The deadlock problem existed in Qt 4.1.5 and PyQt 4.0.1 as well as the current config.

I'm combing through my code to make sure there aren't any objects or data that is getting shared between threads with using a mutex. So while I can't be 100% certain it isn't my code, I'm quickly coming to the belief it isn't.

The crashing behavior (and the deadlock behavior) have both been seen on Linux here as well (2.6.1x kernel with Qt 4.2.1 and PyQt 4.1.1). Likewise for Mac Intel.

As small aside. There was a change note for Qt 4.2.2 that looked interesting. In the X11 sections for changes-4.2.2 there is

* Fixed rare event loop dead-lock when posting many custom events to a receiver in another thread.

Alas, I saw this kind of problem on both Linux and Mac. Maybe Trolltech's solution wasn't comprehensive enough?

I also tried Qt 4.2.2 to no avail. I even tried various combinations of the Qt, PyQt, and sip snapshots. Still a crash. I settled back on Qt 4.2.1 and PyQt 4.1.1 while I get a scoped test case working (in between other deadlines!).

=======================================================================
thread 1 - main thread
================================================================================
#0  0x003e17a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
#1  0x00d99a1f in [EMAIL PROTECTED] () from /lib/tls/libpthread.so.0
#2  0x00214303 in PyThread_acquire_lock (lock=0x971a460, waitflag=1) at 
Python/thread_pthread.h:298
#3  0x001e3457 in PyEval_RestoreThread (tstate=0x96c01b0) at Python/ceval.c:305
#4  0x0020df54 in PyGILState_Ensure () at Python/pystate.c:500
#5  0x00119601 in initsip () from 
/swdevl/share/sip_4.5/lib/python2.4/site-packages/sip.so
#6  0x0050f707 in _ZN9sipQEventD9Ev () from 
/swdevl/share/pyqt_4.0.1/lib/python2.4/site-packages/PyQt4/QtCore.so
#7  0x0050f6b0 in sipQEvent::~sipQEvent$delete () from 
/swdevl/share/pyqt_4.0.1/lib/python2.4/site-packages/PyQt4/QtCore.so
#8  0x007b5f3d in destr_detour48.0.284 () from 
/swdevl/share/qt_4.1.4/lib/libQtCore.so.4
#9  0x007eb110 in QEventDispatcherUNIX::processEvents () from 
/swdevl/share/qt_4.1.4/lib/libQtCore.so.4
#10 0x0601cad7 in QEventDispatcherX11::processEvents () from 
/swdevl/share/qt_4.1.4/lib/libQtGui.so.4
#11 0x007b248d in QEventLoop::processEvents () from 
/swdevl/share/qt_4.1.4/lib/libQtCore.so.4
#12 0x007b2614 in QEventLoop::exec () from 
/swdevl/share/qt_4.1.4/lib/libQtCore.so.4
#13 0x007b5b93 in QCoreApplication::exec () from 
/swdevl/share/qt_4.1.4/lib/libQtCore.so.4
#14 0x05f9a0bd in QApplication::exec () from 
/swdevl/share/qt_4.1.4/lib/libQtGui.so.4
#15 0x012da1c5 in meth_QApplication_exec_ () from 
/swdevl/share/pyqt_4.0.1/lib/python2.4/site-packages/PyQt4/QtGui.so
#16 0x001afef6 in PyCFunction_Call (func=0xb7adda8c, arg=0xb7ef402c, kw=0x0) at 
Objects/methodobject.c:108
#17 0x001eac37 in PyEval_EvalFrame (f=0x96df9d4) at Python/ceval.c:3558 #18 
0x001eb4ac in PyEval_EvalCodeEx (co=0xb7b78620, globals=0xb7b6f46c,
    locals=0x0, args=0x96fcce8, argcount=1, kws=0x96fccec, kwcount=0, 
defs=0xb7b7bbb8, defcount=2, closure=0x0) at Python/ceval.c:2736
#19 0x001e93a4 in PyEval_EvalFrame (f=0x96fcb94) at Python/ceval.c:3650
#20 0x001ea259 in PyEval_EvalFrame (f=0x96f6b64) at Python/ceval.c:3640
#21 0x001eb4ac in PyEval_EvalCodeEx (co=0xb7ed40e0, globals=0xb7f0b824, 
locals=0xb7f0b824, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0,
    defcount=0, closure=0x0) at Python/ceval.c:2736
#22 0x001eb653 in PyEval_EvalCode (co=0xb7ed40e0, globals=0xb7f0b824, 
locals=0xb7f0b824) at Python/ceval.c:484
#23 0x0020f091 in PyRun_FileExFlags (fp=0x96c0008, filename=0xbff02b30 
"dialog.py", start=257, globals=0xb7f0b824,
    locals=0xb7f0b824, closeit=1, flags=0xbfea3b24) at Python/pythonrun.c:1265
#24 0x0020f9ca in PyRun_SimpleFileExFlags (fp=0x96c0008, filename=0xbff02b30 
"dialog.py", closeit=1, flags=0xbfea3b24) at Python/pythonrun.c:860
#25 0x002104c5 in PyRun_AnyFileExFlags (fp=0x96c0008, filename=0xbff02b30 
"dialog.py", closeit=1, flags=0xbfea3b24)
    at Python/pythonrun.c:664
#26 0x00217150 in Py_Main (argc=1, argv=0xbfea3bf4) at Modules/main.c:484
#27 0x080485c6 in main (argc=6, argv=0xbfea3bf4) at Modules/python.c:23


========================================================================
thread 2 - worker thread
=========================================================================
#0  0x003e17a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
#1  0x00d97b26 in pthread_cond_wait@@GLIBC_2.3.2 () from 
/lib/tls/libpthread.so.0
#2  0x006cd895 in QMutexPrivate::wait () from 
/swdevl/share/qt_4.1.4/lib/libQtCore.so.4
#3  0x006c9eb0 in QMutex::lock () from /swdevl/share/qt_4.1.4/lib/libQtCore.so.4
#4  0x007b6946 in QCoreApplication::postEvent () from 
/swdevl/share/qt_4.1.4/lib/libQtCore.so.4
#5  0x005113b5 in meth_QCoreApplication_postEvent () from 
/swdevl/share/pyqt_4.0.1/lib/python2.4/site-packages/PyQt4/QtCore.so
#6  0x001afef6 in PyCFunction_Call (func=0xb7a987cc, arg=0xb7aa778c, kw=0x0) at 
Objects/methodobject.c:108
#7  0x001eac37 in PyEval_EvalFrame (f=0x988ca54) at Python/ceval.c:3558
0xb7b1798c:      
"/swdevl/nwolfe/navajo/mtg_pub/trunk/lib/python/mtg/ui/widget/infobar/main.py"
0xb7b2b344:      "__send_activity_indicator_event"
$53 = 868

#8  0x001ea259 in PyEval_EvalFrame (f=0x97a8674) at Python/ceval.c:3640
0xb7b1798c:      
"/swdevl/nwolfe/navajo/mtg_pub/trunk/lib/python/mtg/ui/widget/infobar/main.py"
0xb7b86604:      "indicator_activity_on"
$55 = 382

#9  0x001eb4ac in PyEval_EvalCodeEx (co=0xb7b26560, globals=0xb7b1fd74, 
locals=0x0, args=0x982d260, argcount=2, kws=0x982d268, kwcount=0,
    defs=0xb7b28f38, defcount=1, closure=0x0) at Python/ceval.c:2736

#10 0x001e93a4 in PyEval_EvalFrame (f=0x982d10c) at Python/ceval.c:3650
0xb7b7a3dc:      
"/swdevl/nwolfe/navajo/mtg_pub/trunk/lib/python/mtg/ui/widget/lister/listings.py"
0xb7b7fa0c:      "_handle_restart_with_refresh"
$56 = 413

#11 0x001eb4ac in PyEval_EvalCodeEx (co=0xb7b85d20, globals=0xb7b6fbdc, 
locals=0x0, args=0xb7aa77b8, argcount=1, kws=0x0, kwcount=0, defs=0x0,
    defcount=0, closure=0x0) at Python/ceval.c:2736
#12 0x0019ec30 in function_call (func=0xb7b0ce64, arg=0xb7aa77ac, kw=0x0) at 
Objects/funcobject.c:548
#13 0x00187044 in PyObject_Call (func=0xb7b0ce64, arg=0xb7aa77ac, kw=0x0) at 
Objects/abstract.c:1756
#14 0x001917c1 in instancemethod_call (func=0xb7c4b824, arg=0xb7aa77ac, kw=0x0) 
at Objects/classobject.c:2447
#15 0x00187044 in PyObject_Call (func=0xb7c4b824, arg=0xb7ef402c, kw=0x0) at 
Objects/abstract.c:1756
#16 0x001e3758 in PyEval_CallObjectWithKeywords (func=0xb7c4b824, 
arg=0xb7ef402c, kw=0x0) at Python/ceval.c:3425
#17 0x0011c894 in initsip () from 
/swdevl/share/sip_4.5/lib/python2.4/site-packages/sip.so
#18 0x00472975 in PyQtProxy::unislot () from 
/swdevl/share/pyqt_4.0.1/lib/python2.4/site-packages/PyQt4/QtCore.so
#19 0x0047252d in PyQtProxy::qt_metacall () from 
/swdevl/share/pyqt_4.0.1/lib/python2.4/site-packages/PyQt4/QtCore.so
#20 0x007ce1ca in QObject::event () from 
/swdevl/share/qt_4.1.4/lib/libQtCore.so.4
#21 0x05f9b9aa in QApplicationPrivate::notify_helper () from 
/swdevl/share/qt_4.1.4/lib/libQtGui.so.4
#22 0x05f9a234 in QApplication::notify () from 
/swdevl/share/qt_4.1.4/lib/libQtGui.so.4
#23 0x012dac50 in sipQApplication::notify () from 
/swdevl/share/pyqt_4.0.1/lib/python2.4/site-packages/PyQt4/QtGui.so
#24 0x007b5f1d in destr_detour48.0.284 () from 
/swdevl/share/qt_4.1.4/lib/libQtCore.so.4
#25 0x007eb110 in QEventDispatcherUNIX::processEvents () from 
/swdevl/share/qt_4.1.4/lib/libQtCore.so.4
#26 0x007b248d in QEventLoop::processEvents () from 
/swdevl/share/qt_4.1.4/lib/libQtCore.so.4
#27 0x007b2614 in QEventLoop::exec () from 
/swdevl/share/qt_4.1.4/lib/libQtCore.so.4
#28 0x006cc1e0 in QThread::exec () from 
/swdevl/share/qt_4.1.4/lib/libQtCore.so.4
#29 0x00487a2a in sipQThread::sipProtect_exec () from 
/swdevl/share/pyqt_4.0.1/lib/python2.4/site-packages/PyQt4/QtCore.so
#30 0x004879c3 in meth_QThread_exec_ () from 
/swdevl/share/pyqt_4.0.1/lib/python2.4/site-packages/PyQt4/QtCore.so
#31 0x001afef6 in PyCFunction_Call (func=0xb7afb24c, arg=0xb7ef402c, kw=0x0) at 
Objects/methodobject.c:108
#32 0x001eac37 in PyEval_EvalFrame (f=0x97b4d2c) at Python/ceval.c:3558
0xb7b79934:      
"/swdevl/nwolfe/navajo/mtg_pub/trunk/lib/python/mtg/ui/utils/worker.py"
0xb7ed3314:      "run"
$57 = 103

#33 0x001eb4ac in PyEval_EvalCodeEx (co=0xb7b894a0, globals=0xb7b6fdfc, 
locals=0x0, args=0xb7a988f8, argcount=1, kws=0x0, kwcount=0, defs=0x0,
    defcount=0, closure=0x0) at Python/ceval.c:2736
#34 0x0019ec30 in function_call (func=0xb7b0c25c, arg=0xb7a988ec, kw=0x0) at 
Objects/funcobject.c:548
#35 0x00187044 in PyObject_Call (func=0xb7b0c25c, arg=0xb7a988ec, kw=0x0) at 
Objects/abstract.c:1756
#36 0x001917c1 in instancemethod_call (func=0xb7c4b874, arg=0xb7a988ec, kw=0x0) 
at Objects/classobject.c:2447
#37 0x00187044 in PyObject_Call (func=0xb7c4b874, arg=0xb7ef402c, kw=0x0) at 
Objects/abstract.c:1756
#38 0x001e3758 in PyEval_CallObjectWithKeywords (func=0xb7c4b874, 
arg=0xb7ef402c, kw=0x0) at Python/ceval.c:3425
#39 0x001195c4 in initsip () from 
/swdevl/share/sip_4.5/lib/python2.4/site-packages/sip.so
#40 0x00467802 in sipVH_QtCore_3 () from 
/swdevl/share/pyqt_4.0.1/lib/python2.4/site-packages/PyQt4/QtCore.so
#41 0x0048815c in sipQThread::run () from 
/swdevl/share/pyqt_4.0.1/lib/python2.4/site-packages/PyQt4/QtCore.so
#42 0x006ce80c in QThreadPrivate::start () from 
/swdevl/share/qt_4.1.4/lib/libQtCore.so.4
#43 0x00d95371 in start_thread () from /lib/tls/libpthread.so.0
#44 0x00cddffe in clone () from /lib/tls/libc.so.6

--kev

Kevin Cureton

Co-Founder

Mind The Gap, Inc.

[EMAIL PROTECTED]

http://www.animationpipeline.com

On Feb 21, 2007, at 2:53 PM, Matt Newell wrote:

On Wednesday 21 February 2007 14:35, Matt Newell wrote:

On Wednesday 21 February 2007 14:23, Michael Guntsche wrote:

It locked for me after I created 70 threads.

Linux version 2.6.18-3-686 (Debian 2.6.18-7) ([EMAIL PROTECTED]) (gcc
version 4.1.2 20061115 (prerelease) (Debian 4.1.1-20)) #1 SMP Mon Dec 4
16:41:14 UTC 2006

qt-4.2.2
some sip snapshot from a few months ago

10 threads seem to run fine without lockups.

If i change the write function to
def write(self, data):
#self.textEdit2.append(data)
print data

it still locks up for me once i get somewhere above 50 threads(seems
inconsistent). Possibly a race condition in the python signal sending, or
even in qt itself?

Matt

_______________________________________________
PyKDE mailing list [email protected]
http://mats.imk.fraunhofer.de/mailman/listinfo/pykde

_______________________________________________
PyKDE mailing list    [email protected]
http://mats.imk.fraunhofer.de/mailman/listinfo/pykde

Re: [PyKDE] Threading problem with PyQt4 and Macosx

Reply via email to