Re: [PyQt] Advice for thread/process output logging, Windows debugging
On Thu, Sep 16, 2010 at 3:15 PM, Erik Janssens erik.janss...@conceptive.be wrote: Hello Steve, We've ran in more or less the same issues with the recent refactoring of Camelot, here are some things I learned (but I haven't found a decent solution yet) : - those kind of bugs tend to appear more frequent on windows than on Linux, I don't know why. But they happen on Linux as well. The best way to discover them is build stress tests and run the garbage collector explicitly in your test - try to reproduce them with stress tests on Linux, since debugging on Linux with gdb is much easier - you can as well disable the garbage collector to see if it prevents the crashes, it usually does - those crashes are related to ownership problems, see the in depth explanation of Phil : http://www.riverbankcomputing.com/pipermail/pyqt/2010-September/027705.html - there are issues when exceptions have been raised in python, since this keeps a stack trace alive with potential references to objects - some typical pythonic constructions should be avoided inside methods of Qt objects, like closures involving other Qt objects or construction of inner classes, they create difficult to track references If you learn more on this subject, I'd be very interested. I was unable to force the crash to occur on other platforms by adding gc.collect() statements in as many places as I thought might be interesting. It remains a Windows only crash, and after some recent cleanups of the threading code, it has become harder to reproduce at all, except on one slow machine of mine. Another developer submitted this patch (which connects our QDialog derived dialog's finished signal to it's deleteLater slot): http://bitbucket.org/tortoisehg/thg/changeset/f71b8c174e9f And it does indeed appear to make the crash un-reproducable.. I did 20 updates back-to-back without any hitches. Which begs the question: What is the canonical way to handle modal dialogs with PyQt4 today? Should we do explicit cleanup like this? -- Steve Borho ___ PyQt mailing listPyQt@riverbankcomputing.com http://www.riverbankcomputing.com/mailman/listinfo/pyqt
[PyQt] Advice for thread/process output logging, Windows debugging
Before I get to my questions, I want to congratulate you folks for such a tremendous toolkit. I can only imagine how much further along TortoiseHg would be today if we had selected PyQt from the start (which was my suggestion at the time). Our port from PyGtk is progressing very well, but we've run into a few snags that I would like some advice about. Both have to do with Mercurial commands running as Python code in a Qthread. We're giving Mercurial a modified user interface object that captures output messages and progress reports and emits them as PyQt signals [1]. What we've found is that this is fairly inefficient; commands run an order of magnitude slower than they do on the console, and they get progressively slower the longer the application is alive. I'm contemplating various buffering techniques to cut down on the number of signals, but I'm curious if people have other suggestions for making this more efficient. The second problem is that on Windows this setup can cause hard crashes after spewing a number of messages to the console like 'QObject::KillTimers: timers cannot be stopped from another thread'. Disabling the output and progress signals does not prevent these crashes, they seemed to be triggered by garbage collection, but I've been unable to determine which objects are the problematic ones. And I've been unable to reproduce the crash on Linux or Mac. What's the best way to debug this? I've downloaded the Python source and compiled a python_d.exe and dll, but even though I thought I built the same revision as the 2.6.5 release I have installed, it appears to reject all the compiled modules in my C:\Python26 folder. Is there a way to make PyQt emit more verbose error messages? -- Steve Borho [1] http://bitbucket.org/tortoisehg/thg/src/tip/tortoisehg/hgqt/thread.py ___ PyQt mailing listPyQt@riverbankcomputing.com http://www.riverbankcomputing.com/mailman/listinfo/pyqt
Re: [PyQt] Advice for thread/process output logging, Windows debugging
Hello Steve, We've ran in more or less the same issues with the recent refactoring of Camelot, here are some things I learned (but I haven't found a decent solution yet) : - those kind of bugs tend to appear more frequent on windows than on Linux, I don't know why. But they happen on Linux as well. The best way to discover them is build stress tests and run the garbage collector explicitly in your test - try to reproduce them with stress tests on Linux, since debugging on Linux with gdb is much easier - you can as well disable the garbage collector to see if it prevents the crashes, it usually does - those crashes are related to ownership problems, see the in depth explanation of Phil : http://www.riverbankcomputing.com/pipermail/pyqt/2010-September/027705.html - there are issues when exceptions have been raised in python, since this keeps a stack trace alive with potential references to objects - some typical pythonic constructions should be avoided inside methods of Qt objects, like closures involving other Qt objects or construction of inner classes, they create difficult to track references If you learn more on this subject, I'd be very interested. Regards, Erik On Thu, 2010-09-16 at 14:48 -0500, Steve Borho wrote: Before I get to my questions, I want to congratulate you folks for such a tremendous toolkit. I can only imagine how much further along TortoiseHg would be today if we had selected PyQt from the start (which was my suggestion at the time). Our port from PyGtk is progressing very well, but we've run into a few snags that I would like some advice about. Both have to do with Mercurial commands running as Python code in a Qthread. We're giving Mercurial a modified user interface object that captures output messages and progress reports and emits them as PyQt signals [1]. What we've found is that this is fairly inefficient; commands run an order of magnitude slower than they do on the console, and they get progressively slower the longer the application is alive. I'm contemplating various buffering techniques to cut down on the number of signals, but I'm curious if people have other suggestions for making this more efficient. The second problem is that on Windows this setup can cause hard crashes after spewing a number of messages to the console like 'QObject::KillTimers: timers cannot be stopped from another thread'. Disabling the output and progress signals does not prevent these crashes, they seemed to be triggered by garbage collection, but I've been unable to determine which objects are the problematic ones. And I've been unable to reproduce the crash on Linux or Mac. What's the best way to debug this? I've downloaded the Python source and compiled a python_d.exe and dll, but even though I thought I built the same revision as the 2.6.5 release I have installed, it appears to reject all the compiled modules in my C:\Python26 folder. Is there a way to make PyQt emit more verbose error messages? ___ PyQt mailing listPyQt@riverbankcomputing.com http://www.riverbankcomputing.com/mailman/listinfo/pyqt
