Re: [PyKDE] problem with exceptions within events
Hello, it seems that Python exceptions generated within event handlers do not get propagated correctly. We use a sys.excepthook to abort gracefully with a messagebox from our application, and this seems to work in most cases but not within event handlers. For instance: The old (ie. pre-SIP 4.2) behaviour was to call PyErr_Print() if an exception was raised in a Python re-implementation of a C++ virtual method. I never really liked this as it meant that the exception couldn't be caught with try/except. In SIP 4.2 I removed the calls to PyErr_Print() and added a check for a Python exception after each called to a C++ method. The theory being that a try/except around the corresponding Python method call would catch any exception raised. There turned out to be two problems with this... - The implementation was buggy. The SIP module was using exceptions in the meantime - it should have been saving and restoring any existing exception. With this fixed exceptions are caught properly. - It only works if the C++ method returns to allow the test for the exception to be made. The one place where this doesn't happen is QApplication.exec_loop(). Therefore you still have to rely on providing your own sys.excepthook. However, it turns out that the excepthook is only called from within PyErr_Print(). So, rather than try and deal with the two conflicting requirements (being able to catch as many exceptions as possible, and being able to detect exceptions in exec_loop()) I plan to simply revert to the pre-SIP 4.2 behaviour - unless anybody has any other bright ideas. I will do this in the next couple of days, leave it for a week for feedback, and release SIP 4.2.1. Phil ___ PyKDE mailing listPyKDE@mats.imk.fraunhofer.de http://mats.imk.fraunhofer.de/mailman/listinfo/pykde
Re: [PyKDE] problem with exceptions within events
Phil Thompson [EMAIL PROTECTED] wrote: So, rather than try and deal with the two conflicting requirements (being able to catch as many exceptions as possible, and being able to detect exceptions in exec_loop()) I plan to simply revert to the pre-SIP 4.2 behaviour - unless anybody has any other bright ideas. Is it possible to translate Python exceptions to C++ exceptions? This way, you can automatic roll out of any C++ method, even blocking ones. Then, instead of explictly checking for python exception after each c++ method call, you wrap C++ calls with try/catch, and you get the wrapped Python exceptions there. This probably requires Qt compilatoin with exception support (-exceptions). I would find totally acceptable to have the pre-SIP 4.2 behaviour in case Qt is compiled without exception support. -- Giovanni Bajo ___ PyKDE mailing listPyKDE@mats.imk.fraunhofer.de http://mats.imk.fraunhofer.de/mailman/listinfo/pykde
Re: [PyKDE] problem with exceptions within events
Phil Thompson [EMAIL PROTECTED] wrote: So, rather than try and deal with the two conflicting requirements (being able to catch as many exceptions as possible, and being able to detect exceptions in exec_loop()) I plan to simply revert to the pre-SIP 4.2 behaviour - unless anybody has any other bright ideas. Is it possible to translate Python exceptions to C++ exceptions? This way, you can automatic roll out of any C++ method, even blocking ones. Then, instead of explictly checking for python exception after each c++ method call, you wrap C++ calls with try/catch, and you get the wrapped Python exceptions there. This probably requires Qt compilatoin with exception support (-exceptions). I would find totally acceptable to have the pre-SIP 4.2 behaviour in case Qt is compiled without exception support. Having the behaviour change depending on how the user (or their distro provider) had built Qt is not a good idea. Phil ___ PyKDE mailing listPyKDE@mats.imk.fraunhofer.de http://mats.imk.fraunhofer.de/mailman/listinfo/pykde
Re: [PyKDE] problem with exceptions within events
Phil Thompson schrieb: Hello, it seems that Python exceptions generated within event handlers do not get propagated correctly. We use a sys.excepthook to abort gracefully with a messagebox from our application, and this seems to work in most cases but not within event handlers. For instance: The old (ie. pre-SIP 4.2) behaviour was to call PyErr_Print() if an exception was raised in a Python re-implementation of a C++ virtual method. I never really liked this as it meant that the exception couldn't be caught with try/except. In SIP 4.2 I removed the calls to PyErr_Print() and added a check for a Python exception after each called to a C++ method. The theory being that a try/except around the corresponding Python method call would catch any exception raised. There turned out to be two problems with this... - The implementation was buggy. The SIP module was using exceptions in the meantime - it should have been saving and restoring any existing exception. With this fixed exceptions are caught properly. - It only works if the C++ method returns to allow the test for the exception to be made. The one place where this doesn't happen is QApplication.exec_loop(). Therefore you still have to rely on providing your own sys.excepthook. However, it turns out that the excepthook is only called from within PyErr_Print(). So, rather than try and deal with the two conflicting requirements (being able to catch as many exceptions as possible, and being able to detect exceptions in exec_loop()) I plan to simply revert to the pre-SIP 4.2 behaviour - unless anybody has any other bright ideas. To recapitulate: 1.) If a Python method, called asynchronous by QApplication.exec_loop() raises an exception, the surrounding C++ method checks this and has to use PyErr_Print() to trigger the excepthook (either the default or your own excepthook). 2.) If a Python method, called synchronous, raises an exceptions, the surrounding C++ method checks this and let the calling Python code handle this exception. Why not inspect the calling frames? A method called asynchronous by QApplication.exec_loop() must have exec_loop() in one of it's outer frames, a method called synchronous outside the exec_loop doesn't have exec_loop() in its outer frames. It's just an idea, don't know if it could work. Ulli ___ PyKDE mailing listPyKDE@mats.imk.fraunhofer.de http://mats.imk.fraunhofer.de/mailman/listinfo/pykde
Re: [PyKDE] problem with exceptions within events
Phil Thompson schrieb: PyQt or any other extension wrapped with SIP could install an optional exception handler hook, that is called by SIP, indicating how to handle the exception (propagate to caller or use PyErr_Print()). It's PyErr_Print() that calls the hook, not the other way around. That hook would have to be installed *after* any application installed hook. Very messy. Sorry, I wasn't clear enough. I do not mean 'sys.excepthook' or any replacement. I mean a function implemented in PyQt called by SIP. SIP calls a method SIP checks if an exception has occured SIP checks, if the wrapped extension has installed a hook function (The module that provides exec_loop would do this) If not, SIP calls PyErr_Print() == End Otherwise SIP calls the hook funtion SIP either propagates the exception or calls PyErr_Print() depending on the return value of the hook function == End The hook function has to check, if one of the calling frames of the traceback frame contains the method exec_loop. Maybe I'm totally wrong with my idea. Ulli ___ PyKDE mailing listPyKDE@mats.imk.fraunhofer.de http://mats.imk.fraunhofer.de/mailman/listinfo/pykde
Re: [PyKDE] problem with exceptions within events
Hi Ulli, On Friday 25 February 2005 16:33, Ulrich Berning wrote: The hook function has to check, if one of the calling frames of the traceback frame contains the method exec_loop. That's exactly Phils point: looking for specific methods in the call stacks are not only ugly as hell, it's prone to errors: - Who prevents users from creating exec_loop methods, functions, ... - What other methods need this special handling?... For that reason, Phil looked for a flag based solution, but couldn't escape this can o'worms theme.. Maybe I'm totally wrong with my idea. No, but it's just not ready for prime time ;-) Pete ___ PyKDE mailing listPyKDE@mats.imk.fraunhofer.de http://mats.imk.fraunhofer.de/mailman/listinfo/pykde
Re: [PyKDE] problem with exceptions within events
On Friday 25 February 2005 3:33 pm, Ulrich Berning wrote: Phil Thompson schrieb: PyQt or any other extension wrapped with SIP could install an optional exception handler hook, that is called by SIP, indicating how to handle the exception (propagate to caller or use PyErr_Print()). It's PyErr_Print() that calls the hook, not the other way around. That hook would have to be installed *after* any application installed hook. Very messy. Sorry, I wasn't clear enough. I do not mean 'sys.excepthook' or any replacement. I mean a function implemented in PyQt called by SIP. SIP calls a method SIP checks if an exception has occured SIP checks, if the wrapped extension has installed a hook function (The module that provides exec_loop would do this) If not, SIP calls PyErr_Print() == End Otherwise SIP calls the hook funtion SIP either propagates the exception or calls PyErr_Print() depending on the return value of the hook function == End The hook function has to check, if one of the calling frames of the traceback frame contains the method exec_loop. Maybe I'm totally wrong with my idea. I can think of a way to make this work - but it's too much of a change for a .1 release. I'll restore the old behaviour for the next snapshot (you'll have to re-build PyQt) and add it to the list of things for SIP v4.3. Phil ___ PyKDE mailing listPyKDE@mats.imk.fraunhofer.de http://mats.imk.fraunhofer.de/mailman/listinfo/pykde
Re: [PyKDE] problem with exceptions within events
On Thursday 24 February 2005 00:44, Giovanni Bajo wrote: I reckon you have slightly misunderstood my problem. I had misunderstood it too. The problem is: when the exception is raised from within the overriden method of an event handler (as opposed to an exception raised from within the constructor of a subclass, or from within a slot called by a signal), the exception is *totally* ignored. There is no acknowledgement whatsoever. Nothing. Nada. I have no way to know that my code did not work. Ive attached a test script based on your original report. I cant reproduce the problem here with sip 4.0.1, PyQt 3.1.2, qt 3.3.3 I would expect the exception raised by the event handler to call our custom excepthook just like other exceptions raised in different contexts, but it would be fine if it *just* printed something on the console. Agreed. -- Toby Dickenson error.py Description: application/python about to send event in event handler Traceback (most recent call last): File error.py, line 8, in customEvent 1/0 # gratuitous exception lets see how it gets handled ZeroDivisionError: integer division or modulo by zero you should see an exception logged above about to post event, should see nothing yet about to process that event in event handler Traceback (most recent call last): File error.py, line 8, in customEvent 1/0 # gratuitous exception lets see how it gets handled ZeroDivisionError: integer division or modulo by zero you *should* see the same exception logged again. Giovanni Bajo [EMAIL PROTECTED] reports that it is omitted ___ PyKDE mailing listPyKDE@mats.imk.fraunhofer.de http://mats.imk.fraunhofer.de/mailman/listinfo/pykde
Re: [PyKDE] problem with exceptions within events
Toby Dickenson [EMAIL PROTECTED] wrote: Ive attached a test script based on your original report. I cant reproduce the problem here with sip 4.0.1, PyQt 3.1.2, qt 3.3.3 Thanks for doing this. The problem can be reproduced if you convert your app.processEvents() call to app.exec_loop(). In that case, I cannot see the traceback anymore. (Again, I'm using Qt 3.3.4, PyQt 3.14, SIP 4.2). There is also another difference: on my computer, the script aborts whenever it hits the first exception. For instance, it aborts on the app.sendEvent() call, and I never see the following string you should see an exception above on the screen. Thus, to reproduce the problem with exec_loop(), I have commented out the first sendEvent() call. I don't know if it is related, but my Qt is compiled with -no-exceptions. I know that Python and C++ exceptions are not related, but I don't know if SIP converts exceptions on bindings boundaries (I know other similar wrapping tools do that). Could Phil enlighten us on this? To sum it up. With this script: = import sys def foo(*args): print except hook sys.excepthook = foo from qt import * app=QApplication([]) class W(QWidget): def customEvent(self, e): print sys.stderr,'in event handler' 1/0 # gratuitous exception lets see how it gets handled w = W() #print sys.stderr,'about to send event' #app.sendEvent(w, QCustomEvent(QEvent.User)) #print sys.stderr,'you should see an exception logged above\n\n' print sys.stderr, 'about to post event, should see nothing yet' app.postEvent(w, QCustomEvent(QEvent.User)) print sys.stderr, 'about to process that event' app.exec_loop() print sys.stderr,'you *should* see the same exception logged again.' print sys.stderr,'Giovanni Bajo [EMAIL PROTECTED] reports that it is omitted\n\n' print 'done' = I get this output: = about to post event, should see nothing yet about to process that event in event handler = -- Giovanni Bajo ___ PyKDE mailing listPyKDE@mats.imk.fraunhofer.de http://mats.imk.fraunhofer.de/mailman/listinfo/pykde
Re: [PyKDE] problem with exceptions within events
Toby Dickenson schrieb: On Thursday 24 February 2005 00:44, Giovanni Bajo wrote: I reckon you have slightly misunderstood my problem. I had misunderstood it too. The problem is: when the exception is raised from within the overriden method of an event handler (as opposed to an exception raised from within the constructor of a subclass, or from within a slot called by a signal), the exception is *totally* ignored. There is no acknowledgement whatsoever. Nothing. Nada. I have no way to know that my code did not work. Ive attached a test script based on your original report. I cant reproduce the problem here with sip 4.0.1, PyQt 3.1.2, qt 3.3.3 I would expect the exception raised by the event handler to call our custom excepthook just like other exceptions raised in different contexts, but it would be fine if it *just* printed something on the console. Agreed. I have attached a more practical example, that shows the problem in more detail. If you use sendEvent(), everything is fine (at least for me) and the exepthook function is called. This is because the event is immediately processed. With postEvent(), the event processing is deferred until control returns to the main loop and the excepthook function is not called. I think, this has to do with the fact, that the ownership of the event is taken by the post event queue and the event is deleted once it has been posted. In your example you first called sendEvent(), then postEvent() and then processEvents(). By calling postEvent() followed by processEvents(), you simulate the behavior of sendEvent(). Uncomment the processEvents() call in my script and you will get the same result as with sendEvent(). Ulli #!/usr/bin/env python ## # Imports# ## import sys, string, traceback, qt ## # Excepthook catching unhandled exceptions # ## def my_excepthook(exc_type, exc_value, exc_traceback): sys.stderr.write(in my_excepthook()\n) trb = string.join(traceback.format_exception(exc_type, exc_value, exc_traceback)) id =qt.QMessageBox.critical(None, Unhandled Exception caught, trb, qt.QMessageBox.Ignore, qt.QMessageBox.Abort) if id == qt.QMessageBox.Abort: sys.stderr.write(Aborting application...\n) qt.qApp.exit() else: sys.stderr.write(Continuing application... (may give you unexpected behavior)\n) ## # Main widget# ## class MainWidget(qt.QWidget): def __init__(self, parent=None, name=None, fl=0): qt.QWidget.__init__(self, parent, name, fl) # Create custom event type self.myEventType = qt.QEvent.Type(qt.QEvent.User) # Create the widget layout self.vbox = qt.QVBoxLayout(self, 11, 6) # A radio button group to select the event method self.groupMethod = qt.QButtonGroup(1, qt.QGroupBox.Horizontal, Event Delivery Method, self) self.groupMethod.setExclusive(True) self.vbox.addWidget(self.groupMethod) self.methodSend = qt.QRadioButton(sendEvent(), self.groupMethod) self.methodSend.setChecked(True) self.methodPost = qt.QRadioButton(postEvent(), self.groupMethod) # A buuton group with a push button self.groupEvent = qt.QButtonGroup(1, qt.QGroupBox.Horizontal, Generate Custom Event, self) self.vbox.addWidget(self.groupEvent) self.button = qt.QPushButton(Raises ZeroDivisonError, self.groupEvent) self.connect(self.button, qt.SIGNAL(clicked()), self.slotButton) # Push button slot def slotButton(self): if self.methodPost.isChecked(): qt.qApp.postEvent(self, qt.QCustomEvent(self.myEventType)) #qt.qApp.processEvents() else: qt.qApp.sendEvent(self, qt.QCustomEvent(self.myEventType)) # Custom event handler def customEvent(self, e): sys.stderr.write(in customEvent()\n) if e.type() == self.myEventType: x = 1 / 0 ## # Main programm #
Re: [PyKDE] problem with exceptions within events
Ulrich Berning [EMAIL PROTECTED] wrote: I have attached a more practical example, that shows the problem in more detail. Thanks. I would call this a very serious bug. BTW, let me notice that this does *not only* happen with an explicit postEvent() call. Even though I am unable to reproduce it in a short snippet at the moment, it happens also for other events (e.g. paintEvent) in my application. Just a few minutes ago I had to use the debugger and trace step by step to find a typo error (raising an exception) within a paintGL() method, because it was *not* reported at all. There is something weird going on. -- Giovanni Bajo ___ PyKDE mailing listPyKDE@mats.imk.fraunhofer.de http://mats.imk.fraunhofer.de/mailman/listinfo/pykde
Re: [PyKDE] problem with exceptions within events
Giovanni Bajo [EMAIL PROTECTED] wrote: I have attached a more practical example, that shows the problem in more detail. Thanks. I would call this a very serious bug. BTW, let me notice that this does *not only* happen with an explicit postEvent() call. Even though I am unable to reproduce it in a short snippet at the moment, it happens also for other events (e.g. paintEvent) in my application. Just a few minutes ago I had to use the debugger and trace step by step to find a typo error (raising an exception) within a paintGL() method, because it was *not* reported at all. There is something weird going on. Some more information: I tried downgrading to my previous configuration (SIP 4.1.1, PyQt 3.13, Qt 3.3.2) and the problem does not occur anymore. I verified that it does not occur in either my application or Ulrich's testcase error_ui.py (after having tweaked for execution with old PyQt which does not have enum types), so I will assume that error_ui is a faithful reduction of my problem. This confirms that this bug is a regression introduced between PyQt 3.13 and 3.14 (or SIP 4.1.1 and 4.2). -- Giovanni Bajo ___ PyKDE mailing listPyKDE@mats.imk.fraunhofer.de http://mats.imk.fraunhofer.de/mailman/listinfo/pykde
Re: [PyKDE] problem with exceptions within events
Giovanni Bajo [EMAIL PROTECTED] wrote: I have attached a more practical example, that shows the problem in more detail. Thanks. I would call this a very serious bug. BTW, let me notice that this does *not only* happen with an explicit postEvent() call. Even though I am unable to reproduce it in a short snippet at the moment, it happens also for other events (e.g. paintEvent) in my application. Just a few minutes ago I had to use the debugger and trace step by step to find a typo error (raising an exception) within a paintGL() method, because it was *not* reported at all. There is something weird going on. Some more information: I tried downgrading to my previous configuration (SIP 4.1.1, PyQt 3.13, Qt 3.3.2) and the problem does not occur anymore. I verified that it does not occur in either my application or Ulrich's testcase error_ui.py (after having tweaked for execution with old PyQt which does not have enum types), so I will assume that error_ui is a faithful reduction of my problem. This confirms that this bug is a regression introduced between PyQt 3.13 and 3.14 (or SIP 4.1.1 and 4.2). That's the information I was waiting for - thanks. There was a change that was intended to make sure that exceptions raised in Python re-implementations would ripple through without having to do anything with exception hooks. I need to check, but I suspect that the exception is being cleared too soon. Phil ___ PyKDE mailing listPyKDE@mats.imk.fraunhofer.de http://mats.imk.fraunhofer.de/mailman/listinfo/pykde
