[issue25489] sys.exit() caught in async event loop exception handler

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25489] sys.exit() caught in async event loop exception handler

[Guido van Rossum]

Guido van Rossum  added the comment:

Andrew, thanks for explaining this.

The key thing I was missing was that the root cause of the problem is that 
Future.__del__ is trying to log an error about the un-awaited task by calling 
the exception handler directly. That actually feels a little dodgy.

This is why I'm not yet comfortable with (d). Looking at 
call_exception_handler(), whether it calls the default handler or a custom 
handler, it explicitly checks for SystemExit and KeyboardInterrupt and 
re-raises those. And only those -- everything ends up logging an error.

Which makes me wonder. Maybe that error in Future.__del__ should not call any 
exception handler at all, but just call logger.error()? Or maybe Future.__del__ 
should catch exceptions coming out of there and log an error? Maybe a modified 
version of (d), but only implemented in Future.__del__, not in 
call_exception_handler()?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25489] sys.exit() caught in async event loop exception handler

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Guido, perhaps you had problems with the problem detection because the asyncio 
uses _asyncio C Extesions by default. It drops some calls from the python stack 
trace.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25489] sys.exit() caught in async event loop exception handler

[Andrew Svetlov]

Andrew Svetlov  added the comment:

I can describe what happens with test_sys_exit_in_exception_handler.py 

1. The 'boom' task raises an exception.
2. The task is not awaited, Future.__del__ calls the exception handler with 
'Task exception was never retrieved' message.
3. The custom handler raises SystemExit.
4. SystemExit bubbles up and swallowed by __del__, the __del__ method cannot 
re-raise.

The question is: what is the behavior expected?
a) Now an exception raised by a custom exception handler is swallowed in this 
particular case (but is propagated if `loop.call_exception_handler()` is called 
from a function other than __del__).
b) Yuri suggested re-schedule an exception generated by 
`loop.call_exception_handler` by `loop.call_soon()`.  asyncio.Handle catches it 
and... calls `call_exception_handler()` with 'Exception in callback ...' 
message.  At the end, we have an endless recursion.
c) asyncio loop can call `loop.stop()` if an exception is raised by 
`loop.call_exception_handler()` from __del__.  I think this behavior is 
terrible: a subtle error can terminate asyncio program.
d) Assume that a custom exception handler should not raise an exception. Catch 
all exceptions in `call_exception_handler`, call sys.unraisablehook(), and 
suppress the exception.

I believe that d) is the best thing that we can do here.

I can prepare a fix if we agree on the solution.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25489] sys.exit() caught in async event loop exception handler

[Gregory P. Smith]

Change by Gregory P. Smith :


--
title: sys.exit() caught in exception handler -> sys.exit() caught in async 
event loop exception handler

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com