Hi Justin,
I think in the long run the event loop (and everything in asyncio) needs to
be much more careful with BaseExceptions. Their treatment is uneven -- if
we're writing "except Exception" they are not caught, but when we're
writing "finally" or "except: ...; raise" they are caught. I think the
right thing to do is to tread BaseException the same way we're treating
Exception, *except* that in cases where we catch it and continue running
the event loop we should probably still break out of the event loop right
away. So a KeyboardError would still break out of the event loop (without
executing the remaining scheduled callbacks; or perhaps after, similar to
the way the new stop() behaves), but the event loop would be in a
consistent state -- if you were to restart the loop the KeyboardError would
be delivered to the callback to which an Exception class would be delivered.
Hope this description makes sense.
Maybe you're interested in improving asyncio by submitting a patch, given
that you're running into this problem and so have a motivation as well as a
use case that can drive the improvements?
On Fri, Dec 4, 2015 at 12:54 AM, Justin Mayfield <too...@gmail.com> wrote:
> I have an experimental library where I manage the lifecycle of event loops
> for purposes of allowing async routines to operate inside a classical
> generator (among other things). I create a new event loop for each use of
> my routine and upon completion or GC this event loop is closed.
>
> My recent testing has left me a bit perplexed as to the right way to deal
> with KeyboardInterrupt and possibly other BaseException exceptions that
> halt execution of an event loop. I've read a few posts on this list
> suggesting that the basic rule of thumb employed within asyncio itself, is
> to only catch Exception because BaseException is generally considered
> unrecoverable (or coro protocol as in GeneratorExit). However I seem to
> have found several instances where this rule is not followed and in the
> except clause, further calls are made that expect the event loop to be in
> good standing.
>
> Because I close the event loop when my routine is done/interrupted the
> eventual GeneratorExit exception thrown by the garbage collector triggers
> some cleanup/close routines in asyncio itself (as well as aiohttp) that
> eventually find themselves staring at RuntimeError('Event loop is closed').
>
> My question to the community is if I'm witnessing bugs in these places
> where naked except clauses are used to perform cleanup/closing actions on
> an event loop or if I'm fundamentally doing something wrong. I hope it's
> not the latter as I've been pretty happy with the results of my experiment
> baring these corner cases.
>
> [Code in question is here:
> https://github.com/mayfield/cellulario/blob/master/cellulario/iocell.py]
>
> Example traceback..
>
> Traceback (most recent call last):
>
> File
> "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/base_events.py",
> line 692, in _create_connection_transport
>
> yield from waiter
>
> GeneratorExit
>
>
> During handling of the above exception, another exception occurred:
>
>
> Traceback (most recent call last):
>
> File "/Users/mayfield/project/syndicate/syndicate/adapters/async.py",
> line 74, in request
>
> result = yield from asyncio.wait_for(r, timeout, loop=self.loop)
>
> File
> "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/tasks.py",
> line 359, in wait_for
>
> return (yield from fut)
>
> File
> "/usr/local/lib/python3.5/site-packages/aiohttp-0.18.3-py3.5-macosx-10.10-x86_64.egg/aiohttp/client.py",
> line 456, in __iter__
>
> resp = yield from self._coro
>
> File
> "/usr/local/lib/python3.5/site-packages/aiohttp-0.18.3-py3.5-macosx-10.10-x86_64.egg/aiohttp/client.py",
> line 173, in _request
>
> conn = yield from self._connector.connect(req)
>
> File
> "/usr/local/lib/python3.5/site-packages/aiohttp-0.18.3-py3.5-macosx-10.10-x86_64.egg/aiohttp/connector.py",
> line 289, in connect
>
> transport, proto = yield from self._create_connection(req)
>
> File
> "/usr/local/lib/python3.5/site-packages/aiohttp-0.18.3-py3.5-macosx-10.10-x86_64.egg/aiohttp/connector.py",
> line 557, in _create_connection
>
> server_hostname=hinfo['hostname'] if sslcontext else None)
>
> File
> "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/base_events.py",
> line 669, in create_connection
>
> sock, protocol_factory, ssl, server_hostname)
>
> File
> "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/base_events.py",
> line 694, in _create_connection_transport
>
> transport.close()
>
> File
> "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/selector_events.py",
> line 566, in close
>
> self._loop.call_soon(self._call_connection_lost, None)
>
> File
> "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/base_events.py",
> line 453, in call_soon
>
> handle = self._call_soon(callback, args)
>
> File
> "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/base_events.py",
> line 462, in _call_soon
>
> self._check_closed()
>
> File
> "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/base_events.py",
> line 289, in _check_closed
>
> raise RuntimeError('Event loop is closed')
>
>
--
--Guido van Rossum (python.org/~guido)
