Hi, Indeed, for applications, I believe that we should rather depend on defining the right EventLoopPolicy (or use the default one if most cases) and call get_event_loop() when required rather than passing the loop explicitly. Most of the time a policy is sufficient to describe how your application is designed and allows to require the loop object as late as possible (which can be very convenient). In fact, I believe that if an application is not using multiple processes, there is probably no reason to use more than one loop, especially when taking the GIL into account,
Regarding the tests, asynctest.TestCase initialize a new loop for each test and sets it as default loop at the beginning of the test. You can set TestCase.forbid_get_event_loop to enforce explicit loop passing in your tests ( http://asynctest.readthedocs.io/en/latest/asynctest.case.html#asynctest.TestCase.forbid_get_event_loop ). TestCase also performs some checks after a test case ran to assess the state of the loop after the test: for instance, did it run during the test (happens when your test case is a generator function without the @coroutine decorator), or checks that scheduled calls where invoked, etc ( http://asynctest.readthedocs.io/en/latest/asynctest.case.html#asynctest.case.asynctest.fail_on ). 2016-10-31 17:27 GMT+01:00 Guido van Rossum <gu...@python.org>: > It's an interesting problem. I would like to rephrase your conclusion: > the implicit loop should only be used when the object you are creating > has a shorter (or equal) lifetime than the loop. I would also think > that the real problem with the code example is that it creates a > variable with global lifetime. That's really an antipattern, > especially when it comes to network connections. But I'm not sure we > can blame asyncio or motor for that -- it's really class A that's to > blame. > > I would suggest different guidelines for libraries than for > applications: Libraries should be robust and always store their own > loop. This is how asyncio itself works and how aiohttp (and other > libraries you name) work. Their test suite (like asyncio's) should > enforce this. > > But applications (and fledgling libraries may do this too) should be > allowed to assume a simpler model of the world: use coroutines (async > def, await) for everything, never store the loop, happily use > get_event_loop() when you really need it. And really, you should only > need it for run_in_executor(). If you find yourself using call_later() > you probably haven't quite figured out how to use coroutines properly. > > Regarding testability of class A, I presume tests could just set a new > default loop and then re-assign A.client = > motor.motor_asyncio.AsyncIOMotorClient()? Or is that class under the > hood a singleton that just wraps more state with global lifetime? Then > that would be a problem. > > There are always exceptions to such rules, but they should all fall in > the category of advanced usage, guarding against rare failures, > "productionizing" (which I would recommend against doing in too early > a stage). > > --Guido > > On Mon, Oct 31, 2016 at 8:16 AM, Andrew Svetlov > <andrew.svet...@gmail.com> wrote: > > Hi. > > > > As people know I prefer explicit asyncio loop and pass it everywhere. > > That's how I build libraries like aiohttp and aiopg, do writing private > > source code. > > I promote this way of code writing on conferences etc. > > Push `asyncio.set_event_loop(None)` at very begin of your code to avoid > > mysterious bugs! > > > > But I feel temptation of using implicit loop -- and users of my > libraries do > > it very often. > > > > Now, after four years of working with asyncio I almost agree with it -- > if > > implicit loop is used *from coroutine*. > > > > But outside of *task context* (every asyncio coroutine is executed by a > > task, you know) it's still dangerous sometimes. > > > > Let's take a close look on the following example: > > > > import motor.motor_asyncio > > > > class A: > > client = motor.motor_asyncio.AsyncIOMotorClient() > > > > What's wrong with it? The code is untestable. AsyncIOMotorClient accepts > > is_loop parameter which should be event loop instance or None. If the > > parameter is None (as in our case) default event loop will be given. > > > > Then we run a test suite. Every test should have own loop for sake of > test > > isolation. In practice it means that every test tool creates an event > loop, > > runs test with it and closes the loop after test finishing. It's very > true > > strategy for testing async code, I know no alternatives. > > But our client was coupled with *default* loop created on module import > > stage. > > As a consequence `await client['db'].find(...)` call will be never > finished > > because client's loop is not iterated by test runner. > > > > It's not specific Motor's (async MongoDB client for Torrnado and Asyncio) > > problem -- aioes.Client, aiohttp.ClientSession and many others does the > > same. > > > > In opposite aiopg, asyncpg and aiomysql have no this problem -- their DB > > connections are created by `await connect(params)` coroutine call. > `await` > > cannot be used in global/class namespace, thus user will always call it > from > > task with proper loop. > > > > Sure, I could forbid aiohttp.ClientSession creation without current task > and > > make usage of this particular class safe (it will break backward > > compatibility for inaccurate users but better to break code early, > right). > > > > Asyncio could introduce `asyncio.get_current_loop()` function which will > > raise exception if implicit loop exists but there is no running task. > After > > that third-party libraries may switch to this function in any case where > the > > loop is required. asyncio itself could do the same. > > > > At the end. > > Accessing implicit event loop is very handy but it's safe only inside > > running task. > > Getting the loop in global namespace may lead to very strange and cryptic > > errors (most likely the next usage will just hang). > > > > Thoughts? > > > > -- > --Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>) > -- Martin <http://www.martiusweb.net> Richard www.martiusweb.net
