Fortunately, Guido has jumped in. He's the authority on asyncio. I recommend you consider his advice very carefully. Here are a couple more commments that I had already written when I saw his post.
Celelibi writes: > But that doesn't mean we can't try to design asyncio to be more > thread-friendly. True, lack of an example to emulate doesn't mean we can't try ... but the fact that asyncio and threads are based on completely different models of concurrency likely does, in combination with that lack. Strong suggestion that "here be dragons". > I'm working on switching out a thread-based client protocol lib for > an asyncio-based lib instead. Have you tried to do this with an alternative to asyncio such as curio (by David Beazley, IIRC) or trio (by Nathaniel Smith)? They provide simpler APIs, and Nathaniel claims that they also provide a concurrency model that's far easier to understand. They're still async def/await-based, but they have a property Nathaniel calls "respect for causality" that seems very useful in understanding programs. > But if it did, I'd end up calling call_soon_threadsafe from the thread > that runs the event loop and wait for the result. Which would just lock > up. If I understand correctly, this is the kind of thing that "respect for causality" helps to identify and to some extent prevent. > Another issue I found is that the client protocol lib class isn't always > instanciated by the same thread, and its constructor indirectly > instanciate asyncio objects (like Event). The relations among the threading library, the asyncio library, and the client protocol library in your software are quite unclear to me. But this sounds very bad for your "divide and conquer" strategy. Asyncio is fundamentally a single-process, single-threaded, single-event-loop approach to handling concurrency. If the asyncio part of your program can ignore what other threads are doing, you might be OK, but this kind of coupling means you can't, and will need to do things like: > In the end, its start method will start the event loop in its own > thread. With the deprecation of the loop argument, Event() finds > the loop it should use by calling get_event_loop() in its > constructor. I therefore had to add a call set_event_loop(), even > though this event loop will never run in this thread. That's a > pretty ugly hack. But it's intended to be a temporary one, no? This is the kind of thing you need to do when you have multiple incompatible models of computation running in the same system. > The deprecation of the loop arguments looks like an incentive to create > the asyncio objects in the threads that use them. It is. A major point (perhaps the whole point) of asyncio is to provide that "single-process, single-threaded, single-event-loop approach" which nevertheless is extremely fast and scalable for I/O-bound programs. If you can't get to the point where all of your interactions with the threaded code can be done in a single thread, asyncio is very unlikely to be a good model for your system. > Which seems pretty crazy to me as a whole. You should think carefully about that feeling. If it's just because you're temporarily in concurrency model hell, you'll get over that once the port to asyncio is done. But if you really need multiple threads because some code exposes them and you can't port that code to asyncio, you need to consider whether asyncio can do what you need. Steve _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/CFSVWPMXSNP6ABLGBSOVVKZMBKNB2ER3/ Code of Conduct: http://python.org/psf/codeofconduct/
