I was contemplating this just now. Let's say you want to stream a (huge) file from disk. You can fire off a reader in a separate thread, using call_in_executor. Now this reader needs to be able to feed the client chunks of the file as they are read, and also you'd want backpressure. A thread-safe and async-aware queue (with limited capacity) seems like the simplest solution by far. It'd be great if something like this was available from the standard library; I'm not entirely sure how to create one myself.
What would the API of this queue look like? Would you need a put/get version that blocks the thread, and a separate put/get version that blocks a task? On Wednesday, February 25, 2015 at 2:43:54 PM UTC+1, Victor Stinner wrote: > > Hi, > > On IRC, someone told me that it took him hours to understand that > asyncio.Queue is not thread-safe and he expected asyncio.Queue to be > thread-safe. I modified the asyncio documentation to mention in almost > all classes that asyncio classes are not thread-safe. > > I didn't touch the doc of lock classes (ex: asyncio.Lock), because I > don't know if they are thread safe or not. Since locks should be used > with "yield from lock", it's not easy to combine them with > loop.call_soon_threadsafe(). > > Maybe we should modify asyncio.Queue and asyncio.Lock to make them > thread-safe? > > What do you think? > > Victor >
