[python-tulip] Thread safety

2015-02-25 Thread Victor Stinner
Hi, On IRC, someone told me that it took him hours to understand that asyncio.Queue is not thread-safe and he expected asyncio.Queue to be thread-safe. I modified the asyncio documentation to mention in almost all classes that asyncio classes are not thread-safe. I didn't touch the doc of lock

Re: [python-tulip] Re: Thread safety

2015-02-25 Thread Victor Stinner
2015-02-25 16:23 GMT+01:00 Andrew Svetlov andrew.svet...@gmail.com: I'm OK with current asyncio locks and queue implementation, they are not-threadsafe by design explicitly. Ok, it expected this answer. It is a more a documentation issue than a design issue :-) For locks you can use `yield

[python-tulip] Re: Thread safety

2015-02-25 Thread Ludovic Gasc
On Wednesday, February 25, 2015 at 2:43:54 PM UTC+1, Victor Stinner wrote: Maybe we should modify asyncio.Queue and asyncio.Lock to make them thread-safe? For me, it's the ideal scenario, but: 1. I have no idea if it's complicated 2. If it should reduce performances when you don't use

Re: [python-tulip] Thread safety

2015-02-25 Thread Ben Darnell
On Wed, Feb 25, 2015 at 8:43 AM, Victor Stinner victor.stin...@gmail.com wrote: Hi, On IRC, someone told me that it took him hours to understand that asyncio.Queue is not thread-safe and he expected asyncio.Queue to be thread-safe. I modified the asyncio documentation to mention in almost

[python-tulip] New #asyncio IRC channel on Freenode

2015-02-25 Thread Victor Stinner
Hi, Since I get more and more questions on asyncio, I chose to create a new IRC channel on the Freenode network: #asyncio. Join us to ask questions, share new articles, etc. My nickname on IRC is haypo. Freenode : https://www.freenode.net/ Victor

Re: [python-tulip] Re: Thread safety

2015-02-25 Thread Andrew Svetlov
-1 I'm OK with current asyncio locks and queue implementation, they are not-threadsafe by design explicitly. For locks you can use `yield from lock.acquire()` (requires nested function definition for passing into `loop.call_soon_threadsafe()`) and `lock.release()`. I personally don't like to

Re: [python-tulip] Macro-benchmark with Django, Flask and AsyncIO (aiohttp.web+API-Hour)

2015-02-25 Thread Ludovic Gasc
To avoid TIME_WAIT, I used sudo sysctl -w tcp_tw_recycle=1 command. About Meinheld, I've tested a little bit in the past. From my little tests, yes it improves performance, but not at the same level as aiohttp+API-Hour version. Moreover, to my knowledge, almost nobody use that on production,

Re: [python-tulip] Re: Thread safety

2015-02-25 Thread Guido van Rossum
Yes, this is all by design. The *only* operation in all of asyncio that's thread-safe is call_soon_threadsafe(). I don't see a reason to add more hybrid methods, but I do think we should provide an example about how a thread and an event loop can communicate using some kind of queue. (There are a

Re: [python-tulip] Macro-benchmark with Django, Flask and AsyncIO (aiohttp.web+API-Hour)

2015-02-25 Thread Victor Stinner
Hi, I don't understand everything, but I agree that keep alive has a huge impact on performances. You should compare servers which all support keep alive, or disable keep alive in the client (or disable it on all servers). In a previous job, I backported httplib and xmlrpclib to get keep alive

[python-tulip] Re: Macro-benchmark with Django, Flask and AsyncIO (aiohttp.web+API-Hour)

2015-02-25 Thread Antoine Pitrou
On Wed, 25 Feb 2015 23:44:33 +0100 Ludovic Gasc gml...@gmail.com wrote: I've disabled keep_alive in api_hour, I quickly tested on agents list webservices via localhost, I've 3334.52 req/s instead of 4179 req/s, 0.233 latency average instead of 0.098 and 884 errors instead of 0 errors. It

Re: [python-tulip] Macro-benchmark with Django, Flask and AsyncIO (aiohttp.web+API-Hour)

2015-02-25 Thread Aymeric Augustin
On 26 févr. 2015, at 00:00, Antoine Pitrou solip...@pitrou.net wrote: On Wed, 25 Feb 2015 23:44:33 +0100 Ludovic Gasc gml...@gmail.com wrote: I've disabled keep_alive in api_hour, I quickly tested on agents list webservices via localhost, I've 3334.52 req/s instead of 4179 req/s, 0.233

[python-tulip] Macro-benchmark with Django, Flask and AsyncIO (aiohttp.web+API-Hour)

2015-02-25 Thread Ludovic Gasc
Hi guys, I'm permitted to send you this e-mail because some people has discussed on this mailing-list about AsyncIO benchmarks. I've published an improved version of my benchmarks: http://blog.gmludo.eu/2015/02/macro-benchmark-with-django-flask-and-asyncio.html Moreover, to reduce the risk to