Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-24 Thread Adam Young
On 11/23/2014 06:13 PM, Robert Collins wrote: On WSGI - if we're in an asyncio world, I don't think WSGI has any relevance today - it has no async programming model. While is has incremental apis and supports generators, thats not close enough to the same thing: so we're going to have to port

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-24 Thread Mike Bayer
On Nov 23, 2014, at 9:24 PM, Donald Stufft don...@stufft.io wrote: There’s a long history of implicit context switches causing buggy software that breaks. As far as I can tell the only downsides to explicit context switches that don’t stem from an inferior interpreter seem to be “some

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-24 Thread Mike Bayer
On Nov 24, 2014, at 9:23 AM, Adam Young ayo...@redhat.com wrote: For pieces such as the Nova compute that talk almost exclusively on the Queue, we should work to remove Monkey patching and use a clear programming model. If we can do that within the context of Eventlet, great. If we

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-24 Thread Jay Pipes
On 11/24/2014 10:43 AM, Mike Bayer wrote: On Nov 24, 2014, at 9:23 AM, Adam Young ayo...@redhat.com wrote: For pieces such as the Nova compute that talk almost exclusively on the Queue, we should work to remove Monkey patching and use a clear programming model. If we can do that within the

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-24 Thread Doug Hellmann
On Nov 24, 2014, at 11:30 AM, Jay Pipes jaypi...@gmail.com wrote: On 11/24/2014 10:43 AM, Mike Bayer wrote: On Nov 24, 2014, at 9:23 AM, Adam Young ayo...@redhat.com wrote: For pieces such as the Nova compute that talk almost exclusively on the Queue, we should work to remove Monkey patching

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-24 Thread Mike Bayer
On Nov 24, 2014, at 12:40 PM, Doug Hellmann d...@doughellmann.com wrote: This is a good point. I’m not sure we can say “we’ll only use explicit/implicit async in certain cases because most of our apps actually mix the cases. We have WSGI apps that send RPC messages and we have other

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-24 Thread Doug Hellmann
On Nov 24, 2014, at 12:57 PM, Mike Bayer mba...@redhat.com wrote: On Nov 24, 2014, at 12:40 PM, Doug Hellmann d...@doughellmann.com wrote: This is a good point. I’m not sure we can say “we’ll only use explicit/implicit async in certain cases because most of our apps actually mix the

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-24 Thread Joshua Harlow
Doug Hellmann wrote: On Nov 24, 2014, at 12:57 PM, Mike Bayermba...@redhat.com wrote: On Nov 24, 2014, at 12:40 PM, Doug Hellmannd...@doughellmann.com wrote: This is a good point. I’m not sure we can say “we’ll only use explicit/implicit async in certain cases because most of our apps

[openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread victor stinner
Hi, I'm happy to announce you that I just finished the last piece of the puzzle to add support for trollius coroutines in Oslo Messaging! See my two changes: * Add a new aiogreen executor: https://review.openstack.org/#/c/136653/ * Add an optional executor callback to dispatcher:

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Robert Collins
On 24 November 2014 at 11:01, victor stinner victor.stin...@enovance.com wrote: Hi, I'm happy to announce you that I just finished the last piece of the puzzle to add support for trollius coroutines in Oslo Messaging! See my two changes: * Add a new aiogreen executor:

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Monty Taylor
On 11/23/2014 06:13 PM, Robert Collins wrote: On 24 November 2014 at 11:01, victor stinner victor.stin...@enovance.com wrote: Hi, I'm happy to announce you that I just finished the last piece of the puzzle to add support for trollius coroutines in Oslo Messaging! See my two changes: * Add

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Donald Stufft
On Nov 23, 2014, at 6:30 PM, Monty Taylor mord...@inaugust.com wrote: On 11/23/2014 06:13 PM, Robert Collins wrote: On 24 November 2014 at 11:01, victor stinner victor.stin...@enovance.com wrote: Hi, I'm happy to announce you that I just finished the last piece of the puzzle to add

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Robert Collins
On 24 November 2014 at 12:35, Donald Stufft don...@stufft.io wrote: For whatever it’s worth, I find explicit async io to be _way_ easier to understand for the same reason I find threaded code to be a rats nest. The co-routine style of asyncio (or Twisted’s inlineCallbacks) solves almost all

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Robert Collins
On 24 November 2014 at 12:30, Monty Taylor mord...@inaugust.com wrote: I'm not going to comment on the pros and cons - I think we all know I'm a fan of threads. But I have been around a while, so - for those who haven't been: FWIW we have *threads* today as a programming model. The

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Mike Bayer
On Nov 23, 2014, at 6:13 PM, Robert Collins robe...@robertcollins.net wrote: So - the technical bits of the plan sound fine. On WSGI - if we're in an asyncio world, *looks around*, we are? when did that happen?Assuming we’re talking explicit async. Rewriting all our code as

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Mike Bayer
On Nov 23, 2014, at 6:35 PM, Donald Stufft don...@stufft.io wrote: For whatever it’s worth, I find explicit async io to be _way_ easier to understand for the same reason I find threaded code to be a rats nest. web applications aren’t explicitly “threaded”. You get a request, load some

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Donald Stufft
On Nov 23, 2014, at 7:21 PM, Mike Bayer mba...@redhat.com wrote: Given that, I’ve yet to understand why a system that implicitly defers CPU use when a routine encounters IO, deferring to other routines, is relegated to the realm of “magic”. Is Python reference counting and garbage

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Donald Stufft
On Nov 23, 2014, at 7:29 PM, Mike Bayer mba...@redhat.com wrote: Glyph wrote a good post that mirrors my opinions on implicit vs explicit here: https://glyph.twistedmatrix.com/2014/02/unyielding.html. this is the post that most makes me think about the garbage collector analogy, re:

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Mike Bayer
On Nov 23, 2014, at 7:30 PM, Donald Stufft don...@stufft.io wrote: On Nov 23, 2014, at 7:21 PM, Mike Bayer mba...@redhat.com wrote: Given that, I’ve yet to understand why a system that implicitly defers CPU use when a routine encounters IO, deferring to other routines, is relegated

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Donald Stufft
On Nov 23, 2014, at 7:55 PM, Mike Bayer mba...@redhat.com wrote: On Nov 23, 2014, at 7:30 PM, Donald Stufft don...@stufft.io wrote: On Nov 23, 2014, at 7:21 PM, Mike Bayer mba...@redhat.com wrote: Given that, I’ve yet to understand why a system that implicitly defers CPU use when

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Mike Bayer
On Nov 23, 2014, at 8:23 PM, Donald Stufft don...@stufft.io wrote: I don’t really take performance issues that seriously for CPython. If you care about performance you should be using PyPy. I like that argument though because the same argument is used against the GCs which you like to use

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Donald Stufft
On Nov 23, 2014, at 9:09 PM, Mike Bayer mba...@redhat.com wrote: On Nov 23, 2014, at 8:23 PM, Donald Stufft don...@stufft.io wrote: I don’t really take performance issues that seriously for CPython. If you care about performance you should be using PyPy. I like that argument though