Hi Michael, ++ On all points. Over the past few days while researching all this, I went from being in the middle to the Eventlet side. Another couple points to make:
* We are not actually using twisted for what it is powerful for - non-blocking I/O handling. The AMQP and Redis libs are still using blocking sockets and queues are being polled with a timer every 100ms with sync operations. There is still some work to be done to use it properly (replacing libs with twisted versions, restructure app). * The places we are using twisted currently may not even need this, using a multi-process or multi-thread model may make more sense. See active thread discussing this right now. * The application where we do need twisted/eventlet - nova-api - is currently still on tornado, so we can start using eventlet there and don't need to worry about converting twisted code already written (no wasted resources). * We have eventlet experts and a large scale app already on hand with swift, so we can bug them with any questions. :) -Eric On Wed, Aug 04, 2010 at 12:03:49PM -0400, Michael Gundlach wrote: > All, > I propose that we go ahead and decide to use Eventlet. We've all had a > chance to talk, the decision isn't going to make itself, and it seems like > we're coalescing around Eventlet: > * In a long discussion about how to keep our code loosely coupled and > easy to maintain (see Architecture For Shared Components on the > openstack list), we concluded that WSGI would be a great solution, and > eday has prototyped an Eventlet-powered WSGI app talking AMQP to > Rabbit. The Twisted WSGI approach apparently isn't stable or > standard, but WSGI works just as you'd expect in Eventlet. > * As hazmat points out in the "twisted v. eventlet" thread, you can't > use pdb with Twisted, while pdb works just as you'd hope in Eventlet. > * hazmat's experience during his lightning talk (see the same thread) is > a good datapoint verifying that people unfamiliar with either project > will probably prefer to work with Eventlet over Twisted: good for > recruiting. > * It's a weaker argument, but it's really hard to find any comparisons > between Twisted and Eventlet on the web that say Twisted is preferable > (has anyone found any?) > And as always, there's the argument that Eventlet code is easier to read > and understand, and therefore to maintain. > I'm going to be working on adding the RS API to Nova, and I'm going to > proceed in Eventlet. If anyone feels strongly that we must use Twisted > instead, speak up; otherwise, shall we call this decision made? > trying-to-build-consensus-ly yrs, > Michael > On Tue, Aug 3, 2010 at 8:13 AM, Soren Hansen <[email protected]> > wrote: > > On Mon, Aug 02, 2010 at 05:21:23PM -0500, Eric Day wrote: > > I took a closer look at the inner workings of how Twisted is currently > > being used, and I think there is still a good chunk of work to be done > > to make things work properly. For example, both Redis and AMQP modules > > are not using the Twisted I/O event loop. Both are still issuing sync > > socket calls in their own libs which could block the entire > > application. > > Yup, this definitely needs fixing. > > These things are very doable in Twisted, it will just require using > > the txRedis and txAMQP modules. It will also introduce more deferreds > > into the mix since we'll have non-blocking connections dependent on > > each other. From this perspective, Eventlet may be easier since we > > could still use the current libraries by patching > > (http://eventlet.net/doc/patching.html). > > Yes, both python-redis and python-amqplib use Python's standard socket > module, so they can be patched. > > Does anyone know how eventlet would deal with libvirt calls? > python-libvirt just calls into libvirt obviously doesn't use Python's > socket module. > -- > Soren Hansen > Systems Architect > The Rackspace Cloud > > _______________________________________________ > Mailing list: https://launchpad.net/~nova > Post to : [email protected] > Unsubscribe : https://launchpad.net/~nova > More help : https://help.launchpad.net/ListHelp > > Confidentiality Notice: This e-mail message (including any attached or > embedded documents) is intended for the exclusive and confidential use of the > individual or entity to which this message is addressed, and unless otherwise > expressly indicated, is confidential and privileged information of Rackspace. > Any dissemination, distribution or copying of the enclosed material is > prohibited. > If you receive this transmission in error, please notify us immediately by > e-mail > at [email protected], and delete the original message. > Your cooperation is appreciated. > _______________________________________________ > Mailing list: https://launchpad.net/~nova > Post to : [email protected] > Unsubscribe : https://launchpad.net/~nova > More help : https://help.launchpad.net/ListHelp _______________________________________________ Mailing list: https://launchpad.net/~nova Post to : [email protected] Unsubscribe : https://launchpad.net/~nova More help : https://help.launchpad.net/ListHelp
