Re: [Openstack] Memory leaks from greenthreads

2012-03-06 Thread Vishvananda Ishaya
There isn't an easy repro case for the problem, which is why it snuck in in the first place. IIRC It only happened when trying to run with reasonable concurrency at a time on a multimachine install. One thing that was very obvious during debugging though is that the number of threads started wa

Re: [Openstack] Memory leaks from greenthreads

2012-03-06 Thread Chris Behrens
I wonder if the issue is gone as well. After the db_pool code was removed, I was doing some separate testing of sqlalchemy's with_lockmode().. and I was able to reproduce a traceback from sqlalchemy that looked extremely similar or identical to the traceback we were hitting when using eventle

Re: [Openstack] Memory leaks from greenthreads

2012-03-06 Thread Yuriy Taraday
Now I had no luck trying to reproduce problem that appeared with db_pool. Maybe it's gone? Who can walk me through the way to that bug? I think, we should fix it if it's not fixed already, bring back db_pool and then eventlet will be all good again, am I right? Kind regards, Yuriy. On Fri, Mar

Re: [Openstack] Memory leaks from greenthreads

2012-03-05 Thread Pádraig Brady
On 03/05/2012 04:02 PM, Johannes Erdfelt wrote: > On Mon, Mar 05, 2012, Pádraig Brady wrote: >> File "/usr/lib64/python2.7/threading.py", line 446, in __init__ >> self.__daemonic = self._set_daemon() >> File "/usr/lib64/python2.7/threading.py", line 470, in _set_daemon >> return curren

Re: [Openstack] Memory leaks from greenthreads

2012-03-05 Thread Johannes Erdfelt
On Mon, Mar 05, 2012, Pádraig Brady wrote: > File "/usr/lib64/python2.7/threading.py", line 446, in __init__ > self.__daemonic = self._set_daemon() > File "/usr/lib64/python2.7/threading.py", line 470, in _set_daemon > return current_thread().daemon > AttributeError: '_GreenThread' obj

Re: [Openstack] Memory leaks from greenthreads

2012-03-05 Thread Pádraig Brady
On 02/29/2012 09:08 PM, Johannes Erdfelt wrote: > On Wed, Feb 29, 2012, Vishvananda Ishaya wrote: >> We have had a memory leak due to an interaction with eventlet for a >> while that Johannes has just made a fix for. >> >> bug: >> https://bugs.launchpad.net/nova/+bug/903199 >> >> fix: >> https://b

Re: [Openstack] Memory leaks from greenthreads

2012-03-01 Thread Vishvananda Ishaya
I agree. It would be awesome if someone could actually make it work. We had a totally broken version using the eventlet db pool 6 months ago. Vish On Mar 1, 2012, at 4:20 PM, Joshua Harlow wrote: > Sad, especially since so much is using the database :-( > > On 3/1/12 2:43 PM, "Adam Young" w

Re: [Openstack] Memory leaks from greenthreads

2012-03-01 Thread Joshua Harlow
Sad, especially since so much is using the database :-( On 3/1/12 2:43 PM, "Adam Young" wrote: On 03/01/2012 02:48 PM, Vishvananda Ishaya wrote: > On Mar 1, 2012, at 9:39 AM, Adam Young wrote: > >> What would the drawbacks be? Probably the first thing people would look to >> from Eventlet is pe

Re: [Openstack] Memory leaks from greenthreads

2012-03-01 Thread Adam Young
On 03/01/2012 02:48 PM, Vishvananda Ishaya wrote: On Mar 1, 2012, at 9:39 AM, Adam Young wrote: What would the drawbacks be? Probably the first thing people would look to from Eventlet is performance. I don't have the hard numbers to compare Eventlet to Apache HTTPD, but I do know that Apache

Re: [Openstack] Memory leaks from greenthreads

2012-03-01 Thread Adam Young
On 03/01/2012 02:48 PM, Vishvananda Ishaya wrote: On Mar 1, 2012, at 9:39 AM, Adam Young wrote: What would the drawbacks be? Probably the first thing people would look to from Eventlet is performance. I don't have the hard numbers to compare Eventlet to Apache HTTPD, but I do know that Apache

Re: [Openstack] Memory leaks from greenthreads

2012-03-01 Thread Vishvananda Ishaya
On Mar 1, 2012, at 9:39 AM, Adam Young wrote: > > What would the drawbacks be? Probably the first thing people would look to > from Eventlet is performance. I don't have the hard numbers to compare > Eventlet to Apache HTTPD, but I do know that Apache is used in enough high > volume sites tha

Re: [Openstack] Memory leaks from greenthreads

2012-03-01 Thread Joshua Harlow
find once we move to a full threaded model. Phil From: openstack-bounces+philip.day=hp@lists.launchpad.net [mailto:openstack-bounces+philip.day=hp@lists.launchpad.net] On Behalf Of Joshua Harlow Sent: 29 February 2012 21:26 To: Vishvananda Ishaya; openstack Subject: Re: [Openstack]

Re: [Openstack] Memory leaks from greenthreads

2012-03-01 Thread Adam Young
I wrote up why I think that, at least for Keystone, we should move the front end over to Apache HTTPD. http://adam.younglogic.com/2012/03/keystone-should-move-to-apache-httpd/ I've reposted it below. Keystone and the other Openstack components run in an href="http://www.eventlet.net";>Eve

Re: [Openstack] Memory leaks from greenthreads

2012-03-01 Thread Day, Phil
x27;ll find once we move to a full threaded model. Phil From: openstack-bounces+philip.day=hp@lists.launchpad.net [mailto:openstack-bounces+philip.day=hp@lists.launchpad.net] On Behalf Of Joshua Harlow Sent: 29 February 2012 21:26 To: Vishvananda Ishaya; openstack Subject: Re:

Re: [Openstack] Memory leaks from greenthreads

2012-02-29 Thread Joshua Harlow
I agree its not trivial, but it might be worth it? Or at least shift the model toward something semi-equivalent (multiprocessing??) If the python community says though that better inherent support for eventlet will arrive (ie something in the core python) someday then I say leave it. Otherwise

Re: [Openstack] Memory leaks from greenthreads

2012-02-29 Thread Johannes Erdfelt
On Wed, Feb 29, 2012, Joshua Harlow wrote: > Just a thought I was having, that others might want to chime in on. > > Has there been any thinking around only using eventlet/greenlet for > webserver endpoints and using something like multiprocessing for > everything else? > > I know its a fundamen

Re: [Openstack] Memory leaks from greenthreads

2012-02-29 Thread Joshua Harlow
Cool. Just a thought I was having, that others might want to chime in on. Has there been any thinking around only using eventlet/greenlet for webserver endpoints and using something like multiprocessing for everything else? I know its a fundamental change, but it would force people to think abo

Re: [Openstack] Memory leaks from greenthreads

2012-02-29 Thread Johannes Erdfelt
On Wed, Feb 29, 2012, Vishvananda Ishaya wrote: > We have had a memory leak due to an interaction with eventlet for a > while that Johannes has just made a fix for. > > bug: > https://bugs.launchpad.net/nova/+bug/903199 > > fix: > https://bitbucket.org/which_linden/eventlet/pull-request/10/monke

[Openstack] Memory leaks from greenthreads

2012-02-29 Thread Vishvananda Ishaya
Hello Everyone, We have had a memory leak due to an interaction with eventlet for a while that Johannes has just made a fix for. bug: https://bugs.launchpad.net/nova/+bug/903199 fix: https://bitbucket.org/which_linden/eventlet/pull-request/10/monkey-patch-threadingcurrent_thread-as Unfortuante