Hrm. It's not piling on with the locking like I originally thought - from the db's perspective there's not a whole hell of a lot going on, actually - even two copies of this script run concurrently cause them to totally get in to some app-level spinning that ends up with at least one script erroring. I'll keep poking, mainly just because it's interesting - but it's certainly not happy.
On 09/25/2011 01:53 PM, Vishvananda Ishaya wrote: > Hey everyone, > > I'm a bit concerned with the connection pooling in the db. It seems that > things are not getting cleaned up properly. I have a repro-case that causes > failures that we have seen before. if I revert the > nova/db/sqlalchemy/session.py to before the eventlet db pool was added I get > no failures. If you want to see the issue, try the attached code. You will > need to run from the nova directory or do python setup.py develop. You will > also need to create a mysql database called test and edit the sql_connection > string if you have a mysql password, etc. Please check this code. If we > can't come up with a fix, I htink we need to revert back to no connection > pooling. > > Run the attached script at least 3 times The code below runs fine the first > couple of times, Then it starts to fail with the following error: > > 2011-09-24 12:51:02,799 INFO sqlalchemy.engine.base.Engine.0x...36d0 [-] > ROLLBACK > Traceback (most recent call last): > File "/Library/Python/2.7/site-packages/eventlet/hubs/hub.py", line 336, in > fire_timers > timer() > File "/Library/Python/2.7/site-packages/eventlet/hubs/timer.py", line 56, in > __call__ > cb(*args, **kw) > File "/Library/Python/2.7/site-packages/eventlet/event.py", line 163, in > _do_send > waiter.switch(result) > File "/Library/Python/2.7/site-packages/eventlet/greenthread.py", line 192, > in main > result = function(*args, **kwargs) > File "dbrepro.py", line 44, in associate > ip = db.fixed_ip_associate_pool(ctxt, 1, instance_id=val) > File "/Users/vishvananda/os/nova/nova/db/api.py", line 352, in > fixed_ip_associate_pool > instance_id, host) > File "/Users/vishvananda/os/nova/nova/db/sqlalchemy/api.py", line 102, in > wrapper > return f(*args, **kwargs) > File "/Users/vishvananda/os/nova/nova/db/sqlalchemy/api.py", line 725, in > fixed_ip_associate_pool > filter_by(host=None).\ > File "/Library/Python/2.7/site-packages/sqlalchemy/orm/query.py", line 1496, > in first > ret = list(self[0:1]) > File "/Library/Python/2.7/site-packages/sqlalchemy/orm/query.py", line 1405, > in __getitem__ > return list(res) > File "/Library/Python/2.7/site-packages/sqlalchemy/orm/query.py", line 1669, > in instances > fetch = cursor.fetchall() > File "/Library/Python/2.7/site-packages/sqlalchemy/engine/base.py", line > 2383, in fetchall > l = self.process_rows(self._fetchall_impl()) > File "/Library/Python/2.7/site-packages/sqlalchemy/engine/base.py", line > 2366, in process_rows > keymap = metadata._keymap > AttributeError: 'NoneType' object has no attribute '_keymap' > > > > > > > > > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
