Re: [sqlalchemy] Connection / disconnect / Pool
On Nov 2, 2010, at 11:46 PM, Warwick Prince wrote: Hi Michael I have an issue I can't fathom regarding Pools.I'm doing testing and hit the following error at exatly the same point each time; File C:\Documents and Settings\wprince\Desktop\PY CODE DEVELOPMENT\pyDAP\DAPForm.py, line 3805, in _processQuery self._queryCount = self._query.count() File C:\Python26\lib\site-packages\sqlalchemy\orm\query.py, line 1888, in count should_nest = should_nest[0] File C:\Python26\lib\site-packages\sqlalchemy\orm\query.py, line 1924, in _col_aggregate mapper=self._mapper_zero()) File C:\Python26\lib\site-packages\sqlalchemy\orm\session.py, line 740, in scalar return self.execute(clause, params=params, mapper=mapper, **kw).scalar() File C:\Python26\lib\site-packages\sqlalchemy\orm\session.py, line 734, in execute return self._connection_for_bind(engine, close_with_result=True).execute( TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30 I'm creating a basic create_engine using defaults for the pool settings (as you can tell). Within each Thread that does the connection, I create a new engine, metadata and scoped session. That thread then does a basic query (in this case NOT using a session at all, just a direct table query) and then I do everything I can think of to close the connections. I do this; engine.dispose() session.commit() session.close() and then the class instance that holds all these is removed and the thread terminates. All works fine (i.e. no errors or exceptions) until I reach the 10 overflow limit and then it dies. Each destroy of an engine and recreate of an engine is at least a second apart so it's not being thrashed at all. What am I missing? The error is local to a single engine. So engine.dispose() has no impact on the issue. While a Session will maintain a single checked out connection, result sets that are returned by engine.execute(some select) and some select.execute() also do so, so be sure to fully exhaust and/or close() those result sets as well. Cheers Warwick Warwick Prince Managing Director mobile: +61 411 026 992 skype: warwickprince phone: +61 7 3102 3730 fax: +61 7 3319 6734 web: www.mushroomsys.com -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Connection / disconnect / Pool
On Nov 3, 2010, at 10:11 AM, Michael Bayer wrote: On Nov 2, 2010, at 11:46 PM, Warwick Prince wrote: Hi Michael I have an issue I can't fathom regarding Pools.I'm doing testing and hit the following error at exatly the same point each time; File C:\Documents and Settings\wprince\Desktop\PY CODE DEVELOPMENT\pyDAP\DAPForm.py, line 3805, in _processQuery self._queryCount = self._query.count() File C:\Python26\lib\site-packages\sqlalchemy\orm\query.py, line 1888, in count should_nest = should_nest[0] File C:\Python26\lib\site-packages\sqlalchemy\orm\query.py, line 1924, in _col_aggregate mapper=self._mapper_zero()) File C:\Python26\lib\site-packages\sqlalchemy\orm\session.py, line 740, in scalar return self.execute(clause, params=params, mapper=mapper, **kw).scalar() File C:\Python26\lib\site-packages\sqlalchemy\orm\session.py, line 734, in execute return self._connection_for_bind(engine, close_with_result=True).execute( TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30 I'm creating a basic create_engine using defaults for the pool settings (as you can tell). Within each Thread that does the connection, I create a new engine, metadata and scoped session. That thread then does a basic query (in this case NOT using a session at all, just a direct table query) and then I do everything I can think of to close the connections. I do this; engine.dispose() session.commit() session.close() and then the class instance that holds all these is removed and the thread terminates. All works fine (i.e. no errors or exceptions) until I reach the 10 overflow limit and then it dies. Each destroy of an engine and recreate of an engine is at least a second apart so it's not being thrashed at all. What am I missing? The error is local to a single engine. So engine.dispose() has no impact on the issue. well, correction, yes if you dispose the engine the pool drops away and is replaced with another, but that doesn't solve the core issue of using connections and not closing them out, since dispose() does not impact connections which are checked out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Connection / disconnect / Pool
Excellent. That'll be it for sure. I have a bunch of result sets that I assumed would just go away.. :-) Cheers Warwick Warwick A. Prince Managing Director Mushroom Systems International P/L On 04/11/2010, at 1:11 AM, Michael Bayer mike...@zzzcomputing.com wrote: On Nov 2, 2010, at 11:46 PM, Warwick Prince wrote: Hi Michael I have an issue I can't fathom regarding Pools.I'm doing testing and hit the following error at exatly the same point each time; File C:\Documents and Settings\wprince\Desktop\PY CODE DEVELOPMENT\pyDAP\DAPForm.py, line 3805, in _processQuery self._queryCount = self._query.count() File C:\Python26\lib\site-packages\sqlalchemy\orm\query.py, line 1888, in count should_nest = should_nest[0] File C:\Python26\lib\site-packages\sqlalchemy\orm\query.py, line 1924, in _col_aggregate mapper=self._mapper_zero()) File C:\Python26\lib\site-packages\sqlalchemy\orm\session.py, line 740, in scalar return self.execute(clause, params=params, mapper=mapper, **kw).scalar() File C:\Python26\lib\site-packages\sqlalchemy\orm\session.py, line 734, in execute return self._connection_for_bind(engine, close_with_result=True).execute( TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30 I'm creating a basic create_engine using defaults for the pool settings (as you can tell). Within each Thread that does the connection, I create a new engine, metadata and scoped session. That thread then does a basic query (in this case NOT using a session at all, just a direct table query) and then I do everything I can think of to close the connections. I do this; engine.dispose() session.commit() session.close() and then the class instance that holds all these is removed and the thread terminates. All works fine (i.e. no errors or exceptions) until I reach the 10 overflow limit and then it dies. Each destroy of an engine and recreate of an engine is at least a second apart so it's not being thrashed at all. What am I missing? The error is local to a single engine. So engine.dispose() has no impact on the issue. While a Session will maintain a single checked out connection, result sets that are returned by engine.execute(some select) and some select.execute() also do so, so be sure to fully exhaust and/or close() those result sets as well. Cheers Warwick Warwick Prince Managing Director mobile: +61 411 026 992 skype: warwickprince phone: +61 7 3102 3730 fax: +61 7 3319 6734 web: www.mushroomsys.com -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Connection / disconnect / Pool
Hi Michael I have an issue I can't fathom regarding Pools.I'm doing testing and hit the following error at exatly the same point each time; File C:\Documents and Settings\wprince\Desktop\PY CODE DEVELOPMENT\pyDAP\DAPForm.py, line 3805, in _processQuery self._queryCount = self._query.count() File C:\Python26\lib\site-packages\sqlalchemy\orm\query.py, line 1888, in count should_nest = should_nest[0] File C:\Python26\lib\site-packages\sqlalchemy\orm\query.py, line 1924, in _col_aggregate mapper=self._mapper_zero()) File C:\Python26\lib\site-packages\sqlalchemy\orm\session.py, line 740, in scalar return self.execute(clause, params=params, mapper=mapper, **kw).scalar() File C:\Python26\lib\site-packages\sqlalchemy\orm\session.py, line 734, in execute return self._connection_for_bind(engine, close_with_result=True).execute( TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30 I'm creating a basic create_engine using defaults for the pool settings (as you can tell). Within each Thread that does the connection, I create a new engine, metadata and scoped session. That thread then does a basic query (in this case NOT using a session at all, just a direct table query) and then I do everything I can think of to close the connections. I do this; engine.dispose() session.commit() session.close() and then the class instance that holds all these is removed and the thread terminates. All works fine (i.e. no errors or exceptions) until I reach the 10 overflow limit and then it dies. Each destroy of an engine and recreate of an engine is at least a second apart so it's not being thrashed at all. What am I missing? Cheers Warwick Warwick Prince Managing Director mobile: +61 411 026 992 skype: warwickprince phone: +61 7 3102 3730 fax: +61 7 3319 6734 web: www.mushroomsys.com -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
