Thanks Michael for your reply. > so to just get dispose() to work correctly, verify for me that this > patch works: Yes, the patch works perfectly, the overflow is reset to its initial value as expected.
> i dont totally understand whats going on there. if the connections > cant open and then they all time out, eventually the "connectionrecord" > that is trying to make the connection should get returned to the pool, > once the reference to the connection in your code is released...so you > shouldnt need the dispose() in theory (but ive never tested such a > scenario so theres many reasons i could be wrong). Well you'll find a test case as attachment, which describes what happening in my app, specifically the retry mecanism. To run the test: 1. python test.py # check if everything runs OK 2. Stop MySQL server 3. re-run the test.py 4. Wait a little (try #2 or #3) 5. Restart the server 6. Or next try, the pool get stuck into its TimeoutError "max pool size reached blabla". In the test.py, there is a comment block, where the TimeoutError is catched. If the pool is then cleaned (calling dispose), everything works ok (with the patch). This is why I primarily wanted to use pool.dispose(). Now I dig a little into the pool.py module. Here's what I found as a possible bug: in the do_get connection, when SA tries to create_connection() and fails (which is the case here when mysql server is out), an exception is raised, the connection is not returned *BUT* the overflow has been set to += 1. So there is a desync between the overflow value and the real overflow state. This seems to explain why SA still raise those TimeoutError when trying to reconnect. Here's a patch which seems to get things work (it includes your previous). This just adjust the overflow *once* the connection is actually created. See attachment. Hope it helps. Cheers, Seb -- Sébastien LELONG sebastien.lelong[at]sirloon.net --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---
pooltest.tar.bz2
Description: application/tbz
Index: pool.py
===================================================================
--- pool.py (revision 2157)
+++ pool.py (working copy)
@@ -363,8 +363,9 @@
except Queue.Empty:
if self._max_overflow > -1 and self._overflow >= self._max_overflow:
raise exceptions.TimeoutError("QueuePool limit of size %d overflow %d reached, connection timed out" % (self.size(), self.overflow()))
+ fresh_con = self.create_connection()
self._overflow += 1
- return self.create_connection()
+ return fresh_con
def dispose(self):
while True:
@@ -373,6 +374,9 @@
conn.close()
except Queue.Empty:
break
+
+ self._overflow = 0 - self.size()
+ self.logger.warning("self._overflow is %s" % self._overflow)
def status(self):
tup = (self.size(), self.checkedin(), self.overflow(), self.checkedout())
