Re: [sqlalchemy] Measure time to checkout a connection from the pool
Thanks Michael. For what it's worth, I ended up doing this: # thread local storage tls = threading.local() # Add custom code to measure the time we spend on the current thread waiting # to get a DB connection from the pool. We do this by defining a decorator # and monkey-patching the relevant function. # https://groups.google.com/forum/#!topic/sqlalchemy/jnulkBclSGk def time_pool_get(function): global tls def wrapper(*args, **kwargs): if not hasattr(tls, 'pool_get_stopwatch'): tls.pool_get_stopwatch = Stopwatch() tls.pool_get_stopwatch.start() try: return function(*args, **kwargs) finally: tls.pool_get_stopwatch.stop() return wrapper from sqlalchemy.pool import QueuePool QueuePool._do_get = time_pool_get(QueuePool._do_get) It's a bit of hack in that: 1. I'm monkey-patching, and 2. I'm monkey-patching an internal function (_do_get). But it does measure what I'm after (I think). --S On Wed, Feb 19, 2014 at 8:15 PM, Michael Bayer mike...@zzzcomputing.comwrote: On Feb 19, 2014, at 9:27 PM, Shahaf Abileah sha...@gmail.com wrote: Hello, I'd like to measure how much time my code spends waiting to check out a connection from the pool. Why? Because I suspect that I have too many workers and not enough connections in the pool, and I'd like to gather some hard numbers to prove my theory one way or the other. I see that there are events I can register for: http://docs.sqlalchemy.org/en/rel_0_9/core/events.html#connection-pool-events. However, off hand I don't see a documented way to get the time spent waiting in checkout. I took a peek at pool.py (I'm using sqlalchemy version 0.9.1) . I see _ConnectionRecord doing the following: def __connect(self): try: self.starttime = time.time() connection = self.__pool._creator() self.__pool.logger.debug(Created new connection %r, connection) return connection except Exception as e: self.__pool.logger.debug(Error on connect(): %s, e) raise Is it safe to use this starttime? Does it measure the time-to-checkout? Or time-to-establish-new-connection-in-pool? Or something else? Is there a better way to get the info I'm looking for? for profiling code you should use the Python profiler module. See http://stackoverflow.com/questions/1171166/how-can-i-profile-a-sqlalchemy-powered-application/1175677#1175677for an introduction to using Python profiling with specifics regarding SQLAlchemy. -- http://twitter.com/shahafabileah -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
[sqlalchemy] Measure time to checkout a connection from the pool
Hello, I'd like to measure how much time my code spends waiting to check out a connection from the pool. Why? Because I suspect that I have too many workers and not enough connections in the pool, and I'd like to gather some hard numbers to prove my theory one way or the other. I see that there are events I can register for: http://docs.sqlalchemy.org/en/rel_0_9/core/events.html#connection-pool-events. However, off hand I don't see a documented way to get the time spent waiting in checkout. I took a peek at pool.py (I'm using sqlalchemy version 0.9.1) . I see _ConnectionRecord doing the following: def __connect(self): try: self.starttime = time.time() connection = self.__pool._creator() self.__pool.logger.debug(Created new connection %r, connection) return connection except Exception as e: self.__pool.logger.debug(Error on connect(): %s, e) raise Is it safe to use this starttime? Does it measure the time-to-checkout? Or time-to-establish-new-connection-in-pool? Or something else? Is there a better way to get the info I'm looking for? thanks! --Shahaf -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] Measure time to checkout a connection from the pool
On Feb 19, 2014, at 9:27 PM, Shahaf Abileah sha...@gmail.com wrote: Hello, I'd like to measure how much time my code spends waiting to check out a connection from the pool. Why? Because I suspect that I have too many workers and not enough connections in the pool, and I'd like to gather some hard numbers to prove my theory one way or the other. I see that there are events I can register for: http://docs.sqlalchemy.org/en/rel_0_9/core/events.html#connection-pool-events. However, off hand I don't see a documented way to get the time spent waiting in checkout. I took a peek at pool.py (I'm using sqlalchemy version 0.9.1) . I see _ConnectionRecord doing the following: def __connect(self): try: self.starttime = time.time() connection = self.__pool._creator() self.__pool.logger.debug(Created new connection %r, connection) return connection except Exception as e: self.__pool.logger.debug(Error on connect(): %s, e) raise Is it safe to use this starttime? Does it measure the time-to-checkout? Or time-to-establish-new-connection-in-pool? Or something else? Is there a better way to get the info I'm looking for? for profiling code you should use the Python profiler module. See http://stackoverflow.com/questions/1171166/how-can-i-profile-a-sqlalchemy-powered-application/1175677#1175677 for an introduction to using Python profiling with specifics regarding SQLAlchemy. signature.asc Description: Message signed with OpenPGP using GPGMail
