Beautiful - there's plenty to work with here. Thanks much, Mike! On Wed, Mar 30, 2011 at 11:51 AM, Michael Bayer <mike...@zzzcomputing.com>wrote:
> For quick hits I usually turn debug logging on, or echo='debug' with > create_engine(), that shows the rows coming in in the log. > > There's no "per row" event interface, you can of course intercept execute > events at the engine and cursor level with ConnectionProxy. > > A blunt approach may be to intercept cursor executes, ensure the statement > is a SELECT, then run the statement distinctly (i.e. a second time) in the > handler, count the rows, or at least do a fetchmany() of N number of rows > and alert if over a certain threshold. Or wrap the query inside of > "SELECT COUNT(*) FROM (<query>)" and get a count that way. > > Another depends on your DBAPI - if its observed that rows are pre-fetched > within the execute, then you could just apply timing to a ConnectionProxy > and look for slow queries (or large jumps in the size of gc.get_objects() > maybe). > > Still another tack, subclass Query, override __iter__, pull out the result > from super().__iter__(), count it, then return iter(result). Query by > default buffers everything anyway. This would depend though on the fact > that your query is returning distinct primary keys - if you have a basic > cartesian product occurring (which is likely), Query's uniquifying of > results might conceal that. > > > On Mar 30, 2011, at 11:31 AM, Rick Morrison wrote: > > > Hi list: > > > > I've recently been plagued by a runaway query somewhere in one of my apps > that mistakenly loads 10s of 1000's of rows, swamping the working set of the > Python process and eventually invoking the OOM killer. > > > > Unfortunately, the database backend I'm using (MSSQL 2005) doesn't > provide a lot in the way of throttling or detection tools for this, and so > I'd like to inject some detection code into SQLA to track this thing down. > Is there an existing listener interface (or an appropriate injection > location for some code) in the (0.6.6) Engine or ResultProxy where it's > possible to watch the number of rows retrieved? > > > > Thanks, > > Rick > > > > > > -- > > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > > To post to this group, send email to sqlalchemy@googlegroups.com. > > To unsubscribe from this group, send email to > sqlalchemy+unsubscr...@googlegroups.com. > > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to sqlalchemy@googlegroups.com. > To unsubscribe from this group, send email to > sqlalchemy+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
