On Nov 18, 2009, at 9:57 AM, James Casbon wrote: > Hi, > > I'm using sqlalchemy to generate a query that returns lots of data. > The trouble is, when calling "query.execute()" instead of returning > the resultproxy straight away and allowing me to fetch data as I would > like, query.execute blocks and the memory usage grows to gigabytes > before getting killed for too much memory. This looks to me like > execute is prefetching the entire result. > > Is there any way to prevent query.execute loading the entire result?
for ORM look into using yield_per() or applying limit()/offset(). without the ORM no rows are buffered on the SQLA side. Note however that MySQLdb is likely prefetching the entire result set in any case (this is psycopg2s behavior but haven't confirmed for MySQLdb). -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=.
