On Nov 18, 3:01 pm, Michael Bayer <mike...@zzzcomputing.com> wrote: > On Nov 18, 2009, at 9:57 AM, James Casbon wrote: > > > Hi, > > > I'm using sqlalchemy to generate a query that returns lots of data. > > The trouble is, when calling "query.execute()" instead of returning > > the resultproxy straight away and allowing me to fetch data as I would > > like, query.execute blocks and the memory usage grows to gigabytes > > before getting killed for too much memory. This looks to me like > > execute is prefetching the entire result. > > > Is there any way to prevent query.execute loading the entire result? > > for ORM look into using yield_per() or applying limit()/offset(). without > the ORM no rows are buffered on the SQLA side. Note however that MySQLdb is > likely prefetching the entire result set in any case (this is psycopg2s > behavior but haven't confirmed for MySQLdb).
Thanks, but not using the ORM. Looks like you have to specify a server side cursor - see SSCursor in http://mysql-python.sourceforge.net/MySQLdb.html I don't recall any way of forcing sqlalchemy to use a particular cursor? James -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=.
