On 1/9/07, Michael Bayer <[EMAIL PROTECTED]> wrote: > > > On Jan 9, 2007, at 5:24 AM, Michele Petrazzo wrote: > > Because, like what I wrote on the psycopg2 ml (lists.initd.org/ > > pipermail/psycopg/2007-January/005250.html) > > without the "server side cursor", the memory usage on the client > > side is proportionally at the number of records returned by the > > query (on my example 400k records, 60MB ram). I think because the > > driver fetch all the data from the server and leave it available > > with the fetchXXX methods. > > really ? im not sure about psycopg2 specifically, but its been my > experience with virtually all database client libraries that as you > fetch rows and discard them, you only pull as much as you fetch into > memory. i.e. if you had a resultset of 4000 rows and just fetched 5 > rows, only 5 rows get pulled over the wire. thats definitely how the > lower level APIs work and it would be pretty bad if psycopg2 didnt > maintain that behavior.
she's right, psycopg2 sucks that way. it's actually lower-level than that, in libpq -- anyone who doesn't want to pull the entire result set back needs to use a cursor. from http://www.postgresql.org/docs/faqs.TODO.html: "Currently, all statement results are transferred to the libpq client before libpq makes the results available to the application. This feature would allow the application to make use of the first result rows while the rest are transferred, or held on the server waiting for them to be requested by libpq. One complexity is that a statement like SELECT 1/col could error out mid-way through the result set." --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---
