Hi, I am using sqlalchemy like this:
entries = session.query(User) for entry in entries: entry.check_it_is_all_right() # includes changing values of columns if necessary session.flush() As you might have noticed, I am iterating over all rows in the database. Since there are like thousands rows, each containing like 10kB of data (and I _do_ need all of the data for each entry), it is unconvenient for me how sqlalchemy treats this: it loads _all_ objects into memory - this takes approx. 30 seconds on a dedicated machine with 100% CPU usage, taking away tens of MB of RAM. (it is my virtual testing machine, it's fine there (although who wants to wait 30 seconds to only realize that the processing fails with exception on first entry :-)), but I most probably can't use such a resource-eater on a production server). It would however suffice that sqlalchemy made the access to the result sequentially, without loading more than is really needed at the moment. I thought the Python iterator protocol (__iter__ and next methods) are exactly tailored for this efficient access (e.g. iterating over lines of file objects (for line in open("file.txt", "r")) works exactly efficiently how I would want sqlalchemy to in my case :-) - Is there a way for efficient "on-demand" iteration over all rows when using orm object mapping, or do I have to use a more low-level protocol (like sqlalchemy without orm (fetchone()), or even Python DB API itself?) Thanks for your suggestions, Boris Duek --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---