On Wed, Feb 22, 2012 at 4:29 PM, Michael Bayer <mike...@zzzcomputing.com> wrote: >> thanks for your reply. I haven't yet tested this with a profiler to see >> exactly what exactly is happening, but the bottom line is that the overall >> memory use grows with each iteration (or transaction processed), to the >> point of grinding the server to a halt, and top shows only the Python >> process involved consuming all the memory. > > yeah like I said that tells you almost nothing until you start looking at > gc.get_objects(). If the size of gc.get_objects() grows continuously for 50 > iterations or more, never decreasing even when gc.collect() is called, then > it's a leak. Otherwise it's just too much data being loaded at once.
I've noticed compiling queries (either explicitly or implicitly) tends to *fragment* memory. There seem to be long-lived caches in the PG compiler at least. I can't remember exactly where, but I could take another look. I'm talking of rather old versions of SQLA, 0.3 and 0.5. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
