Re: [sqlalchemy] session memory management

2011-12-05 Thread Michael Bayer
On Dec 5, 2011, at 1:40 AM, kris wrote: > I have added the flush every 1024 loads, and also set DBSession.autoflush = > False, but I see that I have > gone and used up all available memory again. There are no other references > than the ones in the session. > > A bit more information, I am g

Re: [sqlalchemy] session memory management

2011-12-04 Thread kris
I have added the flush every 1024 loads, and also set DBSession.autoflush = False, but I see that I have gone and used up all available memory again. There are no other references than the ones in the session. A bit more information, I am getting stuck in a routine that adds 'root' pointer to

Re: [sqlalchemy] session memory management

2011-12-03 Thread Michael Bayer
On Dec 3, 2011, at 2:31 PM, kris wrote: > I have a database migration using sqlalchemy migrate that needs to migrate > several million rows. > After running for several days, I see the job has grown to consume all > available memory and > is swapping. > > Can someone point me to a descript

[sqlalchemy] session memory management

2011-12-03 Thread kris
I have a database migration using sqlalchemy migrate that needs to migrate several million rows. After running for several days, I see the job has grown to consume all available memory and is swapping. Can someone point me to a description of what is the best way to manage memory in the ses