In your for loop, every time you iterate the object instance count is increased. Now if you call that loop again before python garbage collector has time to release memory then your for loop will cause more instances to be increased. Python will not release memory to the os for an object until every instance pointing to that object is deleted. This is what is known as a memory leak and yes garbage collected language are at times the worst for this situation if you don't watch how you code.
Now with a little research you will find you can make that for loop load only 1 instance in memory instead of 1 instance per record in the database. If you make it load only a single instance your problem will go away. I am pretty sure this might be covered in the web2py book. On Thu, May 10, 2012 at 6:14 AM, Bruce Wade <bruce.w...@gmail.com> wrote: > Their is no grantee that python will use that exact same memory. It would > also depend how frequently you use that function. > > http://mg.pov.lt/blog/hunting-python-memleaks.html > > > On Thu, May 10, 2012 at 12:57 AM, szimszon <szims...@gmail.com> wrote: > >> If I understand well then python release the mem. for reuse by itself but >> not releasing at OS level. Fine. >> >> But then if I rerun the function I'll expect that the mem allocated for >> python is not growing. Because python reuse it. >> >> I executed the controller function and my mem usage at OS level 2x as >> after first run. It sounds not right. >> >> 2012. május 10., csütörtök 9:41:31 UTC+2 időpontban Bruce Wade a >> következőt írta: >> >>> http://effbot.org/pyfaq/why-**doesnt-python-release-the-** >>> memory-when-i-delete-a-large-**object.htm<http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object.htm> >>> >>> That is a little more details >>> >>> On Thu, May 10, 2012 at 12:37 AM, szimszon <szims...@gmail.com> wrote: >>> >>>> Should the garbage collector not free up the memory? >>>> >>>> 2012. május 10., csütörtök 9:28:48 UTC+2 időpontban Bruce Wade a >>>> következőt írta: >>>> >>>>> That is how python is. If you want something to clear the memory as >>>>> soon as you are done with it you need C++ :D >>>>> >>>>> On Thu, May 10, 2012 at 12:27 AM, szimszon <szims...@gmail.com> wrote: >>>>> >>>>>> Yes I know but it happens over type the mem usage is linearly growing >>>>>> and after the successful execution never released and that is why I ask >>>>>> :( >>>>>> >>>>>> 2012. május 10., csütörtök 9:14:14 UTC+2 időpontban Bruce Wade a >>>>>> következőt írta: >>>>>> >>>>>>> WOW not a good idea: >>>>>>> for row in db( db.file_**propert****ies.id > 0 ).select( >>>>>>> >>>>>>> If you have a lot of records that is going to kill your memory. >>>>>>> >>>>>>> On Thu, May 10, 2012 at 12:10 AM, szimszon <szims...@gmail.com>wrote: >>>>>>> >>>>>>>> I wonder if somebody could help me. >>>>>>>> >>>>>>>> The following code has eaten up ~1,5GB ram and after ended >>>>>>>> successfully it is not released :( >>>>>>>> >>>>>>>> file_properties table has 137314 rows. >>>>>>>> >>>>>>>> Model: >>>>>>>> db.define_table( 'files_store', >>>>>>>> Field( 'name', 'string', >>>>>>>> label = 'Fájlnév' , >>>>>>>> ), >>>>>>>> Field( 'content', 'upload', >>>>>>>> uploadseparate = True, >>>>>>>> autodelete = True, >>>>>>>> label = 'Aláírt fájl', >>>>>>>> ), >>>>>>>> Field( 'status', 'string', >>>>>>>> label = 'Állapot', >>>>>>>> requires = IS_IN_SET( [ >>>>>>>> 'empty', 'notsigned', 'senttosign', 'signed'], >>>>>>>> >>>>>>>> [ 'Üres', 'Aláiratlan', 'Aláírásra elküldve', >>>>>>>> 'Aláírt'] ), >>>>>>>> represent = lambda r,row >>>>>>>> :T( r ) >>>>>>>> ), >>>>>>>> Field( 'state', 'string', >>>>>>>> label = 'Státusz', >>>>>>>> requires = IS_IN_SET( [ >>>>>>>> 'ok', 'error'], >>>>>>>> >>>>>>>> [ 'Rendben', 'Hiba'] ), >>>>>>>> represent = lambda r,row >>>>>>>> :T( r ) >>>>>>>> ), >>>>>>>> Field( 'data_count', 'integer', >>>>>>>> label = 'Adat >>>>>>>> bejegyzések', >>>>>>>> requires = IS_NOT_EMPTY >>>>>>>> (), >>>>>>>> default = 0, >>>>>>>> ), >>>>>>>> Field( 'dirname', 'string', >>>>>>>> label = 'Könyvtár név' >>>>>>>> , >>>>>>>> requires = IS_NOT_EMPTY >>>>>>>> (), >>>>>>>> ), >>>>>>>> Field( 'md5sum', 'string', >>>>>>>> label = T( 'MD5 Sum' ), >>>>>>>> represent = lambdamd5sum >>>>>>>> , junk: '[%s]' % md5sum, >>>>>>>> writable = False ), >>>>>>>> auth.signature, >>>>>>>> format = '%(dirname)s/%(name)s >>>>>>>> (%(id)s)' >>>>>>>> ) >>>>>>>> db.files_store.created_on.read******able = True >>>>>>>> db.define_table( 'file_properties', >>>>>>>> Field( 'files_store_id', db. >>>>>>>> files_store, >>>>>>>> label = 'Aláírt fájl' , >>>>>>>> ), >>>>>>>> Field( 'kezdo_oldal', 'integer', >>>>>>>> label = 'Számla kezdő >>>>>>>> oldal', >>>>>>>> requires = IS_NOT_EMPTY >>>>>>>> (), >>>>>>>> ), >>>>>>>> Field( 'oldalszam', 'integer', >>>>>>>> label = 'Számla >>>>>>>> oldalainak száma', >>>>>>>> requires = IS_NOT_EMPTY >>>>>>>> (), >>>>>>>> ), >>>>>>>> Field( 'szamla_sorszama', 'string', >>>>>>>> label = 'Számla >>>>>>>> sorszáma', >>>>>>>> #requires = >>>>>>>> IS_NOT_IN_DB( db, 'file_properties' ), >>>>>>>> unique = True, >>>>>>>> ), >>>>>>>> Field( 'ugyfel_azonosito', 'string' >>>>>>>> , >>>>>>>> label = 'Ügyfél >>>>>>>> azonosító', >>>>>>>> requires = IS_NOT_EMPTY >>>>>>>> (), >>>>>>>> ), >>>>>>>> Field( 'folyoszamla_azonosito', >>>>>>>> 'string', >>>>>>>> label = 'Folyószámla >>>>>>>> azonosító', >>>>>>>> requires = IS_NOT_EMPTY >>>>>>>> (), >>>>>>>> ), >>>>>>>> Field( 'teljesites_idopontja', >>>>>>>> 'date', >>>>>>>> label = 'Teljesítés >>>>>>>> időpontja', >>>>>>>> requires = IS_DATE( >>>>>>>> '%Y.%m.%d.' ) ), >>>>>>>> auth.signature, >>>>>>>> ), >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Controller: >>>>>>>> >>>>>>>> def autoadjust(): >>>>>>>> lista = list() >>>>>>>> last_row = None >>>>>>>> next_page_number = 0 >>>>>>>> for row in db( db.file_properties.id > 0 ).select( >>>>>>>> >>>>>>>> orderby = ( db.file_properties.files_** >>>>>>>> store****_id, >>>>>>>> >>>>>>>> db.file_properties.id ) >>>>>>>> >>>>>>>> ): >>>>>>>> if last_row: >>>>>>>> if last_row.files_store_id == row.files_store_id: >>>>>>>> if next_page_number > row.kezdo_oldal: >>>>>>>> row.update_record( kezdo_oldal =next_page_number >>>>>>>> ) >>>>>>>> >>>>>>>> else: >>>>>>>> last_row = row >>>>>>>> else: >>>>>>>> last_row = row >>>>>>>> next_page_number = row.kezdo_oldal + row.oldalszam >>>>>>>> >>>>>>>> lista.append( TR( >>>>>>>> TD( row.id ), >>>>>>>> TD( row.files_store_id.name >>>>>>>> ), >>>>>>>> TD( row.kezdo_oldal ), >>>>>>>> TD( row.oldalszam ) >>>>>>>> ) ) >>>>>>>> lista = TABLE( *lista ) >>>>>>>> return dict( lista = lista ) >>>>>>>> >>>>>>>> >>>>>>>> View: >>>>>>>> {{extend 'layout.html'}} >>>>>>>> {{block statusbar}} >>>>>>>> <h4>Admin felület</h4> >>>>>>>> {{end}} >>>>>>>> >>>>>>>> <h2>AdjustPage</h2> >>>>>>>> >>>>>>>> {{=lista}} >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> -- >>>>>>> Regards, >>>>>>> Bruce Wade >>>>>>> http://ca.linkedin.com/in/**bruc****elwade<http://ca.linkedin.com/in/brucelwade> >>>>>>> http://www.wadecybertech.com >>>>>>> http://www.fittraineronline.**co****m<http://www.fittraineronline.com>- >>>>>>> Fitness Personal Trainers Online >>>>>>> http://www.warplydesigned.com >>>>>>> >>>>>>> >>>>> >>>>> >>>>> -- >>>>> -- >>>>> Regards, >>>>> Bruce Wade >>>>> http://ca.linkedin.com/in/**bruc**elwade<http://ca.linkedin.com/in/brucelwade> >>>>> http://www.wadecybertech.com >>>>> http://www.fittraineronline.**co**m <http://www.fittraineronline.com>- >>>>> Fitness Personal Trainers Online >>>>> http://www.warplydesigned.com >>>>> >>>>> >>> >>> >>> -- >>> -- >>> Regards, >>> Bruce Wade >>> http://ca.linkedin.com/in/**brucelwade<http://ca.linkedin.com/in/brucelwade> >>> http://www.wadecybertech.com >>> http://www.fittraineronline.**com <http://www.fittraineronline.com> - >>> Fitness Personal Trainers Online >>> http://www.warplydesigned.com >>> >>> > > > -- > -- > Regards, > Bruce Wade > http://ca.linkedin.com/in/brucelwade > http://www.wadecybertech.com > http://www.fittraineronline.com - Fitness Personal Trainers Online > http://www.warplydesigned.com > > -- -- Regards, Bruce Wade http://ca.linkedin.com/in/brucelwade http://www.wadecybertech.com http://www.fittraineronline.com - Fitness Personal Trainers Online http://www.warplydesigned.com
