Ok, you don't need it to works all the time. Did you get it to update your records?
If not, and if as you said it is a one trip, you can just treat the whole records batch by batch... Look here : http://web2py.com/books/default/chapter/29/14#Populating-database-with-dummy-data for i in range(10): populate(db.mytable,100) db.commit() In this example only 100 records are populated at a time between db.commit()... So maybe you just have to wrap for loop that will db.commit() a couples of time during your processing. Richard On Thu, May 10, 2012 at 3:32 PM, szimszon <szims...@gmail.com> wrote: > I had to store files and a lot of properties for it. It was in csv. But > after I processed it we figured out that not all value was correct in csv > but it was a bit redundant. So I can correct it by go through all the > records row by row. So that was a one time trip. > > I just realized after the process I had no memory left. So now I'm > investigating what happened... > > 2012. május 10., csütörtök 21:00:05 UTC+2 időpontban Richard a következőt > írta: >> >> Yes but in this case it is not for the entire reecords... >> >> Why would you return a full list of all the records? >> >> I don't understand what is the purpose of listar that you return in the >> view under a html table, why do you need to return all the 100000+ entries? >> >> Richard >> >> On Thu, May 10, 2012 at 2:56 PM, szimszon <szims...@gmail.com> wrote: >> >>> In book it is a recommended way to iterate over sql results: >>> >>> http://web2py.com/books/**default/chapter/29/6<http://web2py.com/books/default/chapter/29/6> >>> >>> You can do all the steps in one statement: >>> >>> 1. >>> 2. >>> 3. >>> >>> >>> for row in db(db.person.name=='Alex').sel**ect(): >>> >>> print row.name >>> Alex >>> >>> >>> >>> >>> 2012. május 10., csütörtök 20:42:22 UTC+2 időpontban Bruce Wade a >>> következőt írta: >>> >>>> Sorry, you really need to read more about how python works. If you >>>> learn how for loops work and memory you will understand the problem. >>>> >>>> One solution do the query before the for loop then loop through the >>>> objects. This may help at bit. Research xrange vs range >>>> >>>> >>>> On Thu, May 10, 2012 at 11:30 AM, szimszon <szims...@gmail.com> wrote: >>>> >>>>> Sorry I don't understand. What do you mean "achieve with join"? >>>>> >>>>> There is an empty for loop with db.executesql() without join. And it >>>>> is eating up the memory. :( >>>>> >>>>> 2012. május 10., csütörtök 19:12:30 UTC+2 időpontban Richard a >>>>> következőt írta: >>>>> >>>>>> You can't manage what you want to achieve with join? >>>>>> >>>>>> Richard >>>>>> >>>>>> On Thu, May 10, 2012 at 10:48 AM, szimszon <szims...@gmail.com>wrote: >>>>>> >>>>>>> Sorry for my dumbness but if something is wrong with my code please >>>>>>> point me the right line. I'm not so good in English if it comes to >>>>>>> "object >>>>>>> instance count" and so. Yeah I know I should go and do some milkmaid >>>>>>> job :) >>>>>>> but I'm curious. >>>>>>> >>>>>>> I'm just define some variable: >>>>>>> >>>>>>> lista = list() >>>>>>> last_row = None >>>>>>> next_page_number = 0 >>>>>>> >>>>>>> Go in a for loop that just assign the db query result one-by-one to >>>>>>> row variable. >>>>>>> >>>>>>> After that I assign the TABLE() helper to a list variable. That is >>>>>>> it. >>>>>>> >>>>>>> >>>>>>> >>>>>>> 2012. május 10., csütörtök 16:34:49 UTC+2 időpontban Bruce Wade a >>>>>>> következőt írta: >>>>>>> >>>>>>>> Using direct sql query or DAL is going to cause the exact same >>>>>>>> problem in this situation. >>>>>>>> >>>>>>>> On Thu, May 10, 2012 at 7:28 AM, szimszon <szims...@gmail.com>wrote: >>>>>>>> >>>>>>>>> It's postgres:// in a >>>>>>>>> Version 1.99.7 (2012-04-23 11:26:23) dev of web2py, and >>>>>>>>> >>>>>>>>> Python 2.7.3 (default, Apr 20 2012, 22:44:07) >>>>>>>>> [GCC 4.6.3] on linux2 >>>>>>>>> >>>>>>>>> python-psycopg2 2.4.5-1 >>>>>>>>> >>>>>>>>> >>>>>>>>> 2012. május 10., csütörtök 15:40:36 UTC+2 időpontban rochacbruno a >>>>>>>>> következőt írta: >>>>>>>>> >>>>>>>>>> Just for curiosity, what happens if you do it in pure sql? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> for row in db.executesql("**sele********ct * from >>>>>>>>>> file_properties where id > 0"): >>>>>>>>>> >>>>>>>>>> # do something >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Does it have a lower memory usage? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, May 10, 2012 at 4:14 AM, Bruce Wade <<bruce.w...@gmail.com> >>>>>>>>>> > wrote: >>>>>>>>>> >>>>>>>>>>> for row in db( db.file_**propert********ies.id > 0 ).select( >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> Bruno Rocha >>>>>>>>>> [http://rochacbruno.com.br] >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> -- >>>>>>>> Regards, >>>>>>>> Bruce Wade >>>>>>>> http://ca.linkedin.com/in/**bruc******elwade<http://ca.linkedin.com/in/brucelwade> >>>>>>>> http://www.wadecybertech.com >>>>>>>> http://www.fittraineronline.**co******m<http://www.fittraineronline.com>- >>>>>>>> Fitness Personal Trainers Online >>>>>>>> http://www.warplydesigned.com >>>>>>>> >>>>>>>> >>>>>> >>>> >>>> >>>> -- >>>> -- >>>> Regards, >>>> Bruce Wade >>>> http://ca.linkedin.com/in/**bruc**elwade<http://ca.linkedin.com/in/brucelwade> >>>> http://www.wadecybertech.com >>>> http://www.fittraineronline.**co**m <http://www.fittraineronline.com>- >>>> Fitness Personal Trainers Online >>>> http://www.warplydesigned.com >>>> >>>> >>
