Perhaps you need to write your own csv export. Save a csv file to /tmp instead and then when your finished creating your csv file then stream it for download.
This might reduce the memory requirements? Though not sure what you can do about the fact web2py cache's your select into memory, maybe do it in chunks as you generate the csv file? -Thadeus On Sat, Jan 30, 2010 at 2:37 PM, mdipierro <mdipie...@cs.depaul.edu> wrote: > The problem is that all records are copied into memory before being > serialized + the entire csv is copied into memory. We need to find a > better way when exporting a large of records. > > On Jan 30, 12:19 pm, Johann Spies <johann.sp...@gmail.com> wrote: >> It happened more than once that a csv-export using appadmin resulted >> in incomplete data both with Postgresql-backend and sqlite. >> >> This one was on sqlite: >> >> Selected records: 139413 >> csv-exported (first effort): 287 records. >> csv-exported (second effort) 216 records >> >> This after a lot of time and slowing down the computer to a near standstill. >> >> I suppose it might be easier and more reliable directly form the database. >> >> Regards >> Johann > > -- > You received this message because you are subscribed to the Google Groups > "web2py-users" group. > To post to this group, send email to web...@googlegroups.com. > To unsubscribe from this group, send email to > web2py+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/web2py?hl=en. > > -- You received this message because you are subscribed to the Google Groups "web2py-users" group. To post to this group, send email to web...@googlegroups.com. To unsubscribe from this group, send email to web2py+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/web2py?hl=en.
