The web2py DAL sipmly provides an interface to your database. There is no overhead that depends on the size of your dataset. If you database can handle it, web2py can.
There are exceptions. For example the requires=IS_IN_DB(..,) validator builds an in-memory list of all the references in order to make the HTML dropdown. This is ok for a small number of references. You want to validate without the dropdown (requires=[IS_IN_DB(...)], note the []) and perhaps implement auto completion (it is discussed in the book). You also have to avoid db(...).select() but use pagination db (...).select(limitby=(min,max)) instead. appadmin does it by default. Some plugins do not do pagination and break on large datasets. You have to ask their authors. MySQL, PostgreSQL, Firebird, Informix, and DB2 handle pagination well. MSSQL and Oracle do not. Avoid them. Massimo On Jan 14, 9:12 am, Johann Spies <johann.sp...@gmail.com> wrote: > Is web2py suitable if I want to work with large datasets? > > I am currently developing a database and want to use web2py to make it > available to the client. > > Up to now I was using the shell and appadmin interfaces to the databasis. > > When trying out the webgrid-slice > fromhttp://www.web2pyslices.com/main/slices/take_slice/39and also the > "Quick Table Management Snippet" > fromhttp://www.web2pyslices.com/main/slices/take_slice/42to develop an > interface to one of the tables containing about 168800 records python > used up all the resources on my computer (more than 3.4G of memory) > and I had to kill the process. > > In both cases I referred to the table as the datasource. > > What I do not understand is that in the appadmin interface I do not > have the same problem. > > How do I prevent web2py loading whole dataset into memory? After all > what is the use of a sql database if the everything is loaded into > RAM? > > Regards > Johann
-- You received this message because you are subscribed to the Google Groups "web2py-users" group. To post to this group, send email to web...@googlegroups.com. To unsubscribe from this group, send email to web2py+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/web2py?hl=en.
