I would suggest you move some tables to conditional models. If you have a lot of
Field(....,requires=IS_NO_EMPTY()) you can ne = IS_NO_EMPTY() and then reuse the same object Field(,....requires=ne) You can do this for all validators. Set defaults in controllers if complex. you may want to do from mytables inport define_tables define_tables(db) and move the table definition in modules/mytables.py On Aug 15, 7:18 pm, Kevin Ivarsen <kivar...@gmail.com> wrote: > Hi Anthony, > > Thanks for the quick response. > > I do use migrate=False already. I have not tried bytecode compiling > the app, but earlier today I did a quick test: I wrapped the contents > of my long db.py in an "if False:" block (causing it to compile, but > not run, the code for each request), and compared this to an empty > db.py (nothing to compile or run). It looks like compiling the code > took about 7ms - about 5% of the total overhead. I don't think > bytecode compiling will produce the dramatic improvement that I'm > hoping to find (though I will of course try it when I get a chance.) > > Conditional models might help some - at the expense of additional > planning and maintenance required during development. In any case, I > still worry about the number of tables that need to be shared across > at least two controllers, and the lower bound on performance those > will impose. I was hoping to find a more global way to speed up > models. > > Optimizing DAL creation would be an interesting area for future web2py > development, IMHO. When I get some more time I might try to look at > this in more detail. > > Cheers, > Kevin > > On Aug 15, 3:59 pm, Anthony <abasta...@gmail.com> wrote: > > > > > > > > > First, have you set migrate to False for all tables, and have you bytecode > > compiled the app? That should speed things up. > > > Also, do you need all 70 tables for any requests, or can they be broken up > > based on the particular controller/function called? If so, you may speed > > things up with conditional models. Any files in /models/controller1 will > > only be executed on requests for controller1, and any files in > > /models/controller1/function1 will only be executed on requests for > > controller1/function1. (We're working on more general functionality for > > conditional models that won't be based solely on the controller and > > function.) > > > Anthony > > > On Monday, August 15, 2011 6:30:06 PM UTC-4, Kevin Ivarsen wrote: > > > Hello, > > > > I'm using web2py with a relatively large legacy database. We have > > > about 70 tables, and models/db.py is close to 1000 lines long. > > > > The problem I'm facing is that all of these Field() constructors and > > > define_table() calls take about 150-200ms to execute, and this > > > overhead occurs each time any request is made to web2py. Compare this > > > to a minimal web2py app, which might have a (reasonable) 10-30ms of > > > overhead before response data starts flowing to the client. > > > > I was hoping that a simple solution would be to declare the DAL > > > structure once in an importable module, which would be run once on > > > web2py startup rather than at each request. I could then deepcopy it > > > in my model file (which I think would be faster than all the calls to > > > define_table(), Field(), etc.), and then establish a connection to the > > > database from this copied DAL. > > > > Unfortunately, there are a few problems with this: > > > 1. If a DAL instance is to be connected to the database, it must > > > happen in the constructor. It doesn't seem that you can do "db = > > > DAL(None)" and then establish a connection after the fact. Also, it > > > looks like some db-specific behavior is set up in the DAL constructor > > > based on the connection URL - this wouldn't happen in the case of > > > DAL(None). > > > > 2. Table and Field instances have a reference to db, so it seems > > > that define_table() needs to be called *after* the DAL connection has > > > been established in order to set up these references. > > > > I suppose it would be possible to deepcopy a DAL(None) instance that > > > has been established with Tables and Fields, and monkey-patch the db > > > references thoughout the DAL structure, but chances seem good that > > > this will create subtle bugs down the road if the internal DAL > > > implementation ever changes. > > > > Can anyone suggest a better way to speed up DAL creation for large > > > schemas? > > > > Thanks, > > > Kevin
