Hi Anthony, Thanks for the quick response.
I do use migrate=False already. I have not tried bytecode compiling the app, but earlier today I did a quick test: I wrapped the contents of my long db.py in an "if False:" block (causing it to compile, but not run, the code for each request), and compared this to an empty db.py (nothing to compile or run). It looks like compiling the code took about 7ms - about 5% of the total overhead. I don't think bytecode compiling will produce the dramatic improvement that I'm hoping to find (though I will of course try it when I get a chance.) Conditional models might help some - at the expense of additional planning and maintenance required during development. In any case, I still worry about the number of tables that need to be shared across at least two controllers, and the lower bound on performance those will impose. I was hoping to find a more global way to speed up models. Optimizing DAL creation would be an interesting area for future web2py development, IMHO. When I get some more time I might try to look at this in more detail. Cheers, Kevin On Aug 15, 3:59 pm, Anthony <abasta...@gmail.com> wrote: > First, have you set migrate to False for all tables, and have you bytecode > compiled the app? That should speed things up. > > Also, do you need all 70 tables for any requests, or can they be broken up > based on the particular controller/function called? If so, you may speed > things up with conditional models. Any files in /models/controller1 will > only be executed on requests for controller1, and any files in > /models/controller1/function1 will only be executed on requests for > controller1/function1. (We're working on more general functionality for > conditional models that won't be based solely on the controller and > function.) > > Anthony > > > > > > > > On Monday, August 15, 2011 6:30:06 PM UTC-4, Kevin Ivarsen wrote: > > Hello, > > > I'm using web2py with a relatively large legacy database. We have > > about 70 tables, and models/db.py is close to 1000 lines long. > > > The problem I'm facing is that all of these Field() constructors and > > define_table() calls take about 150-200ms to execute, and this > > overhead occurs each time any request is made to web2py. Compare this > > to a minimal web2py app, which might have a (reasonable) 10-30ms of > > overhead before response data starts flowing to the client. > > > I was hoping that a simple solution would be to declare the DAL > > structure once in an importable module, which would be run once on > > web2py startup rather than at each request. I could then deepcopy it > > in my model file (which I think would be faster than all the calls to > > define_table(), Field(), etc.), and then establish a connection to the > > database from this copied DAL. > > > Unfortunately, there are a few problems with this: > > 1. If a DAL instance is to be connected to the database, it must > > happen in the constructor. It doesn't seem that you can do "db = > > DAL(None)" and then establish a connection after the fact. Also, it > > looks like some db-specific behavior is set up in the DAL constructor > > based on the connection URL - this wouldn't happen in the case of > > DAL(None). > > > 2. Table and Field instances have a reference to db, so it seems > > that define_table() needs to be called *after* the DAL connection has > > been established in order to set up these references. > > > I suppose it would be possible to deepcopy a DAL(None) instance that > > has been established with Tables and Fields, and monkey-patch the db > > references thoughout the DAL structure, but chances seem good that > > this will create subtle bugs down the road if the internal DAL > > implementation ever changes. > > > Can anyone suggest a better way to speed up DAL creation for large > > schemas? > > > Thanks, > > Kevin
