On Sunday 07 June 2009 11:13:49 Alexey Nezhdanov wrote: > 1) When initialising, SQLField uses dir() call to find any clashes > with reserved names. I was a bit incorrect here. It checks for already defined fields too. Anyways - I optimised it by commenting it out :-P In practice I'd recommend similar approach: 1) don't check for clashes in the production environment (SQLDB init parameter?) 2) (optional) maintain cache of names for each table. Don't use dir().
> 2) is_integer is a fast call, but with 1.1k (!) calls ... Replaced it with my own version: integer_pat=re.compile('[0-9]+$') is_integer=lambda x: integer_pat.match(x) it's about 2.3 times faster. C version would be even better. > 3) sqlhtml_validators is an obvious bug... 8/9 of work is wasted. > ... I suspect that dictionary building may be moved into module namespace did that with some tweaks (same validators get reused). Dangerous approach may introduce hidden dataflow, but that could be prevented by somehow making these validators read-only. Got another 9.5% model time speedup. (0.096s vs 0.106s). Though this figures get more and more meaningless as model init time reduces it's share in total page generation time. Probably I'll just switch to time measuring of urllib.urlopen(). ...OK, switched. Here are stats. Each line is in the format model_init_time/urlopen_measured_time - average over 100 calls These timings are not comparable with my previous emails because I did some additional arrangements that should make these times more than usual, but more stable across tests. Also application is compiled this time. 1) no optimisations 0.1219/0.1649 2) skip CREATE TABLE in SQLTable init 0.0865/0.1261 3) + three steps, described above in this mail. 0.0439/0.0855 4) + lazy tables init in model This one is application specific. For instance in my application it causes 4 tables out of 16 to be initialised in model and 2 more - in controller. This is for front page. Other pages will have different figures. Yet, here are timings: 0.0194/0.0622 5) And finally, for mere comparison, timings of (4) again, but this time with non-compiled source. 0.0208/0.1216 Some conclusions: 1) always compile your app for production. It saves you about 50% CPU. 2) With very little effort it was possible to speedup web2py app further 2.65 times, dropping page generation times from tenths of second to hundredths. I hope at least some of that work will find it's way into mainline. I'm attaching all patches that I used to this mail. These patches are against quite old web2py - Version 1.61.4 (2009-04-21 10:02:50) (I didn't like how newer versions looked like so I stick to initially installed version). -- Sincerely yours Alexey Nezhdanov --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "web2py Web Framework" group. To post to this group, send email to web2py@googlegroups.com To unsubscribe from this group, send email to web2py+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/web2py?hl=en -~----------~----~----~----~------~----~------~--~---
--- sql.py.old 2009-05-23 14:28:15.286720712 +0400 +++ sql.py 2009-06-07 12:38:16.509986136 +0400 @@ -1053,6 +1058,8 @@ return False return True +integer_pat=re.compile('[0-9]+$') +is_integer=lambda x: integer_pat.match(x) #0m3.688s class SQLTable(dict):
--- sql.py.old 2009-05-23 14:28:15.286720712 +0400 +++ sql.py 2009-06-07 12:38:56.777987959 +0400 @@ -487,6 +487,10 @@ """ def __getitem__(self, key): + value = dict.__getitem__(self, str(key)) + if not callable(value) or key[0]=='_' or not isinstance(value, types.FunctionType): return value + print 'calling',value + value.__call__() # That must redefine table in-place return dict.__getitem__(self, str(key)) def __setitem__(self, key, value):
--- sql.py.old 2009-05-23 14:28:15.286720712 +0400 +++ sql.py 2009-06-07 11:54:40.977982790 +0400 @@ -331,12 +331,9 @@ } -def sqlhtml_validators(field_type, length): - v = { +preset_validators = { 'boolean': [], - 'string': validators.IS_LENGTH(length), 'text': validators.IS_LENGTH(2 ** 16), - 'password': validators.IS_LENGTH(length), 'blob': [], 'upload': [], 'double': validators.IS_FLOAT_IN_RANGE(-1e100, 1e100), @@ -346,10 +343,19 @@ 'datetime': validators.IS_DATETIME(), 'reference': validators.IS_INT_IN_RANGE(0, 1e100), } - try: - return v[field_type[:9]] - except KeyError: - return [] +preset_validators2 = { + 'string': {}, + 'password': {}, + } +def sqlhtml_validators(field_type, length): + typ=field_type[:9] + try: return preset_validators[typ] + except: + try: return preset_validators2[typ][length] + except: + if typ not in preset_validators2: return [] + preset_validators2[typ]=validator=validators.IS_LENGTH(length) + return validator def sql_represent(obj, fieldtype, dbname):
--- sql.py.old 2009-05-23 14:28:15.286720712 +0400 +++ sql.py 2009-06-07 12:39:48.649986330 +0400 @@ -1607,8 +1608,8 @@ ): self.name = fieldname = cleanup(fieldname) - if fieldname in dir(SQLTable) or fieldname[0] == '_': - raise SyntaxError, 'SQLField: invalid field name' +# if fieldname in dir(SQLTable) or fieldname[0] == '_': +# raise SyntaxError, 'SQLField: invalid field name' if isinstance(type, SQLTable): type = 'reference ' + type._tablename if not length and type == 'string':
--- sql.py.old 2009-05-23 14:28:15.286720712 +0400 +++ sql.py 2009-06-07 12:40:39.565985360 +0400 @@ -930,13 +930,14 @@ if self._uri == 'None': args['migrate'] = False return t - sql_locker.acquire() - try: + if args['migrate']: + sql_locker.acquire() + try: query = t._create(migrate=args['migrate']) - except BaseException, e: + except BaseException, e: sql_locker.release() raise e - sql_locker.release() + sql_locker.release() self.tables.append(tablename) return t