[sqlalchemy] Re: SQLAlchemy 0.4 beta2 released
On Wednesday 15 August 2007 04:26:31 Michael Bayer wrote: On Aug 14, 2007, at 4:35 PM, Michael Bayer wrote: On Aug 14, 2007, at 12:38 PM, svilen wrote: --- orm.attribute AttributeManager.init_attr(): the saving this one eventualy does is too small, compared to a property call of ._state. i havent benched this in a while but my recollection is that the AttributeError raise is *much* slower than pre-calling this method. a single function call is always faster than an exception throw. however, i see that the exception throw is being suppressed also with a hasattr() being called every timeim not sure why thats that way now so i might change it back to throwing AttributeError. the results are in, running test/prof/masseagerload.py. this test is very heavy on creating new instances from mapper rows, which is where the initialization of _state comes in. no init_attr(), detect missing with AttributeError Profiled target 'masseagerload', wall time: 0.59 seconds Profile report for target 'masseagerload' (masseagerload.prof) 57039 function calls (55962 primitive calls) in 0.489 CPU seconds init_attr(), detect missing with AttributeError Profiled target 'masseagerload', wall time: 0.53 seconds 57549 function calls (56472 primitive calls) in 0.426 CPU seconds init_attr(), detect missing with hasattr Profiled target 'masseagerload', wall time: 0.56 seconds 57549 function calls (56472 primitive calls) in 0.431 CPU seconds no init_attr(), detect missing with hasattr Profiled target 'masseagerload', wall time: 0.49 seconds 57039 function calls (55962 primitive calls) in 0.390 CPU seconds im not exactly sure why the hasattr() call, being present and then removed, doesnt change the number of function calls. anyway, the times vary a little bit but the hasattr call, even though its called many more times than the AttributeError gets raised, is slightly faster than raising AttributeError. so no AttributeError, and I like getting rid of init_attr() very much so its out in r3313. heh. First, the init_attr as it was, has never worked - it sets up a private __sa_attr_state, while what is used is plain non-private _sa_attr_state attribute (note starting underscores). Second, i went to r3312, let init_attr() set a _state as plain dict and removed _state as property. The difference plain-dict/property (in favor of plain dict) is like 2-3%. property: loaded 10 items each with 500 subitems 523034 function calls (512957 primitive calls) in 2.556 CPU s loaded 30 items each with 500 subitems 1564374 function calls (1534297 primitive calls) in 7.796 CPU s loaded 16 items each with 1500 subitems 2499436 function calls (2451359 primitive calls) in 12.518 CPU s plain dict: loaded 10 items each with 500 subitems 513014 function calls (502937 primitive calls) in 2.525 CPU s loaded 30 items each with 500 subitems 1534314 function calls (1504237 primitive calls) in 7.623 CPU s loaded 16 items each with 1500 subitems 2451404 function calls (2403327 primitive calls) in 12.196 CPU s up to you --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: SQLAlchemy 0.4 beta2 released
On Tuesday 14 August 2007 23:05:44 Michael Bayer wrote:
On Aug 14, 2007, at 3:30 PM, [EMAIL PROTECTED] wrote:
databases/sqlite: (reflecttable)
pragma_names is missing the BOOLEAN word/type - nulltype
btw why isn't each dialect-typeclass adding it's own entry to
that pragma_names, respectively to the colspecs ?
Or, each class to have those pragmaword and basetype, and the
dicts to be made by walking locals() if issubclass(..) ?
Anyway, these dicts (the grammar) should be automaticaly built
from available typeclasses...
patches welcome
here 2 versions.
One is simple, walking the module.namespace for
issubclass(TypeEngine), expecting to find .pragma and .colspec in
that class and collects them. The .colspec can probably be figured
out from __bases__ (as in other version)
pre
def _issubclass( obj, klas):
'fail/fool-proof issubclass() - works with ANY argument'
from types import ClassType
return isinstance(obj,(type,ClassType)) and issubclass(obj,klas)
def collect_colspecs( namespace): #this can be moved out of here
colspecs = {}
pragma_names = {}
for obj in namespace.itervalues():
if _issubclass( kl, sqlalchemy.TypeEngine):
c = getattr( kl, 'colspec', None) #or 'basetype'
p = getattr( kl, 'pragma', None) #or 'sqltype' or rawtype
if c and p:
colspec[c]=kl
pragma_names[c]=kl
return colspecs, pragma_names
class SLNumeric(sqltypes.Numeric):
colspec,pragma = sqltypes.Numeric, 'NUMERIC'
def get_col_spec(self):
if self.precision is None:
return NUMERIC
else:
return NUMERIC(%(precision)s, %(length)s)%self.__dict__
class SLInteger(sqltypes.Integer):
colspec,pragma = sqltypes.Integer, 'INTEGER'
def get_col_spec(self): return self.pragma
...
colspecs, pragma_names = collect_colspecs( locals() )
/pre
the other one uses metaclass, and .pragma is set up, and guesses
colspec's abstract_type from __bases.
pre
class MetaDialectType( type): #this can be moved out of here
def __new__( metacls, name, bases, dict_):
#find top-most abstract_type base
abstract_type = None
for b in bases:
#XXX is walk in depth needed?
#e.g. if allowed class SLInt2( SLInteger):...
if issubclass( b, sqltypes.TypeEngine):
abstract_type = b
break
assert abstract_type, 'class %s: cannot find any abstract \
base type; do inherit from some sqlalchemy type' % name
try:
pragma = dict_['pragma']
except KeyError:
assert 0, 'class %s: cannot find any pragma' % name
klas = type.__new__( metacls, name, bases, dict_)
metacls.colspecs[ abstract_type] = klas
metacls.pragma_names[ pragma]=klas
return klas
class SLMetaDialectType( MetaDialectType):
colspecs = {}
pragma_names = {}
class SLNumeric( sqltypes.Numeric):
__metaclass__ = SLMetaDialectType
pragma = 'NUMERIC'
def get_col_spec(self):
r = self.pragma
if self.precision is not None:
r += (%(precision)s, %(length)s) % self.__dict__
return r
class SLInteger( sqltypes.Integer):
__metaclass__ = SLMetaDialectType
pragma = 'INTEGER'
def get_col_spec(self): return self.pragma
...
colspecs = SLMetaDialectType.colspecs
pragma_names = SLMetaDialectType.pragma_names
/pre
==
There are 2 choices to make:
- walk locals() vs using metaclass
- whether to touch get_col_spec()s
i wanted to have everything specified only once.
Therefore the get_col_spec() redefinition.
It can be:
1 left as is, just adding a separate .pragma (no gain,
consistency-wise, e.g. VARCHR in one place and VARCHAR in another)
2 remade to use the self.pragma where equivalent (like 80% of
places) - a lot of same code repeated
3 put a default one in some base class for all dialect-types, e.g.
DialectBaseType, which can be then used for filtering locals() or to
bring metaclass
4 created in the metaclass unless explicitly specified - this is most
obscure.
btw i suggest some namechanges, colspec - abstract_type and
pragma_name - rawdb_type; or something alike.
ciao
svil
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: SQLAlchemy 0.4 beta2 released
I had in mind that the metaclass approach would be used, but not
necesarily with the walking stuff going on.
if you really want to think about this, the idea for a types overhaul
is ticket #526. that breaks up the DDL from the adaptation side
of things. a metaclass approach would be at the base of it
controlling a registry of information about types.
On Aug 15, 2007, at 12:15 PM, [EMAIL PROTECTED] wrote:
On Tuesday 14 August 2007 23:05:44 Michael Bayer wrote:
On Aug 14, 2007, at 3:30 PM, [EMAIL PROTECTED] wrote:
databases/sqlite: (reflecttable)
pragma_names is missing the BOOLEAN word/type - nulltype
btw why isn't each dialect-typeclass adding it's own entry to
that pragma_names, respectively to the colspecs ?
Or, each class to have those pragmaword and basetype, and the
dicts to be made by walking locals() if issubclass(..) ?
Anyway, these dicts (the grammar) should be automaticaly built
from available typeclasses...
patches welcome
here 2 versions.
One is simple, walking the module.namespace for
issubclass(TypeEngine), expecting to find .pragma and .colspec in
that class and collects them. The .colspec can probably be figured
out from __bases__ (as in other version)
pre
def _issubclass( obj, klas):
'fail/fool-proof issubclass() - works with ANY argument'
from types import ClassType
return isinstance(obj,(type,ClassType)) and issubclass(obj,klas)
def collect_colspecs( namespace): #this can be moved out of here
colspecs = {}
pragma_names = {}
for obj in namespace.itervalues():
if _issubclass( kl, sqlalchemy.TypeEngine):
c = getattr( kl, 'colspec', None) #or 'basetype'
p = getattr( kl, 'pragma', None) #or 'sqltype' or rawtype
if c and p:
colspec[c]=kl
pragma_names[c]=kl
return colspecs, pragma_names
class SLNumeric(sqltypes.Numeric):
colspec,pragma = sqltypes.Numeric, 'NUMERIC'
def get_col_spec(self):
if self.precision is None:
return NUMERIC
else:
return NUMERIC(%(precision)s, %(length)s)%self.__dict__
class SLInteger(sqltypes.Integer):
colspec,pragma = sqltypes.Integer, 'INTEGER'
def get_col_spec(self): return self.pragma
...
colspecs, pragma_names = collect_colspecs( locals() )
/pre
the other one uses metaclass, and .pragma is set up, and guesses
colspec's abstract_type from __bases.
pre
class MetaDialectType( type): #this can be moved out of here
def __new__( metacls, name, bases, dict_):
#find top-most abstract_type base
abstract_type = None
for b in bases:
#XXX is walk in depth needed?
#e.g. if allowed class SLInt2( SLInteger):...
if issubclass( b, sqltypes.TypeEngine):
abstract_type = b
break
assert abstract_type, 'class %s: cannot find any abstract \
base type; do inherit from some sqlalchemy type' % name
try:
pragma = dict_['pragma']
except KeyError:
assert 0, 'class %s: cannot find any pragma' % name
klas = type.__new__( metacls, name, bases, dict_)
metacls.colspecs[ abstract_type] = klas
metacls.pragma_names[ pragma]=klas
return klas
class SLMetaDialectType( MetaDialectType):
colspecs = {}
pragma_names = {}
class SLNumeric( sqltypes.Numeric):
__metaclass__ = SLMetaDialectType
pragma = 'NUMERIC'
def get_col_spec(self):
r = self.pragma
if self.precision is not None:
r += (%(precision)s, %(length)s) % self.__dict__
return r
class SLInteger( sqltypes.Integer):
__metaclass__ = SLMetaDialectType
pragma = 'INTEGER'
def get_col_spec(self): return self.pragma
...
colspecs = SLMetaDialectType.colspecs
pragma_names = SLMetaDialectType.pragma_names
/pre
==
There are 2 choices to make:
- walk locals() vs using metaclass
- whether to touch get_col_spec()s
i wanted to have everything specified only once.
Therefore the get_col_spec() redefinition.
It can be:
1 left as is, just adding a separate .pragma (no gain,
consistency-wise, e.g. VARCHR in one place and VARCHAR in another)
2 remade to use the self.pragma where equivalent (like 80% of
places) - a lot of same code repeated
3 put a default one in some base class for all dialect-types, e.g.
DialectBaseType, which can be then used for filtering locals() or to
bring metaclass
4 created in the metaclass unless explicitly specified - this is most
obscure.
btw i suggest some namechanges, colspec - abstract_type and
pragma_name - rawdb_type; or something alike.
ciao
svil
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to
[sqlalchemy] Re: SQLAlchemy 0.4 beta2 released
On Wednesday 15 August 2007 19:51:30 Michael Bayer wrote: On Aug 15, 2007, at 10:52 AM, [EMAIL PROTECTED] wrote: Second, i went to r3312, let init_attr() set a _state as plain dict and removed _state as property. The difference plain-dict/property (in favor of plain dict) is like 2-3%. property: loaded 10 items each with 500 subitems 523034 function calls (512957 primitive calls) in 2.556 CPU s loaded 30 items each with 500 subitems 1564374 function calls (1534297 primitive calls) in 7.796 CPU s loaded 16 items each with 1500 subitems 2499436 function calls (2451359 primitive calls) in 12.518 CPU s plain dict: loaded 10 items each with 500 subitems 513014 function calls (502937 primitive calls) in 2.525 CPU s loaded 30 items each with 500 subitems 1534314 function calls (1504237 primitive calls) in 7.623 CPU s loaded 16 items each with 1500 subitems 2451404 function calls (2403327 primitive calls) in 12.196 CPU s up to you great, post a patch for that on trac. #741, using _sa_attr_state, set-up in mapper._create_instance() btw (if i havent asked yet) - do u have a place describing all things that happen to some user's object/class once given into SA's arms? e.g. all the attributes that grow on the instances (and has to be avoided by user), replacement of __init__, all the descriptors for attributes, etc? i can prepare something like half a page on this theme, but u'll have to fill up as i surely have no much idea of which is what/why. Title like impact on user's object and class --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: SQLAlchemy 0.4 beta2 released
On Wednesday 15 August 2007 20:54:27 Michael Bayer wrote: I had in mind that the metaclass approach would be used, but not necesarily with the walking stuff going on. the walking is a quick and dirty and very simple way to get away with it - for now. if you really want to think about this, the idea for a types overhaul is ticket #526. that breaks up the DDL from the adaptation side of things. a metaclass approach would be at the base of it controlling a registry of information about types. types... my static_type lib is just about types, and all the metainfo u could hang on them... lets see. u have multiple types (and conversions) here, for same item: a) the SA-abstract type in schema - eg. types.String b) the specific dialect implementation of a), e.g. sqlite.SLText c) the python type that is expected to live between SAdialect and dbapi, both ways d) the actual SQL server name/representation for the type e) python type that is expected to come out from SA of when loading, eg. unicode f) python type that is expected to go into SA when assigning. eg. str/utf8 g) validation for e) (to SA) - so x.myint = 'qq' fails (or x.mystr=12 succeeds) h) validation for f) (from SA) - so attempting to load x.myint from column containing 'qq' fails my experience says conversion and validation is same ONE thing, implementation-wise. u make one of these notions and use it to implement both, e.g. i have static_type.validators which i use for conversion and/or validation. The g,h conversions/validations are outside of sql-related-scope, they are only application-side related. i guess they will be mostly user-specified, with very few automatical. For example, in dbcook i set up another layer of type-translation on top of SA, exactly to address this issue. So application types stay applicational, and match to whatever (independently) at persistency level (SA). Thus i can use same model file, with same names, once having the (my) types as heavy-checking statictypes, and once as empty classes (plainwrap.py) used just to match the SA-type underneath. Each dialect keeps its own registry of types, used for bidirectional abstract-dialect match, plus the actual sql (pragma) stuff like typenames/args and retrieving them back for reflection. This coresponds to current colspecs and pragma_names dicts. Reflection should be configurable whether to stop at dialect level (SLint) or go back to abstract types (types.Int) - see my autoload.py. are there any cases of diality? e.g. str1 and str2 both being str? current situation: IMO right now typewise all is ok, but u have one conversion only, happening inside the dialect, implicitly, cannot be extended/chained; and registries are separated and inconsistent and hard to find. you wanted also specific converters as per dialect, e.g. python.str(-SA.date-SA.sqlite.SLdate)-sqlite.datetime might be different from python.str(-SA.date-SA.postgress.PGdate)-psycopg.datetime so, do u want the Convertors/validators network to follow the way of Types network? i.e. abstract convertors and (actual) dialect-implementations? in another registry? Are there going to be priorities/ordering in (auto) matching of convertors? e.g. u may have str2int and str2anynumber, which to choose... btw. chaining converters changes/pushes expected (i/o) pythontype. The question what is the expected in/out pythontype? should ask the first converter in the chain from user-side, and result can be singular or multiple, e.g. anything to str, or (bool,int,long,float) to float). am i missing something so far? btw why isn't each dialect-typeclass adding it's own entry to that pragma_names, respectively to the colspecs ? Anyway, these dicts (the grammar) should be automaticaly built from available typeclasses... patches welcome here 2 versions. One is simple, walking the module.namespace for issubclass(TypeEngine), expecting to find .pragma and .colspec in that class and collects them. The .colspec can probably be figured out from __bases__ (as in other version) the other one uses metaclass, and .pragma is set up, and guesses colspec's abstract_type from __bases. == There are 2 choices to make: - walk locals() vs using metaclass - whether to touch get_col_spec()s i wanted to have everything specified only once. Therefore the get_col_spec() redefinition. It can be: 1 left as is, just adding a separate .pragma (no gain, consistency-wise, e.g. VARCHR in one place and VARCHAR in another) 2 remade to use the self.pragma where equivalent (like 80% of places) - a lot of same code repeated 3 put a default one in some base class for all dialect-types, e.g. DialectBaseType, which can be then used for filtering locals() or to bring metaclass 4 created in the metaclass unless explicitly specified - this is most obscure. btw i suggest some namechanges, colspec - abstract_type and pragma_name
[sqlalchemy] Re: SQLAlchemy 0.4 beta2 released
Reflection should be configurable whether to stop at dialect level (SLint) or go back to abstract types (types.Int) - see my autoload.py. why would one want to stop the reflection from going back to abstract types? i.e. if the current reflection (dialevt-level) is made to autoguess the abstract SA type, would this break something? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: SQLAlchemy 0.4 beta2 released
On Thursday 16 August 2007 00:33:57 [EMAIL PROTECTED] wrote: Reflection should be configurable whether to stop at dialect level (SLint) or go back to abstract types (types.Int) - see my autoload.py. why would one want to stop the reflection from going back to abstract types? i.e. if the current reflection (dialevt-level) is made to autoguess the abstract SA type, would this break something? Answering myself, dialects may have richer types that SA.abstract ones, in which case extra ones stay as is. Also, seems there are duplicates, because of which the abstract-dialect is not 1:1, e.g. postgres has bigint and int that are both SA.integer. how about these? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: SQLAlchemy 0.4 beta2 released
performance-wise - do u have any test/target for profiling? else i can
repeat some tests i did somewhen in february (if i remember them..)
=
while looking to replace all {} with dict/Dict(), i found some things.
Here the list, quite random, probably some can be just ignored if not
an actual issue - i may have misunderstood things; have a look.
(btw The overall candidates for the replacement are like 60-70 lines,
all else are kwargs or lookup-tables.)
---
database/informix.py:
ischema_names = { ... } has duplicate keys/entries
---
database/*
get_col_spec(self) etc:
these string-formats may be better without the artificial dict,
eg. return self._extend(CHAR(%(length)s) % {'length': self.length})
- return self._extend(CHAR(%s) % self.length )
or
- return self._extend(CHAR(%(length)s) % self.__dict__ )
no idea if get_col_spec() is used that much to have crucial impact on
speed though, at least it looks simpler.
---
orm.util.AliasedClauses._create_row_adapter()
class AliasedRowAdapter( object):
1. can't this be made as standalone class, returning an instance,
initialized with the map, which is then __call__()ed ?
2. this can be faster if:
a) has_key = __contains__ #instead of yet another funccall
b) __getitem__ uses try except instead of double lookup key in map
---
orm.mapper
Mapper._instance():
WTF is the **{'key':value, ... } ?
eg. if extension.populate_instance(self, context, row, instance,
**{'instancekey':identitykey, 'isnew':isnew}) ...
same thing is done as separate variable a page later;
btw there are several of these **{} in the file
also, Mapper._options is redundant (leftover?) neverused
---
orm.attribute
AttributeManager.init_attr():
the saving this one eventualy does is too small, compared to a
property call of ._state.
AttributeManager.register_attribute():
the def _get_state(self) that is made into as property _state can
be made eventualy faster with try-except instead of 'if'.
btw: cant that ._state property be removed alltogether (i.e. made a
plain attribute? then init_attr() MUST be there seting it up as plain
dict.
---
orm/unitofwork
UOWTask._sort_circular_dependencies():
def get_dependency_task(obj, depprocessor):
try:
dp = dependencies[obj]
except KeyError:
dp = dependencies.setdefault(obj, {})
isnt just the setdefault() enough?
---
engine.url.
def translate_connect_args(self, names):
this assumes the order of passed names matches the order of
attribute_names inside... very fragile. Why not use set of kwargs
like (attr_name=replacement_name defaulting to None), then just use
the non empty ones?
def _parse_rfc1738_args():
the 'opts' dict is redundant, would be cleaner if args are just
passed to URL( name=value)
---
topological.py
QueueDependencySorter.sort():
'cycles' is redundant neverused variable
---
util:
ThreadLocal:
- wouldnt be faster if the key in the _tdict is tuple(id,key) and
not some formatted string off these? or the key is nonhashable?
- the engine/threadlocal.TLEngine._session() issues a hasattr() on
such object. how does it actualy work? IMO it always fails
==
hey, thanks for the MetaData.reflect()!
svil
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: SQLAlchemy 0.4 beta2 released
On Aug 14, 11:29 am, Michael Bayer [EMAIL PROTECTED] wrote: I just put beta2 up. The important things in this release are major MS-SQL and Oracle fixes, after we got some on-the-ground testing done. Theres also a new function we're planning to use in Pylons called engine_from_config(), which reads a configuration dictionary (such as from a .ini file) and returns an Engine instance. Also some enhancements to Session.execute() which may become important as we move more towards the session-as-executor model; if you are using multiple engines, execute() will search through the ClauseElement for tables that it can link to bound engines, thus reducing the need to pass a mapper along to it. 0.4 looks awesome. Some names jump out at me though: * sessionmaker - Strange that it's a noun, and not a verb (like make_session) * bind/binds arguments - Strange that it's a verb, and not a noun (why not engine/engines?) Keep up the good work! -- Brian Beck / Adventurer of the First Order / www.brianbeck.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: SQLAlchemy 0.4 beta2 released
databases/sqlite: (reflecttable) pragma_names is missing the BOOLEAN word/type - nulltype btw why isn't each dialect-typeclass adding it's own entry to that pragma_names, respectively to the colspecs ? Or, each class to have those pragmaword and basetype, and the dicts to be made by walking locals() if issubclass(..) ? Anyway, these dicts (the grammar) should be automaticaly built from available typeclasses... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: SQLAlchemy 0.4 beta2 released
On Aug 14, 2007, at 3:30 PM, [EMAIL PROTECTED] wrote: databases/sqlite: (reflecttable) pragma_names is missing the BOOLEAN word/type - nulltype btw why isn't each dialect-typeclass adding it's own entry to that pragma_names, respectively to the colspecs ? Or, each class to have those pragmaword and basetype, and the dicts to be made by walking locals() if issubclass(..) ? Anyway, these dicts (the grammar) should be automaticaly built from available typeclasses... patches welcome --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: SQLAlchemy 0.4 beta2 released
On Aug 14, 2007, at 12:38 PM, svilen wrote:
performance-wise - do u have any test/target for profiling? else i can
repeat some tests i did somewhen in february (if i remember them..)
look in test/perf for some, just added a mass-insert/mass-select test.
also the current branch im doing today will chop out about 60% of
function call overhead from the abovementioned test.
---
database/*
get_col_spec(self) etc:
these string-formats may be better without the artificial dict,
eg. return self._extend(CHAR(%(length)s) % {'length': self.length})
- return self._extend(CHAR(%s) % self.length )
or
- return self._extend(CHAR(%(length)s) % self.__dict__ )
no idea if get_col_spec() is used that much to have crucial impact on
speed though, at least it looks simpler.
im fine with that (not crucial tho)
---
orm.util.AliasedClauses._create_row_adapter()
class AliasedRowAdapter( object):
1. can't this be made as standalone class, returning an instance,
initialized with the map, which is then __call__()ed ?
is it faster to say self.map or to say map from locals() ? its
probably not very crucial either way.
2. this can be faster if:
a) has_key = __contains__ #instead of yet another funccall
b) __getitem__ uses try except instead of double lookup key in map
im not sure try/except is faster here - im pretty sure a missing key
is likely and exception throws are very expensive. would be worth a
try to see if the missing key exception actually occurs here.
---
orm.mapper
Mapper._instance():
WTF is the **{'key':value, ... } ?
eg. if extension.populate_instance(self, context, row, instance,
**{'instancekey':identitykey, 'isnew':isnew}) ...
same thing is done as separate variable a page later;
btw there are several of these **{} in the file
i think thats a product of a refactoring where the ** was originally
not there.
also, Mapper._options is redundant (leftover?) neverused
yeah gone along with the **{} in r3303
---
orm.attribute
AttributeManager.init_attr():
the saving this one eventualy does is too small, compared to a
property call of ._state.
i havent benched this in a while but my recollection is that the
AttributeError raise is *much* slower than pre-calling this method.
a single function call is always faster than an exception throw.
however, i see that the exception throw is being suppressed also with
a hasattr() being called every timeim not sure why thats that way
now so i might change it back to throwing AttributeError.
AttributeManager.register_attribute():
the def _get_state(self) that is made into as property _state can
be made eventualy faster with try-except instead of 'if'.
exception throws are slower. but like above says, the throw here can
be prevented if init_attr() is called.
btw: cant that ._state property be removed alltogether (i.e. made a
plain attribute? then init_attr() MUST be there seting it up as plain
dict.
it should be named something non-collisionworthy such as the current
_sa_attr_state.
---
orm/unitofwork
UOWTask._sort_circular_dependencies():
def get_dependency_task(obj, depprocessor):
try:
dp = dependencies[obj]
except KeyError:
dp = dependencies.setdefault(obj, {})
isnt just the setdefault() enough?
this should just say dependencies[obj] = dp = {}
---
engine.url.
def translate_connect_args(self, names):
this assumes the order of passed names matches the order of
attribute_names inside... very fragile. Why not use set of kwargs
like (attr_name=replacement_name defaulting to None), then just use
the non empty ones?
patches welcome
def _parse_rfc1738_args():
the 'opts' dict is redundant, would be cleaner if args are just
passed to URL( name=value)
this is an old patch we got from someone, improvement patches welcome
---
topological.py
QueueDependencySorter.sort():
'cycles' is redundant neverused variable
would look good in the patch too
---
util:
ThreadLocal:
- wouldnt be faster if the key in the _tdict is tuple(id,key) and
not some formatted string off these? or the key is nonhashable?
good catch, should be patched
- the engine/threadlocal.TLEngine._session() issues a hasattr() on
such object. how does it actualy work? IMO it always fails
patch which includes a test case to add into test/engine/
transaction.py would be welcome
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: SQLAlchemy 0.4 beta2 released
On Aug 14, 2007, at 2:12 PM, Brian Beck wrote: 0.4 looks awesome. Some names jump out at me though: * sessionmaker - Strange that it's a noun, and not a verb (like make_session) the verb would be, make_session_maker. or make_session_class, except in some cases it might not be an actual class... * bind/binds arguments - Strange that it's a verb, and not a noun (why not engine/engines?) bind is used as a noun here. it is slightly weird and there was much discussion about this so youre a little late for that train :). the reason its not engine anymore is because it can be a Connection also. engines and connections are collectively called connectables, but nobody understood that term...so its bind. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: SQLAlchemy 0.4 beta2 released
orm.util.AliasedClauses._create_row_adapter() class AliasedRowAdapter( object): 1. can't this be made as standalone class, returning an instance, initialized with the map, which is then __call__()ed ? is it faster to say self.map or to say map from locals() ? its probably not very crucial either way. well.. u save on class' creation :-), get a class that could be used elsewhere, and eventualy more obvious code - runtime outer-namespace bindings in python are veeery strange beasts sometimes, with funny side effects. 2. this can be faster if: a) has_key = __contains__ #instead of yet another funccall b) __getitem__ uses try except instead of double lookup key in map im not sure try/except is faster here - im pretty sure a missing key is likely and exception throws are very expensive. would be worth a try to see if the missing key exception actually occurs here. i'll have to check, last 3 years i've being doing everything assuming that a (lookup:exception) is faster than (lookup-if-lookup:lookup-else) in both attribute and dict lookups. Plus that a function call is the most expensive python thing ever... apart of repeated a.x a.x a.x attribute access. h time to measure the myths. --- orm.attribute AttributeManager.init_attr(): the saving this one eventualy does is too small, compared to a property call of ._state. i havent benched this in a while but my recollection is that the AttributeError raise is *much* slower than pre-calling this method. a single function call is always faster than an exception throw. however, i see that the exception throw is being suppressed also with a hasattr() being called every timeim not sure why thats that way now so i might change it back to throwing AttributeError. btw: cant that ._state property be removed alltogether (i.e. made a plain attribute? then init_attr() MUST be there seting it up as plain dict. it should be named something non-collisionworthy such as the current _sa_attr_state. the only gain from the property is that it is readonly, i.e. obj._state = None is not allowed (as well as del'ete). But i can do obj._sa_attr_state = None (and all goes to hell). If that property disappears alltogether, we are left with a plain _sa_attr_state attribute, which can be now set to None and deleted... which is more or less same as before less the complications with the property. Fair trade, no? --- util: ThreadLocal: - wouldnt be faster if the key in the _tdict is tuple(id,key) and not some formatted string off these? or the key is nonhashable? good catch, should be patched - the engine/threadlocal.TLEngine._session() issues a hasattr() on such object. how does it actualy work? IMO it always fails patch which includes a test case to add into test/engine/ transaction.py would be welcome i've never used these so no much idea what to do here - was just passing code along my eyeline. will prepare some patches for the other things. ciao svilen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: SQLAlchemy 0.4 beta2 released
On 8/14/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: im not sure try/except is faster here - im pretty sure a missing key is likely and exception throws are very expensive. would be worth a try to see if the missing key exception actually occurs here. i'll have to check, last 3 years i've being doing everything assuming that a (lookup:exception) is faster than (lookup-if-lookup:lookup-else) in both attribute and dict lookups. Only if, like Mike said, the common case is for the key to be present. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: SQLAlchemy 0.4 beta2 released
On Aug 14, 2007, at 4:35 PM, Michael Bayer wrote: On Aug 14, 2007, at 12:38 PM, svilen wrote: --- orm.attribute AttributeManager.init_attr(): the saving this one eventualy does is too small, compared to a property call of ._state. i havent benched this in a while but my recollection is that the AttributeError raise is *much* slower than pre-calling this method. a single function call is always faster than an exception throw. however, i see that the exception throw is being suppressed also with a hasattr() being called every timeim not sure why thats that way now so i might change it back to throwing AttributeError. the results are in, running test/prof/masseagerload.py. this test is very heavy on creating new instances from mapper rows, which is where the initialization of _state comes in. no init_attr(), detect missing with AttributeError Profiled target 'masseagerload', wall time: 0.59 seconds Profile report for target 'masseagerload' (masseagerload.prof) 57039 function calls (55962 primitive calls) in 0.489 CPU seconds init_attr(), detect missing with AttributeError Profiled target 'masseagerload', wall time: 0.53 seconds 57549 function calls (56472 primitive calls) in 0.426 CPU seconds init_attr(), detect missing with hasattr Profiled target 'masseagerload', wall time: 0.56 seconds 57549 function calls (56472 primitive calls) in 0.431 CPU seconds no init_attr(), detect missing with hasattr Profiled target 'masseagerload', wall time: 0.49 seconds 57039 function calls (55962 primitive calls) in 0.390 CPU seconds im not exactly sure why the hasattr() call, being present and then removed, doesnt change the number of function calls. anyway, the times vary a little bit but the hasattr call, even though its called many more times than the AttributeError gets raised, is slightly faster than raising AttributeError. so no AttributeError, and I like getting rid of init_attr() very much so its out in r3313. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
