Re: [sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
Hi, I know this is an OLD threat but I was searching the group to see If I was not the first one doing this. I am not sure I understand very well what this threat is all about, but I want to extend the Column class for a different reason. I want to add extra functionality to the Column class which is absolutely NOT SA related. SA functionality should not be effected though. say I want to add a config value and some methods for rendering and validating screens: def MyColumn(Column): def __init(): dosomething to init def ExtraInfo(self): do_something_not_sa_related validation = 'someregex' and use MyColumn in places where I normally use Column(..) What do I need to take into account, I've done some tests and Error hell broke loose, where the errors are hidden deep inside SA so hard to overcome. Martijn On Dec 11, 2008, at 16:20 , Michael Bayer wrote: On Dec 11, 2008, at 3:37 AM, Angri wrote: Here it is: http://www.sqlalchemy.org/trac/ticket/1244 Maybe it is good idea to drop some new lines in faq? Something like this: Q: How should I extend sqlalchemy.schema.Column? A: You surely dont need it. Recommended way to achive your possible needs is to write instance-factory function which decorates creation of sqlalchemy.schema.Column instances. Q: But I'm really need it! A: Ok. To subclass Column, this is the current recipe: from sqlalchemy.sql.util import Annotated, annotated_classes class MyColumn(Column): ... class AnnotatedMyColumn(Annotated, MyColumn): pass annotated_classes[MyColumn] = AnnotatedMyColumn Do not forget to put AnnotatedMyColumn in the module namespace, or your schema will not be pickleable! Correct me please if I am wrong somewhere and excuse me for my English. Well the AnnotatedMyColumn part is less than ideal since its an internal. the way that works could very likely change. Creating an AnnotatedXXX class *can* be automated. the pickle thing just might be a caveat we'd document or arrange for an exception to occur (like putting a throw in a __getstate__ method). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~--- -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
Well, it sounds like you're taking the wrong approach to me. I'd subclass
your ORM objects. Add some simple hooks so that you can use the built in
dictionary mixin. Then override update method to apply the validators.
Something like this.
class ValidatorAspect:
validators = {}
@classmethod
def add_validator():
@classmethod
def del_validator():
from UserDict import DictMixin
class MyDataObject(Base,ValidatorAspect,DictMixin):
def __getitem__():
def __setitem__():
def __delitem__():
def keys():
def update():
#Loop over inputs
#Apply validator if present
My $.02
Sean
On Mon, Feb 28, 2011 at 10:17 AM, Martijn Moeling mart...@xs4us.nu wrote:
Hi,
I know this is an OLD threat but I was searching the group to see If I
was not the first one doing this.
I am not sure I understand very well what this threat is all about, but I
want to extend the Column class for a different reason.
I want to add extra functionality to the Column class which is absolutely
NOT SA related. SA functionality should not be effected though.
say I want to add a config value and some methods for rendering and
validating screens:
def MyColumn(Column):
def __init():
dosomething to init
def ExtraInfo(self):
do_something_not_sa_related
validation = 'someregex'
and use MyColumn in places where I normally use Column(..)
What do I need to take into account, I've done some tests and Error hell
broke loose, where the errors are hidden deep inside SA so hard to overcome.
Martijn
On Dec 11, 2008, at 16:20 , Michael Bayer wrote:
On Dec 11, 2008, at 3:37 AM, Angri wrote:
Here it is: http://www.sqlalchemy.org/trac/ticket/1244
Maybe it is good idea to drop some new lines in faq? Something like
this:
Q: How should I extend sqlalchemy.schema.Column?
A: You surely dont need it. Recommended way to achive your possible
needs is to write instance-factory function which decorates creation
of sqlalchemy.schema.Column instances.
Q: But I'm really need it!
A: Ok. To subclass Column, this is the current recipe:
from sqlalchemy.sql.util import Annotated, annotated_classes
class MyColumn(Column):
...
class AnnotatedMyColumn(Annotated, MyColumn):
pass
annotated_classes[MyColumn] = AnnotatedMyColumn
Do not forget to put AnnotatedMyColumn in the module namespace, or
your schema will not be pickleable!
Correct me please if I am wrong somewhere and excuse me for my
English.
Well the AnnotatedMyColumn part is less than ideal since its an
internal. the way that works could very likely change. Creating an
AnnotatedXXX class *can* be automated. the pickle thing just might be
a caveat we'd document or arrange for an exception to occur (like
putting a throw in a __getstate__ method).
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
Column can be subclassed but because they are intensively used in complex expression transformations, your custom class may be used in more scenarios than you first anticipate. There are two scenarios where Column objects are copied, and in one case copied into an altered class, so the copying of Column uses an attribute called _constructor to point to which class should be used when creating this copy. Usually setting that to Column: class MyColumn(Column): _constructor = Column # go nuts is all you need. On Feb 28, 2011, at 10:17 AM, Martijn Moeling wrote: Hi, I know this is an OLD threat but I was searching the group to see If I was not the first one doing this. I am not sure I understand very well what this threat is all about, but I want to extend the Column class for a different reason. I want to add extra functionality to the Column class which is absolutely NOT SA related. SA functionality should not be effected though. say I want to add a config value and some methods for rendering and validating screens: def MyColumn(Column): def __init(): dosomething to init def ExtraInfo(self): do_something_not_sa_related validation = 'someregex' and use MyColumn in places where I normally use Column(..) What do I need to take into account, I've done some tests and Error hell broke loose, where the errors are hidden deep inside SA so hard to overcome. Martijn On Dec 11, 2008, at 16:20 , Michael Bayer wrote: On Dec 11, 2008, at 3:37 AM, Angri wrote: Here it is: http://www.sqlalchemy.org/trac/ticket/1244 Maybe it is good idea to drop some new lines in faq? Something like this: Q: How should I extend sqlalchemy.schema.Column? A: You surely dont need it. Recommended way to achive your possible needs is to write instance-factory function which decorates creation of sqlalchemy.schema.Column instances. Q: But I'm really need it! A: Ok. To subclass Column, this is the current recipe: from sqlalchemy.sql.util import Annotated, annotated_classes class MyColumn(Column): ... class AnnotatedMyColumn(Annotated, MyColumn): pass annotated_classes[MyColumn] = AnnotatedMyColumn Do not forget to put AnnotatedMyColumn in the module namespace, or your schema will not be pickleable! Correct me please if I am wrong somewhere and excuse me for my English. Well the AnnotatedMyColumn part is less than ideal since its an internal. the way that works could very likely change. Creating an AnnotatedXXX class *can* be automated. the pickle thing just might be a caveat we'd document or arrange for an exception to occur (like putting a throw in a __getstate__ method). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~--- -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
On Feb 28, 2011, at 18:21 , Michael Bayer wrote: Column can be subclassed but because they are intensively used in complex expression transformations, your custom class may be used in more scenarios than you first anticipate. There are two scenarios where Column objects are copied, and in one case copied into an altered class, so the copying of Column uses an attribute called _constructor to point to which class should be used when creating this copy. Usually setting that to Column: class MyColumn(Column): _constructor = Column # go nuts LOL!! Thanks is all you need. On Feb 28, 2011, at 10:17 AM, Martijn Moeling wrote: Hi, I know this is an OLD threat but I was searching the group to see If I was not the first one doing this. I am not sure I understand very well what this threat is all about, but I want to extend the Column class for a different reason. I want to add extra functionality to the Column class which is absolutely NOT SA related. SA functionality should not be effected though. say I want to add a config value and some methods for rendering and validating screens: def MyColumn(Column): def __init(): dosomething to init def ExtraInfo(self): do_something_not_sa_related validation = 'someregex' and use MyColumn in places where I normally use Column(..) What do I need to take into account, I've done some tests and Error hell broke loose, where the errors are hidden deep inside SA so hard to overcome. Martijn On Dec 11, 2008, at 16:20 , Michael Bayer wrote: On Dec 11, 2008, at 3:37 AM, Angri wrote: Here it is: http://www.sqlalchemy.org/trac/ticket/1244 Maybe it is good idea to drop some new lines in faq? Something like this: Q: How should I extend sqlalchemy.schema.Column? A: You surely dont need it. Recommended way to achive your possible needs is to write instance-factory function which decorates creation of sqlalchemy.schema.Column instances. Q: But I'm really need it! A: Ok. To subclass Column, this is the current recipe: from sqlalchemy.sql.util import Annotated, annotated_classes class MyColumn(Column): ... class AnnotatedMyColumn(Annotated, MyColumn): pass annotated_classes[MyColumn] = AnnotatedMyColumn Do not forget to put AnnotatedMyColumn in the module namespace, or your schema will not be pickleable! Correct me please if I am wrong somewhere and excuse me for my English. Well the AnnotatedMyColumn part is less than ideal since its an internal. the way that works could very likely change. Creating an AnnotatedXXX class *can* be automated. the pickle thing just might be a caveat we'd document or arrange for an exception to occur (like putting a throw in a __getstate__ method). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~--- -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
Here it is: http://www.sqlalchemy.org/trac/ticket/1244 Maybe it is good idea to drop some new lines in faq? Something like this: Q: How should I extend sqlalchemy.schema.Column? A: You surely dont need it. Recommended way to achive your possible needs is to write instance-factory function which decorates creation of sqlalchemy.schema.Column instances. Q: But I'm really need it! A: Ok. To subclass Column, this is the current recipe: from sqlalchemy.sql.util import Annotated, annotated_classes class MyColumn(Column): ... class AnnotatedMyColumn(Annotated, MyColumn): pass annotated_classes[MyColumn] = AnnotatedMyColumn Do not forget to put AnnotatedMyColumn in the module namespace, or your schema will not be pickleable! Correct me please if I am wrong somewhere and excuse me for my English. On 11 дек, 01:23, Michael Bayer [EMAIL PROTECTED] wrote: hey send it as an email attachment, or create a ticket in trac as guest/guest and attach it there: http://www.sqlalchemy.org/trac/newticket On Dec 10, 2008, at 5:19 PM, Angri wrote: if you'd like to submit a patch which defines __visit_name__ for all ClauseElements and removes the logic from VisitableType to guess the name, it will be accepted. The second half of VisitableType still may be needed since it improves performance. Ok, I did it. Can not find where I can attach file to message in google groups, so I put it in paste bin:http://paste.org/index.php?id=4463 I made some automated tests to make sure that nothing will break. Existing Visitable's descendants after applying the patch will have exactly the same value of __visit_name__ property. I also put small test from first message in the topic. It fails with vanila trunk and pass ok with patched. Take a look, please. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
if you'd like to submit a patch which defines __visit_name__ for all ClauseElements and removes the logic from VisitableType to guess the name, it will be accepted. The second half of VisitableType still may be needed since it improves performance. Ok, I did it. Can not find where I can attach file to message in google groups, so I put it in paste bin: http://paste.org/index.php?id=4463 I made some automated tests to make sure that nothing will break. Existing Visitable's descendants after applying the patch will have exactly the same value of __visit_name__ property. I also put small test from first message in the topic. It fails with vanila trunk and pass ok with patched. Take a look, please. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
hey send it as an email attachment, or create a ticket in trac as guest/guest and attach it there: http://www.sqlalchemy.org/trac/newticket On Dec 10, 2008, at 5:19 PM, Angri wrote: if you'd like to submit a patch which defines __visit_name__ for all ClauseElements and removes the logic from VisitableType to guess the name, it will be accepted. The second half of VisitableType still may be needed since it improves performance. Ok, I did it. Can not find where I can attach file to message in google groups, so I put it in paste bin: http://paste.org/index.php?id=4463 I made some automated tests to make sure that nothing will break. Existing Visitable's descendants after applying the patch will have exactly the same value of __visit_name__ property. I also put small test from first message in the topic. It fails with vanila trunk and pass ok with patched. Take a look, please. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
I can not agree that extending is safe as I've encountered another problem with custom class name: http://www.sqlalchemy.org/trac/browser/sqlalchemy/trunk/lib/sqlalchemy/sql/util.py#L145 And I guess it is not the last :( I see, you propose not to extend class Column and write a function that creates instances but it seems to me that this approach is not good because it is not pythonic, it is not object-orientedly, it is hard to write, support and extend. Continuing experiments I created the following (ugly) code. Maybe it will help someone with similar needs. class CustomColumnMetaclass(sqlalchemy.Column.__metaclass__): def __init__(cls, clsname, bases, dct): cls.__name__ = sqlalchemy.Column.__name__ cls.__visit_name__ = sqlalchemy.Column.__visit_name__ super(CustomColumnMetaclass, cls).__init__(clsname, bases, dct) class CustomColumn(sqlalchemy.Column): __metaclass__ = CustomColumnMetaclass Btw, I still think that relying on class name is a bad way to do things. What do you think Michael, how difficult it can be to rewrite those pieces of code to use more OOP-like technics, particularly explicitly define class properties (inheritable class properties!) instead of doing things like eval(Annotated%s % element.__class__.__name__)? On Dec 7, 1:33 am, Michael Bayer [EMAIL PROTECTED] wrote: On Dec 6, 2008, at 4:27 PM, Angri wrote: 1. What about another side-effects depending on clsname? Is it actually safe to extend sqlalchemy.schema.Column, or it may have unpredictable behavior similar to that i've encountered? The Column object is one of the most key classes in all of SQLAlchemy and we do put it through some fairly intricate copy/proxy patterns particularly when using the ORM. Extending it should be safe, although this is usually not needed. For custom creation patterns as you're seeking here its more straightforward to build a creation function, so that the resulting object is returned unchanged in its type. 2. (almost offtopic) Is 'exec' really need there? What's wrong with closures? the visit step is called very intensively during statement compilation so exec'ing the direct code instead of relying upon getattr() with a composed name at runtime is an optimization to reduce function-call and attribute-retrieval overhead. Just as a point of reference I tried rewriting our visit dispatcher in C, and my experiments showed that using the exec approach you see there performs just as well - though the time savings compared to a basic getattr() approach are very small. Since you raised the issue, I went to try a different approach which is probably the best possible approach without using exec, which is this: visit_name = cls.__dict__[__visit_name__] if isinstance(visit_name, str): getter = operator.attrgetter(visit_%s % visit_name) def _compiler_dispatch(self, visitor, **kw): return getter(visitor)(self, **kw) else: def _compiler_dispatch(self, visitor, **kw): return getattr(visitor, 'visit_%s' % self.__visit_name__)(self, **kw) Above, we use operator.attrgetter so that the string composition is already handled, and the attrgetter itself is a native object which performs as fast as direct access. This change still adds a few function calls per compile. Our bench of a single select() compile goes from 183 to 187, and two of the zoomark_orm tests fail past the 5% threshhold, with tests three and four moving from 6623 to 6723 and 23345 to 23861 function calls, respectively. Which is an extremely small amount, so its not a terribly big deal either way. So the exec approach is saving a tiny amount of call counts, but not necessarily any actual time. I'd be willing to scrap it if it continues to scare other developers. The reason I'm at all comfortable with exec is that we're already using 'exec' for decorators - its a technique used by the decorators module (which I'd like to transition to at some point) to faithfully represent the calling signature of a decorated function. 3. Maybe I should send it to developers mailing list? eitherdevel is not very active. though this is a pretty develop-y subject... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
On Dec 9, 2008, at 4:09 PM, Angri wrote: I can not agree that extending is safe as I've encountered another problem with custom class name: http://www.sqlalchemy.org/trac/browser/sqlalchemy/trunk/lib/sqlalchemy/sql/util.py#L145 And I guess it is not the last :( It probably is the last. That's something which has been added recently. It previously worked in a dynamic fashion but was changed to what you see there for pickling purposes. However it can be expanded to support classes which aren't present, so that you wouldn't notice it. This technique is also something I picked up from a very well known and respected Python developer. I see, you propose not to extend class Column and write a function that creates instances but it seems to me that this approach is not good because it is not pythonic, it is not object-orientedly, it is hard to write, support and extend. It's how most of SQLA clause elements are invoked, in fact. Python's philosophy is defnitely not in the vein of use objects and inheritance for everything, you're thinking of Java, which does not provide standalone functions in the language. Using functions to return objects provides a convenient layer of variability between the composed structure of what the function returns and the public signature of that function. Subclassing is definitely not the only way to extend something and the problem you're experiencing is a variant of the so-called fragile base problem.What strikes me as most unpythonic here is that you think you need a rewrite of the library in a Java-like GOF style when a simple, one line function will solve your problem. Btw, I still think that relying on class name is a bad way to do things. Here is where the technique is derived from, from the visitor.py module in Python's own compiler module: http://svn.python.org/view/python/branches/release26-maint/Lib/compiler/visitor.py?rev=66717view=auto What do you think Michael, how difficult it can be to rewrite those pieces of code to use more OOP-like technics, particularly explicitly define class properties (inheritable class properties!) instead of doing things like eval(Annotated%s % element.__class__.__name__)? When I first started Python, I used a traditional GOF-visitor pattern, with explicit visit_XXX and dispatcher methods. It's extremely verbose and tedious to refactor. It didn't take too long for me to realize I was porting Java methodologies that are entirely inappropriate for a dynamic language like Python. Subclassing Column in any case requires knowledge of the visitation logic - you either need to implement visit_XXX() in the GOF style, or __visit_name__ = 'whatever' in the Pythonic style. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
On Dec 9, 2008, at 4:09 PM, Angri wrote: I can not agree that extending is safe as I've encountered another problem with custom class name: http://www.sqlalchemy.org/trac/browser/sqlalchemy/trunk/lib/sqlalchemy/sql/util.py#L145 rev 5454 removes AnnotatedColumn's reliance upon names within the expression package. To subclass Column, this is the current recipe: from sqlalchemy.sql.util import Annotated, annotated_classes class MyColumn(Column): __visit_name__ = column class AnnotatedMyColumn(Annotated, MyColumn): __visit_name__ = column annotated_classes[MyColumn] = AnnotatedMyColumn There are of course metaclass methods of making this automatic. But here we have pure traditional OOP style - make all the classes explicitly, register them, etc. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
On Dec 9, 2008, at 4:09 PM, Angri wrote:
particularly
explicitly define class properties (inheritable class properties!)
if you'd like to submit a patch which defines __visit_name__ for all
ClauseElements and removes the logic from VisitableType to guess the
name, it will be accepted. The second half of VisitableType still may
be needed since it improves performance.
instead of doing things like eval(Annotated%s %
element.__class__.__name__)?
The __name__ based logic is gone.
The code that used to be there, which is the code i was mentioning,
was this:
return object.__new__(
type.__new__(type, Annotated%s %
element.__class__.__name__, (Annotated, element.__class__), {})
)
This generates a new type dynamically and is the preferred way to do
this. The __name__ is only used there to give the resultling class a
name and not any kind of lookup semantics.
Unfortunately the resulting class is not pickleable since it is not
declared in the module. So there are two options I'm aware of.
Either do this:
class AnnotatedClauseElement(Annotated, ClauseElement):
class AnnotatedFromClause(Annotated, FromClause):
...
class AnnotatedColumnElement(Annotated, ColumnElement):
...
continue for 100 more classes in sqlalchemy.expression
Or do the Exec thing we have. A third alternative would be great if
you have one in mind.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
On Wednesday 10 December 2008 00:42:14 Michael Bayer wrote:
On Dec 9, 2008, at 4:09 PM, Angri wrote:
particularly
explicitly define class properties (inheritable class
properties!)
if you'd like to submit a patch which defines __visit_name__ for
all ClauseElements and removes the logic from VisitableType to
guess the name, it will be accepted. The second half of
VisitableType still may be needed since it improves performance.
instead of doing things like eval(Annotated%s %
element.__class__.__name__)?
The __name__ based logic is gone.
The code that used to be there, which is the code i was mentioning,
was this:
return object.__new__(
type.__new__(type, Annotated%s %
element.__class__.__name__, (Annotated, element.__class__), {})
)
This generates a new type dynamically and is the preferred way to
do this. The __name__ is only used there to give the resultling
class a name and not any kind of lookup semantics.
Unfortunately the resulting class is not pickleable since it is not
declared in the module. So there are two options I'm aware of.
Either do this:
class AnnotatedClauseElement(Annotated, ClauseElement):
class AnnotatedFromClause(Annotated, FromClause):
...
class AnnotatedColumnElement(Annotated, ColumnElement):
...
continue for 100 more classes in sqlalchemy.expression
Or do the Exec thing we have. A third alternative would be great
if you have one in mind.
one can inject things in module's locals() via locals().update( ...)
but i'm not sure if this is a feature-to-stay.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
On Dec 6, 2008, at 4:27 PM, Angri wrote: 1. What about another side-effects depending on clsname? Is it actually safe to extend sqlalchemy.schema.Column, or it may have unpredictable behavior similar to that i've encountered? The Column object is one of the most key classes in all of SQLAlchemy and we do put it through some fairly intricate copy/proxy patterns particularly when using the ORM. Extending it should be safe, although this is usually not needed. For custom creation patterns as you're seeking here its more straightforward to build a creation function, so that the resulting object is returned unchanged in its type. 2. (almost offtopic) Is 'exec' really need there? What's wrong with closures? the visit step is called very intensively during statement compilation so exec'ing the direct code instead of relying upon getattr() with a composed name at runtime is an optimization to reduce function-call and attribute-retrieval overhead. Just as a point of reference I tried rewriting our visit dispatcher in C, and my experiments showed that using the exec approach you see there performs just as well - though the time savings compared to a basic getattr() approach are very small. Since you raised the issue, I went to try a different approach which is probably the best possible approach without using exec, which is this: visit_name = cls.__dict__[__visit_name__] if isinstance(visit_name, str): getter = operator.attrgetter(visit_%s % visit_name) def _compiler_dispatch(self, visitor, **kw): return getter(visitor)(self, **kw) else: def _compiler_dispatch(self, visitor, **kw): return getattr(visitor, 'visit_%s' % self.__visit_name__)(self, **kw) Above, we use operator.attrgetter so that the string composition is already handled, and the attrgetter itself is a native object which performs as fast as direct access. This change still adds a few function calls per compile. Our bench of a single select() compile goes from 183 to 187, and two of the zoomark_orm tests fail past the 5% threshhold, with tests three and four moving from 6623 to 6723 and 23345 to 23861 function calls, respectively. Which is an extremely small amount, so its not a terribly big deal either way. So the exec approach is saving a tiny amount of call counts, but not necessarily any actual time. I'd be willing to scrap it if it continues to scare other developers. The reason I'm at all comfortable with exec is that we're already using 'exec' for decorators - its a technique used by the decorators module (which I'd like to transition to at some point) to faithfully represent the calling signature of a decorated function. 3. Maybe I should send it to developers mailing list? eitherdevel is not very active. though this is a pretty develop-y subject... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
