Re: [sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
On Feb 28, 2011, at 18:21 , Michael Bayer wrote: > Column can be subclassed but because they are intensively used in complex > expression transformations, your custom class may be used in more scenarios > than you first anticipate. > > There are two scenarios where Column objects are copied, and in one case > copied into an altered class, so the "copying" of Column uses an attribute > called _constructor to point to which class should be used when creating this > copy. Usually setting that to Column: > > class MyColumn(Column): >_constructor = Column > > # go nuts LOL!! Thanks > > is all you need. > > > > On Feb 28, 2011, at 10:17 AM, Martijn Moeling wrote: > >> Hi, >> >> I know this is an "OLD" threat but I was searching the group to see If I was >> not the first one doing this. >> >> I am not sure I understand very well what this threat is all about, but I >> want to extend the Column class for a different reason. >> >> I want to add extra functionality to the Column class which is absolutely >> NOT SA related. SA functionality should not be effected though. >> >> say I want to add a config value and some methods for rendering and >> validating screens: >> >> def MyColumn(Column): >> >> def __init(): >> dosomething to init >> >> def ExtraInfo(self): >> do_something_not_sa_related >> >> validation = 'someregex' >> >> >> and use MyColumn in places where I normally use Column(..) >> >> What do I need to take into account, I've done some tests and "Error hell" >> broke loose, where the errors are hidden deep inside SA so hard to overcome. >> >> Martijn >> >> On Dec 11, 2008, at 16:20 , Michael Bayer wrote: >> >>> >>> >>> On Dec 11, 2008, at 3:37 AM, Angri wrote: >>> Here it is: http://www.sqlalchemy.org/trac/ticket/1244 Maybe it is good idea to drop some new lines in faq? Something like this: Q: How should I extend sqlalchemy.schema.Column? A: You surely dont need it. Recommended way to achive your possible needs is to write instance-factory function which decorates creation of sqlalchemy.schema.Column instances. Q: But I'm really need it! A: Ok. To subclass Column, this is the current recipe: from sqlalchemy.sql.util import Annotated, annotated_classes class MyColumn(Column): ... class AnnotatedMyColumn(Annotated, MyColumn): pass annotated_classes[MyColumn] = AnnotatedMyColumn Do not forget to put AnnotatedMyColumn in the module namespace, or your schema will not be pickleable! Correct me please if I am wrong somewhere and excuse me for my English. >>> >>> Well the AnnotatedMyColumn part is less than ideal since its an >>> internal. the way that works could very likely change. Creating an >>> AnnotatedXXX class *can* be automated. the pickle thing just might be >>> a caveat we'd document or arrange for an exception to occur (like >>> putting a throw in a __getstate__ method). >>> >>> --~--~-~--~~~---~--~~ >>> You received this message because you are subscribed to the Google Groups >>> "sqlalchemy" group. >>> To post to this group, send email to sqlalchemy@googlegroups.com >>> To unsubscribe from this group, send email to >>> sqlalchemy+unsubscr...@googlegroups.com >>> For more options, visit this group at >>> http://groups.google.com/group/sqlalchemy?hl=en >>> -~--~~~~--~~--~--~--- >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "sqlalchemy" group. >> To post to this group, send email to sqlalchemy@googlegroups.com. >> To unsubscribe from this group, send email to >> sqlalchemy+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/sqlalchemy?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to sqlalchemy@googlegroups.com. > To unsubscribe from this group, send email to > sqlalchemy+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
Column can be subclassed but because they are intensively used in complex expression transformations, your custom class may be used in more scenarios than you first anticipate. There are two scenarios where Column objects are copied, and in one case copied into an altered class, so the "copying" of Column uses an attribute called _constructor to point to which class should be used when creating this copy. Usually setting that to Column: class MyColumn(Column): _constructor = Column # go nuts is all you need. On Feb 28, 2011, at 10:17 AM, Martijn Moeling wrote: > Hi, > > I know this is an "OLD" threat but I was searching the group to see If I was > not the first one doing this. > > I am not sure I understand very well what this threat is all about, but I > want to extend the Column class for a different reason. > > I want to add extra functionality to the Column class which is absolutely NOT > SA related. SA functionality should not be effected though. > > say I want to add a config value and some methods for rendering and > validating screens: > > def MyColumn(Column): > > def __init(): > dosomething to init > > def ExtraInfo(self): > do_something_not_sa_related > > validation = 'someregex' > > > and use MyColumn in places where I normally use Column(..) > > What do I need to take into account, I've done some tests and "Error hell" > broke loose, where the errors are hidden deep inside SA so hard to overcome. > > Martijn > > On Dec 11, 2008, at 16:20 , Michael Bayer wrote: > >> >> >> On Dec 11, 2008, at 3:37 AM, Angri wrote: >> >>> >>> Here it is: http://www.sqlalchemy.org/trac/ticket/1244 >>> >>> Maybe it is good idea to drop some new lines in faq? Something like >>> this: >>> >>> Q: How should I extend sqlalchemy.schema.Column? >>> A: You surely dont need it. Recommended way to achive your possible >>> needs is to write instance-factory function which decorates creation >>> of sqlalchemy.schema.Column instances. >>> >>> Q: But I'm really need it! >>> A: Ok. To subclass Column, this is the current recipe: >>> >>> from sqlalchemy.sql.util import Annotated, annotated_classes >>> >>> class MyColumn(Column): >>> ... >>> >>> class AnnotatedMyColumn(Annotated, MyColumn): >>> pass >>> >>> annotated_classes[MyColumn] = AnnotatedMyColumn >>> >>> Do not forget to put AnnotatedMyColumn in the module namespace, or >>> your schema will not be pickleable! >>> >>> Correct me please if I am wrong somewhere and excuse me for my >>> English. >> >> Well the AnnotatedMyColumn part is less than ideal since its an >> internal. the way that works could very likely change. Creating an >> AnnotatedXXX class *can* be automated. the pickle thing just might be >> a caveat we'd document or arrange for an exception to occur (like >> putting a throw in a __getstate__ method). >> >> --~--~-~--~~~---~--~~ >> You received this message because you are subscribed to the Google Groups >> "sqlalchemy" group. >> To post to this group, send email to sqlalchemy@googlegroups.com >> To unsubscribe from this group, send email to >> sqlalchemy+unsubscr...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/sqlalchemy?hl=en >> -~--~~~~--~~--~--~--- >> > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to sqlalchemy@googlegroups.com. > To unsubscribe from this group, send email to > sqlalchemy+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
Well, it sounds like you're taking the wrong approach to me. I'd subclass
your ORM objects. Add some simple hooks so that you can use the built in
dictionary mixin. Then override update method to apply the validators.
Something like this.
class ValidatorAspect:
validators = {}
@classmethod
def add_validator():
@classmethod
def del_validator():
from UserDict import DictMixin
class MyDataObject(Base,ValidatorAspect,DictMixin):
def __getitem__():
def __setitem__():
def __delitem__():
def keys():
def update():
#Loop over inputs
#Apply validator if present
My $.02
Sean
On Mon, Feb 28, 2011 at 10:17 AM, Martijn Moeling wrote:
> Hi,
>
> I know this is an "OLD" threat but I was searching the group to see If I
> was not the first one doing this.
>
> I am not sure I understand very well what this threat is all about, but I
> want to extend the Column class for a different reason.
>
> I want to add extra functionality to the Column class which is absolutely
> NOT SA related. SA functionality should not be effected though.
>
> say I want to add a config value and some methods for rendering and
> validating screens:
>
> def MyColumn(Column):
>
>def __init():
>dosomething to init
>
>def ExtraInfo(self):
>do_something_not_sa_related
>
>validation = 'someregex'
>
>
> and use MyColumn in places where I normally use Column(..)
>
> What do I need to take into account, I've done some tests and "Error hell"
> broke loose, where the errors are hidden deep inside SA so hard to overcome.
>
> Martijn
>
> On Dec 11, 2008, at 16:20 , Michael Bayer wrote:
>
> >
> >
> > On Dec 11, 2008, at 3:37 AM, Angri wrote:
> >
> >>
> >> Here it is: http://www.sqlalchemy.org/trac/ticket/1244
> >>
> >> Maybe it is good idea to drop some new lines in faq? Something like
> >> this:
> >>
> >> Q: How should I extend sqlalchemy.schema.Column?
> >> A: You surely dont need it. Recommended way to achive your possible
> >> needs is to write instance-factory function which decorates creation
> >> of sqlalchemy.schema.Column instances.
> >>
> >> Q: But I'm really need it!
> >> A: Ok. To subclass Column, this is the current recipe:
> >>
> >> from sqlalchemy.sql.util import Annotated, annotated_classes
> >>
> >> class MyColumn(Column):
> >> ...
> >>
> >> class AnnotatedMyColumn(Annotated, MyColumn):
> >> pass
> >>
> >> annotated_classes[MyColumn] = AnnotatedMyColumn
> >>
> >> Do not forget to put AnnotatedMyColumn in the module namespace, or
> >> your schema will not be pickleable!
> >>
> >> Correct me please if I am wrong somewhere and excuse me for my
> >> English.
> >
> > Well the AnnotatedMyColumn part is less than ideal since its an
> > internal. the way that works could very likely change. Creating an
> > AnnotatedXXX class *can* be automated. the pickle thing just might be
> > a caveat we'd document or arrange for an exception to occur (like
> > putting a throw in a __getstate__ method).
> >
> > --~--~-~--~~~---~--~~
> > You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> > To post to this group, send email to sqlalchemy@googlegroups.com
> > To unsubscribe from this group, send email to
> sqlalchemy+unsubscr...@googlegroups.com
> > For more options, visit this group at
> http://groups.google.com/group/sqlalchemy?hl=en
> > -~--~~~~--~~--~--~---
> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> To unsubscribe from this group, send email to
> sqlalchemy+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/sqlalchemy?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
Hi, I know this is an "OLD" threat but I was searching the group to see If I was not the first one doing this. I am not sure I understand very well what this threat is all about, but I want to extend the Column class for a different reason. I want to add extra functionality to the Column class which is absolutely NOT SA related. SA functionality should not be effected though. say I want to add a config value and some methods for rendering and validating screens: def MyColumn(Column): def __init(): dosomething to init def ExtraInfo(self): do_something_not_sa_related validation = 'someregex' and use MyColumn in places where I normally use Column(..) What do I need to take into account, I've done some tests and "Error hell" broke loose, where the errors are hidden deep inside SA so hard to overcome. Martijn On Dec 11, 2008, at 16:20 , Michael Bayer wrote: > > > On Dec 11, 2008, at 3:37 AM, Angri wrote: > >> >> Here it is: http://www.sqlalchemy.org/trac/ticket/1244 >> >> Maybe it is good idea to drop some new lines in faq? Something like >> this: >> >> Q: How should I extend sqlalchemy.schema.Column? >> A: You surely dont need it. Recommended way to achive your possible >> needs is to write instance-factory function which decorates creation >> of sqlalchemy.schema.Column instances. >> >> Q: But I'm really need it! >> A: Ok. To subclass Column, this is the current recipe: >> >> from sqlalchemy.sql.util import Annotated, annotated_classes >> >> class MyColumn(Column): >> ... >> >> class AnnotatedMyColumn(Annotated, MyColumn): >> pass >> >> annotated_classes[MyColumn] = AnnotatedMyColumn >> >> Do not forget to put AnnotatedMyColumn in the module namespace, or >> your schema will not be pickleable! >> >> Correct me please if I am wrong somewhere and excuse me for my >> English. > > Well the AnnotatedMyColumn part is less than ideal since its an > internal. the way that works could very likely change. Creating an > AnnotatedXXX class *can* be automated. the pickle thing just might be > a caveat we'd document or arrange for an exception to occur (like > putting a throw in a __getstate__ method). > > --~--~-~--~~~---~--~~ > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to sqlalchemy@googlegroups.com > To unsubscribe from this group, send email to > sqlalchemy+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en > -~--~~~~--~~--~--~--- > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
On Dec 11, 2008, at 3:37 AM, Angri wrote: > > Here it is: http://www.sqlalchemy.org/trac/ticket/1244 > > Maybe it is good idea to drop some new lines in faq? Something like > this: > > Q: How should I extend sqlalchemy.schema.Column? > A: You surely dont need it. Recommended way to achive your possible > needs is to write instance-factory function which decorates creation > of sqlalchemy.schema.Column instances. > > Q: But I'm really need it! > A: Ok. To subclass Column, this is the current recipe: > > from sqlalchemy.sql.util import Annotated, annotated_classes > > class MyColumn(Column): >... > > class AnnotatedMyColumn(Annotated, MyColumn): >pass > > annotated_classes[MyColumn] = AnnotatedMyColumn > > Do not forget to put AnnotatedMyColumn in the module namespace, or > your schema will not be pickleable! > > Correct me please if I am wrong somewhere and excuse me for my > English. Well the AnnotatedMyColumn part is less than ideal since its an internal. the way that works could very likely change. Creating an AnnotatedXXX class *can* be automated. the pickle thing just might be a caveat we'd document or arrange for an exception to occur (like putting a throw in a __getstate__ method). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
Here it is: http://www.sqlalchemy.org/trac/ticket/1244 Maybe it is good idea to drop some new lines in faq? Something like this: Q: How should I extend sqlalchemy.schema.Column? A: You surely dont need it. Recommended way to achive your possible needs is to write instance-factory function which decorates creation of sqlalchemy.schema.Column instances. Q: But I'm really need it! A: Ok. To subclass Column, this is the current recipe: from sqlalchemy.sql.util import Annotated, annotated_classes class MyColumn(Column): ... class AnnotatedMyColumn(Annotated, MyColumn): pass annotated_classes[MyColumn] = AnnotatedMyColumn Do not forget to put AnnotatedMyColumn in the module namespace, or your schema will not be pickleable! Correct me please if I am wrong somewhere and excuse me for my English. On 11 дек, 01:23, Michael Bayer <[EMAIL PROTECTED]> wrote: > hey send it as an email attachment, or create a ticket in trac as > guest/guest and attach it there: http://www.sqlalchemy.org/trac/newticket > > On Dec 10, 2008, at 5:19 PM, Angri wrote: > > > > > > >> if you'd like to submit a patch which defines __visit_name__ for all > >> ClauseElements and removes the logic from VisitableType to guess the > >> name, it will be accepted. The second half of VisitableType still > >> may > >> be needed since it improves performance. > > > Ok, I did it. Can not find where I can attach file to message in > > google groups, so I put it in paste bin:http://paste.org/index.php?id=4463 > > > I made some automated tests to make sure that nothing will break. > > Existing Visitable's descendants after applying the patch will have > > exactly the same value of __visit_name__ property. I also put small > > test from first message in the topic. It fails with vanila trunk and > > pass ok with patched. > > > Take a look, please. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
hey send it as an email attachment, or create a ticket in trac as guest/guest and attach it there: http://www.sqlalchemy.org/trac/newticket On Dec 10, 2008, at 5:19 PM, Angri wrote: > >> if you'd like to submit a patch which defines __visit_name__ for all >> ClauseElements and removes the logic from VisitableType to guess the >> name, it will be accepted. The second half of VisitableType still >> may >> be needed since it improves performance. > > Ok, I did it. Can not find where I can attach file to message in > google groups, so I put it in paste bin: http://paste.org/index.php?id=4463 > > I made some automated tests to make sure that nothing will break. > Existing Visitable's descendants after applying the patch will have > exactly the same value of __visit_name__ property. I also put small > test from first message in the topic. It fails with vanila trunk and > pass ok with patched. > > Take a look, please. > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
> if you'd like to submit a patch which defines __visit_name__ for all > ClauseElements and removes the logic from VisitableType to guess the > name, it will be accepted. The second half of VisitableType still may > be needed since it improves performance. Ok, I did it. Can not find where I can attach file to message in google groups, so I put it in paste bin: http://paste.org/index.php?id=4463 I made some automated tests to make sure that nothing will break. Existing Visitable's descendants after applying the patch will have exactly the same value of __visit_name__ property. I also put small test from first message in the topic. It fails with vanila trunk and pass ok with patched. Take a look, please. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
On Wednesday 10 December 2008 00:42:14 Michael Bayer wrote:
> On Dec 9, 2008, at 4:09 PM, Angri wrote:
> > particularly
> > explicitly define class properties (inheritable class
> > properties!)
>
> if you'd like to submit a patch which defines __visit_name__ for
> all ClauseElements and removes the logic from VisitableType to
> guess the name, it will be accepted. The second half of
> VisitableType still may be needed since it improves performance.
>
> > instead of doing things like eval("Annotated%s" %
> > element.__class__.__name__)?
>
> The __name__ based logic is gone.
>
> The code that used to be there, which is the code i was mentioning,
> was this:
>
>return object.__new__(
> type.__new__(type, "Annotated%s" %
> element.__class__.__name__, (Annotated, element.__class__), {})
> )
>
> This generates a new type dynamically and is the preferred way to
> do this. The __name__ is only used there to give the resultling
> class a name and not any kind of lookup semantics.
>
> Unfortunately the resulting class is not pickleable since it is not
> declared in the module. So there are two options I'm aware of.
> Either do this:
>
>
> class AnnotatedClauseElement(Annotated, ClauseElement):
>
>
> class AnnotatedFromClause(Annotated, FromClause):
>...
>
> class AnnotatedColumnElement(Annotated, ColumnElement):
>...
>
> < continue for 100 more classes in sqlalchemy.expression>
>
> Or do the "Exec" thing we have. A third alternative would be great
> if you have one in mind.
one can inject things in module's locals() via locals().update( ...)
but i'm not sure if this is a feature-to-stay.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
On Dec 9, 2008, at 4:09 PM, Angri wrote:
> particularly
> explicitly define class properties (inheritable class properties!)
if you'd like to submit a patch which defines __visit_name__ for all
ClauseElements and removes the logic from VisitableType to guess the
name, it will be accepted. The second half of VisitableType still may
be needed since it improves performance.
> instead of doing things like eval("Annotated%s" %
> element.__class__.__name__)?
The __name__ based logic is gone.
The code that used to be there, which is the code i was mentioning,
was this:
return object.__new__(
type.__new__(type, "Annotated%s" %
element.__class__.__name__, (Annotated, element.__class__), {})
)
This generates a new type dynamically and is the preferred way to do
this. The __name__ is only used there to give the resultling class a
name and not any kind of lookup semantics.
Unfortunately the resulting class is not pickleable since it is not
declared in the module. So there are two options I'm aware of.
Either do this:
class AnnotatedClauseElement(Annotated, ClauseElement):
class AnnotatedFromClause(Annotated, FromClause):
...
class AnnotatedColumnElement(Annotated, ColumnElement):
...
< continue for 100 more classes in sqlalchemy.expression>
Or do the "Exec" thing we have. A third alternative would be great if
you have one in mind.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
On Dec 9, 2008, at 4:09 PM, Angri wrote: > > I can not agree that extending is "safe" as I've encountered another > problem with custom class name: > http://www.sqlalchemy.org/trac/browser/sqlalchemy/trunk/lib/sqlalchemy/sql/util.py#L145 rev 5454 removes AnnotatedColumn's reliance upon names within the expression package. To subclass Column, this is the current recipe: from sqlalchemy.sql.util import Annotated, annotated_classes class MyColumn(Column): __visit_name__ = "column" class AnnotatedMyColumn(Annotated, MyColumn): __visit_name__ = "column" annotated_classes[MyColumn] = AnnotatedMyColumn There are of course metaclass methods of making this automatic. But here we have pure "traditional OOP" style - make all the classes explicitly, register them, etc. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
On Dec 9, 2008, at 4:09 PM, Angri wrote:
>
> I can not agree that extending is "safe" as I've encountered another
> problem with custom class name:
> http://www.sqlalchemy.org/trac/browser/sqlalchemy/trunk/lib/sqlalchemy/sql/util.py#L145
> And I guess it is not the last :(
It probably is the last. That's something which has been added
recently. It previously worked in a dynamic fashion but was changed
to what you see there for pickling purposes. However it can be
expanded to support classes which aren't present, so that you wouldn't
notice it. This technique is also something I picked up from a very
well known and respected Python developer.
> I see, you propose not to extend class Column and write "a function
> that creates instances" but it seems to me that this approach is not
> good because it is not "pythonic", it is not "object-oriented"ly, it
> is hard to write, support and extend.
It's how most of SQLA clause elements are invoked, in fact. Python's
philosophy is defnitely not in the vein of "use objects and
inheritance for everything", you're thinking of Java, which does not
provide standalone functions in the language. Using functions to
return objects provides a convenient layer of variability between the
composed structure of what the function returns and the public
signature of that function.
Subclassing is definitely not the only way to extend something and the
problem you're experiencing is a variant of the so-called "fragile
base" problem.What strikes me as most "unpythonic" here is that
you think you need a rewrite of the library in a Java-like GOF style
when a simple, one line function will solve your problem.
>
> Btw, I still think that relying on class name is a bad way to do
> things.
Here is where the technique is derived from, from the "visitor.py"
module in Python's own compiler module:
http://svn.python.org/view/python/branches/release26-maint/Lib/compiler/visitor.py?rev=66717&view=auto
> What do you think Michael, how difficult it can be to rewrite
> those pieces of code to use more OOP-like technics, particularly
> explicitly define class properties (inheritable class properties!)
> instead of doing things like eval("Annotated%s" %
> element.__class__.__name__)?
When I first started Python, I used a traditional GOF-visitor pattern,
with explicit visit_XXX and dispatcher methods. It's extremely
verbose and tedious to refactor. It didn't take too long for me to
realize I was porting Java methodologies that are entirely
inappropriate for a dynamic language like Python. Subclassing Column
in any case requires knowledge of the visitation logic - you either
need to implement visit_XXX() in the GOF style, or __visit_name__ =
'whatever' in the Pythonic style.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
I can not agree that extending is "safe" as I've encountered another
problem with custom class name:
http://www.sqlalchemy.org/trac/browser/sqlalchemy/trunk/lib/sqlalchemy/sql/util.py#L145
And I guess it is not the last :(
I see, you propose not to extend class Column and write "a function
that creates instances" but it seems to me that this approach is not
good because it is not "pythonic", it is not "object-oriented"ly, it
is hard to write, support and extend.
Continuing experiments I created the following (ugly) code. Maybe it
will help someone with similar needs.
class CustomColumnMetaclass(sqlalchemy.Column.__metaclass__):
def __init__(cls, clsname, bases, dct):
cls.__name__ = sqlalchemy.Column.__name__
cls.__visit_name__ = sqlalchemy.Column.__visit_name__
super(CustomColumnMetaclass, cls).__init__(clsname, bases, dct)
class CustomColumn(sqlalchemy.Column):
__metaclass__ = CustomColumnMetaclass
Btw, I still think that relying on class name is a bad way to do
things. What do you think Michael, how difficult it can be to rewrite
those pieces of code to use more OOP-like technics, particularly
explicitly define class properties (inheritable class properties!)
instead of doing things like eval("Annotated%s" %
element.__class__.__name__)?
On Dec 7, 1:33 am, Michael Bayer <[EMAIL PROTECTED]> wrote:
> On Dec 6, 2008, at 4:27 PM, Angri wrote:
>
> > 1. What about another side-effects depending on clsname? Is it
> > actually safe to extend sqlalchemy.schema.Column, or it may have
> > unpredictable behavior similar to that i've encountered?
>
> The Column object is one of the most key classes in all of SQLAlchemy
> and we do put it through some fairly intricate copy/proxy patterns
> particularly when using the ORM. Extending it should be "safe",
> although this is usually not needed. For custom creation patterns as
> you're seeking here its more straightforward to build a creation
> function, so that the resulting object is returned unchanged in its
> type.
>
>
>
> > 2. (almost offtopic) Is 'exec' really need there? What's wrong with
> > closures?
>
> the visit step is called very intensively during statement compilation
> so exec'ing the direct code instead of relying upon getattr() with a
> composed name at runtime is an optimization to reduce function-call
> and attribute-retrieval overhead. Just as a point of reference I
> tried rewriting our "visit" dispatcher in C, and my experiments showed
> that using the exec approach you see there performs just as well -
> though the time savings compared to a basic getattr() approach are
> very small.
>
> Since you raised the issue, I went to try a different approach which
> is probably the best possible approach without using exec, which is
> this:
>
> visit_name = cls.__dict__["__visit_name__"]
> if isinstance(visit_name, str):
> getter = operator.attrgetter("visit_%s" % visit_name)
> def _compiler_dispatch(self, visitor, **kw):
> return getter(visitor)(self, **kw)
> else:
> def _compiler_dispatch(self, visitor, **kw):
> return getattr(visitor, 'visit_%s' %
> self.__visit_name__)(self, **kw)
>
> Above, we use operator.attrgetter so that the string composition is
> already handled, and the attrgetter itself is a native object which
> performs as fast as direct access. This change still adds a few
> function calls per compile. Our bench of a single select() compile
> goes from 183 to 187, and two of the zoomark_orm tests fail past the
> 5% threshhold, with tests three and four moving from 6623 to 6723 and
> 23345 to 23861 function calls, respectively. Which is an extremely
> small amount, so its not a terribly big deal either way. So the exec
> approach is saving a tiny amount of call counts, but not necessarily
> any actual time. I'd be willing to scrap it if it continues to scare
> other developers.
>
> The reason I'm at all comfortable with exec is that we're already
> using 'exec' for decorators - its a technique used by the "decorators"
> module (which I'd like to transition to at some point) to faithfully
> represent the calling signature of a decorated function.
>
>
>
> > 3. Maybe I should send it to developers mailing list?
>
> eitherdevel is not very active. though this is a pretty develop-y
> subject...
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: Extending sqlalchemy.schema.Column and metaprogramming traps
On Dec 6, 2008, at 4:27 PM, Angri wrote:
> 1. What about another side-effects depending on clsname? Is it
> actually safe to extend sqlalchemy.schema.Column, or it may have
> unpredictable behavior similar to that i've encountered?
The Column object is one of the most key classes in all of SQLAlchemy
and we do put it through some fairly intricate copy/proxy patterns
particularly when using the ORM. Extending it should be "safe",
although this is usually not needed. For custom creation patterns as
you're seeking here its more straightforward to build a creation
function, so that the resulting object is returned unchanged in its
type.
>
> 2. (almost offtopic) Is 'exec' really need there? What's wrong with
> closures?
the visit step is called very intensively during statement compilation
so exec'ing the direct code instead of relying upon getattr() with a
composed name at runtime is an optimization to reduce function-call
and attribute-retrieval overhead. Just as a point of reference I
tried rewriting our "visit" dispatcher in C, and my experiments showed
that using the exec approach you see there performs just as well -
though the time savings compared to a basic getattr() approach are
very small.
Since you raised the issue, I went to try a different approach which
is probably the best possible approach without using exec, which is
this:
visit_name = cls.__dict__["__visit_name__"]
if isinstance(visit_name, str):
getter = operator.attrgetter("visit_%s" % visit_name)
def _compiler_dispatch(self, visitor, **kw):
return getter(visitor)(self, **kw)
else:
def _compiler_dispatch(self, visitor, **kw):
return getattr(visitor, 'visit_%s' %
self.__visit_name__)(self, **kw)
Above, we use operator.attrgetter so that the string composition is
already handled, and the attrgetter itself is a native object which
performs as fast as direct access. This change still adds a few
function calls per compile. Our bench of a single select() compile
goes from 183 to 187, and two of the zoomark_orm tests fail past the
5% threshhold, with tests three and four moving from 6623 to 6723 and
23345 to 23861 function calls, respectively. Which is an extremely
small amount, so its not a terribly big deal either way. So the exec
approach is saving a tiny amount of call counts, but not necessarily
any actual time. I'd be willing to scrap it if it continues to scare
other developers.
The reason I'm at all comfortable with exec is that we're already
using 'exec' for decorators - its a technique used by the "decorators"
module (which I'd like to transition to at some point) to faithfully
represent the calling signature of a decorated function.
>
> 3. Maybe I should send it to developers mailing list?
eitherdevel is not very active. though this is a pretty develop-y
subject...
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
