[sqlalchemy] Re: How to get setter-like behaviour universally across a custom type? (with UTC DateTime examples)
because an attribute can be mapped to multiple columns, i.e.http://www.sqlalchemy.org/docs/orm/mapper_config.html#mapping-a-class Ahh... thanks. as I did this example and thought back upon how often people want to poke around the mapper configuration, i started trying to think of how to turn the mapper hierarchy into a more of a well described DOM-type of system. In this case one thing I thought of would be to add a first_column accessor to ColumnProperty. because an attribute can be mapped to multiple columns, i.e.http://www.sqlalchemy.org/docs/orm/mapper_config.html#mapping-a-class Ah... thanks. as I did this example and thought back upon how often people want to poke around the mapper configuration, i started trying to think of how to turn the mapper hierarchy into a more of a well described DOM-type of system. In this case one thing I thought of would be to add a first_column accessor to ColumnProperty. Allowing multiple columns does make it more DOM-like. But once you know ColumnProperty.columns is a sequence (eg: I didn't), columns[0] is just as good as first_column. I know very little about your internal implementation, but I might as well toss a thought out there on navigating a mapper configuration... it would be more intuitive/explicit to me if there were a DbColumnProperty base class, and two derived classes: SingleDbColumnProperty and MultiDbColumnProperty (and I guess you would need RelationProperty, and probably more). The former with a simple .column attribute, and the latter with your current .columns sequence attribute. The current situation of checking for the existence of a .columns attribute to find real persistent data is not that intuitive. It would be nicer to look through the mapper properties for DbColumn instances. Also, I would bet that the vast majority of implementations would be SingleColumnProperty, so the niggly detail of columns[0] wouldn't exist (although that would obviously break a lot of existing code, unless SingleColumnProperty always had single entry .columns... but then that is what you have now). Overall it is great right now and my commentary is probably pointless and/or naive. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: How to get setter-like behaviour universally across a custom type? (with UTC DateTime examples)
On Jul 2, 2011, at 2:25 PM, Russ wrote: I know very little about your internal implementation, but I might as well toss a thought out there on navigating a mapper configuration... it would be more intuitive/explicit to me if there were a DbColumnProperty base class, and two derived classes: SingleDbColumnProperty and MultiDbColumnProperty (and I guess you would need RelationProperty, and probably more). The former with a simple .column attribute, and the latter with your current .columns sequence attribute. I try to avoid requiring the use of isinstance() to navigate a hierarchy. Ideally there'd be just one kind of node (right now, mapper or column(s)) and one kind of edge (currently column property, relationship property, as well as some others like synonym property).When working with a DOM like elementtree, you know what kind of node you're on based on how you got there, i.e. did you look at .tail, children, attr. Hence the idea of .columns as well as first_column, i.e. like jquery's :first. I'm having kind of a big idea on this now. like an 0.8 kind of thing -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: How to get setter-like behaviour universally across a custom type? (with UTC DateTime examples)
On Jul 1, 2011, at 1:19 AM, Russ wrote: The only thing I'm still unsure of in the code is why mapper.columns is a collection and it required checking columns[0], but I can either just live with that or look into it later. because an attribute can be mapped to multiple columns, i.e. http://www.sqlalchemy.org/docs/orm/mapper_config.html#mapping-a-class-against-multiple-tables . as I did this example and thought back upon how often people want to poke around the mapper configuration, i started trying to think of how to turn the mapper hierarchy into a more of a well described DOM-type of system.In this case one thing I thought of would be to add a first_column accessor to ColumnProperty. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: How to get setter-like behaviour universally across a custom type? (with UTC DateTime examples)
I discovered the sqlalchemy.orm.validates decorator, which executes when I want it to (on attribute assignment) so I got out my hacksaw and tried to make it work in one fell swoop... implementing automatic UTC assignment for all UTCEnforcedDateTime columns. I'm not comfortable with it yet... but a mostly-inclusive demo code snippet is here: http://pastebin.com/wB4BLzax Where I intend to do that last bit of hackery (utcConversionList and __sa_validators__ work) by introspecting through a large number of classes (all having the TableDefMixin) and looking for UTCEnforcedDateTime Column definitions that should be added to the utcConversionList for the class. Method tbd there, but not important. Although it is doing what I want it to do (hands off UTC assignment where expected), I'm really not comfortable with it. It seems *highly* sketchy, quite indirect, and it seems like there must be a better way to simply set a default validator for a custom Column type. Is there? It would be much tidier to put the validator into the UTCEnforcedDateTime class. If not... can I rely on the direct setting of __sa_validators__ working in the future? Using the orm.validates decorator in each and every class is obviously the better choice for future compatibility, but if I want one-shot assignment as I do above, I can't use the orm.validates because it is too late to set __sa_validators__ directly as orm.validates does, and I needed to be brutal and go to the __dict__ directly. I expect there is a simple answer and all my hacking/exploration related to my posts above was pointless... except it has been highly educational to root through the SQLAlchemy implementation here. Tidbits learned include now understanding descriptors (never had cause to before), and learning about attrgetter and attrsetter. Stepping into the attribute assignment (InstrumentedAttribute.__set__) was highly confusing until reading up on those bits!! instance_state() and instance_dict() instantly returning was somewhat mysterious for a while! Thanks, Russ -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: How to get setter-like behaviour universally across a custom type? (with UTC DateTime examples)
you're close, I've added a note to the validates documentation at http://www.sqlalchemy.org/docs/orm/mapper_config.html#simple-validators that will point a reader on to the more comprehensive solution. I've added an example for you as well as Jason who asked almost the same question earlier, which illustrates the TypeDecorator in conjunction with an attribute listener that is applied to all occurrences of the target type, an approach also used by the mutable attributes extension, and which we may look into adding as more of a built in feature in the future although the recipe should be straightforward. http://www.sqlalchemy.org/trac/wiki/UsageRecipes/ValidateAllOccurrencesOfType On Jun 30, 2011, at 2:42 AM, Russ wrote: I discovered the sqlalchemy.orm.validates decorator, which executes when I want it to (on attribute assignment) so I got out my hacksaw and tried to make it work in one fell swoop... implementing automatic UTC assignment for all UTCEnforcedDateTime columns. I'm not comfortable with it yet... but a mostly-inclusive demo code snippet is here: http://pastebin.com/wB4BLzax Where I intend to do that last bit of hackery (utcConversionList and __sa_validators__ work) by introspecting through a large number of classes (all having the TableDefMixin) and looking for UTCEnforcedDateTime Column definitions that should be added to the utcConversionList for the class. Method tbd there, but not important. Although it is doing what I want it to do (hands off UTC assignment where expected), I'm really not comfortable with it. It seems *highly* sketchy, quite indirect, and it seems like there must be a better way to simply set a default validator for a custom Column type. Is there? It would be much tidier to put the validator into the UTCEnforcedDateTime class. If not... can I rely on the direct setting of __sa_validators__ working in the future? Using the orm.validates decorator in each and every class is obviously the better choice for future compatibility, but if I want one-shot assignment as I do above, I can't use the orm.validates because it is too late to set __sa_validators__ directly as orm.validates does, and I needed to be brutal and go to the __dict__ directly. I expect there is a simple answer and all my hacking/exploration related to my posts above was pointless... except it has been highly educational to root through the SQLAlchemy implementation here. Tidbits learned include now understanding descriptors (never had cause to before), and learning about attrgetter and attrsetter. Stepping into the attribute assignment (InstrumentedAttribute.__set__) was highly confusing until reading up on those bits!! instance_state() and instance_dict() instantly returning was somewhat mysterious for a while! Thanks, Russ -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: How to get setter-like behaviour universally across a custom type? (with UTC DateTime examples)
I've added an example for you as well as Jason who asked almost the same question earlier, which illustrates the TypeDecorator in conjunction with an attribute listener that is applied to all occurrences of the target type, an approach also used by the mutable attributes extension, and which we may look into adding as more of a built in feature in the future although the recipe should be straightforward. http://www.sqlalchemy.org/trac/wiki/UsageRecipes/ValidateAllOccurrenc... Thanks!! I would *never* have associated Jason's question with mine, but no matter... the end result is perfect. I had actually gotten it completely working with my messy __sa_validators__ hackery (and a heck of a lot of questionable introspection), but I now have it working with the event system as you have suggested. It is MUCH cleaner and I like it a lot. Here is my final implementation: http://pastebin.com/33Zkfz1h The only thing I'm still unsure of in the code is why mapper.columns is a collection and it required checking columns[0], but I can either just live with that or look into it later. Also - prior to your suggestion I was still on SQLAlchemy 0.6.6 and this prompted me to make the leap to 0.7.1 ... all code now working fine after that transition with only minor hiccups. That was an excellent introduction to the new event system as well... thanks again! Russ -- Code is reproduced below as well, in case the pastebin ever fails: from pytz import UTC import sqlalchemy as sa import sqlalchemy.orm as orm import globalenv class UTCEnforcedDateTime(sa.types.TypeDecorator): DateTime type that ensures datetime objects are offset-aware UTC. impl = sa.types.DateTime def process_bind_param(self, value, engine): if (value is not None) and (value.tzinfo != UTC): raise Exception(Data MUST be offset-aware UTC!) return value def process_result_value(self, value, engine): if value is not None: return value.replace(tzinfo = UTC) return value def _EnsureUTC(target, value, oldvalue, initiator): 'Set' Event handler for all UTCEnforcedDateTime columns. This handler simply ensures that the provided 'value' is an offset- aware UTC datetime. SQLAlchemy validator (for @validates) for use with UTCEnforcedDateTime. Use of this validator will convert times to UTC on assignment (so that the UTCEnforcedDateTime implementation doesn't throw an exception on commit). dt = value if dt == None: return dt if dt.tzinfo == UTC: return dt tz = globalenv.LocalTZ #pytz timezone that naive datetimes are in #Convert naive time to local time... # - normalize is needed to deal with DST funkiness dt_tz = tz.normalize(tz.localize(dt)) return dt_tz.astimezone(UTC) @sa.event.listens_for(orm.mapper, mapper_configured) def _Configure_UTCEnforcedDateTime_Setter(mapper, class_): A mapper-configured listener that is triggered every time an ORM_ class mapper is registered (once per class). This event handler makes sure that any defined UTCEnforcedDateTime are always receiving data with properly determined UTC offset-aware values (with the use of the _EnsureUTC handler). Adapted from sample code here: http://www.sqlalchemy.org/trac/wiki/UsageRecipes/ValidateAllOccurrencesOfType for prop in mapper.iterate_properties: if hasattr(prop, 'columns'): #it is a column (not a relation) if isinstance(prop.columns[0].type, UTCEnforcedDateTime): #Set up a listener for datetime setting events... classAttr = getattr(class_, prop.key) #the attr of the mapped class sa.event.listen( classAttr, #We want to listen for when classAttr... set, #has a set event (like a property setter)... _EnsureUTC,#and use _EnsureUTC as the handler... retval = True) #and allow _EnsureUTC to change the attr with it's return -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: How to get setter-like behaviour universally across a custom type? (with UTC DateTime examples)
When I realized that process_bind_param only happens on commit, I decided to switch my strategy to simply confirming that all incoming outgoing datetime values are offset-aware UTC using this simpler code: http://pastebin.com/gLfCUkX3 Sorry - I messed up that code segment on edit for pastebin. Here is a cleaned up version: http://pastebin.com/HcnnmXtV -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
