good on yer! one thing that i sorta failed is making a clean set of tests for this. if u aren't too far in the usage, save some major future headaches and do yourself a favour, make such test set. All the things u've tried has to be (simple) testcases - it will be the _spec_ of how the thing works / what u want from it. Anyone else on the same path could also use it then ;-)
i hope that once u do that u'll get an idea of where exactly u have failed and fix it. as of expiration, it also did me some trouble. i have not stepped on that session-gone one but i can see how it happens. The (expected) life-times of SA-touched things aren't that obvious, and aren't documented. e.g. Objects living longer than session or in an extreme case, the mappers, might be troublesome. what i can suggest is, put debug statements and run with echo=True and watch what goes on behind and how SA-activities intertwist with traits' /your ones. ciao svilen On Sunday 22 March 2009 04:00:27 Christiaan Putter wrote: > Hi svilen, > > Thanks for your advice, going through your code helped a lot in > understanding how SA works. > > I've gotten traited classes to behave like classes generated by > SA's declarative extension. After a lot of stepping through code I > realised the main problem was that SA removes values from an > instance __dict__ once they have been expired. Traits smartly (or > not so smartly) picks this up and on the next access to those > attributes and returns their default values. Whereas SA would > normally refresh the values from SQL. This caused some serious > headaches, and really strange behaviour. > > My solution was to the overwrite the class's __getattribute__ with: > > def __getattribute__(self, key): > try: > dict = super(HasTraitsORM, > self).__getattribute__('__dict__') > > if ('_sa_state' in dict): > state = dict['_sa_state'] > if key in state.manager.keys(): > return state.get_impl(key).get(state) > > except Exception, e: > print "Exception in SA getattr for %s: %s" % (key, e) > pass > > return super(HasTraitsORM, self).__getattribute__(key) > > > Which tries to first get an attribute from the instance's state > (which will issue SQL if needed), and if that fails pass the > request along to its super class where Traits will do it's magic if > needed. > > This seems to work in the tests if been trying. Most of the work > is done by SA declarative, and I've just added a thin layer on top > to get it to behave well with Traited classes and map traits to SA > columns. > > One issue I'm still having though is that after commits all > attributes get expired. If you close the session the instance was > attached to the data you just commited can't be accessed any more. > This is also the behaviour of plain declarative classes. This is > rather unintuitive I believe since once you've comitted an instance > and closed the session you can't use the data you just set any > more. In my case this means records being displayed in a gui get > corrupted. > > Though what's strange is that if you access the attributes after > the commit and before you close the session, and only then close > the session, the values remain in the instance's __dict__ and the > class behaves normally again. Is this intended for some reason? > Is there a workaround? Personally I don't think quantum physics > should apply to the bits of my programs and thus merely observing > them should not change their behaviour. From going through the > source I found a 'dont_expire_missing' attribute on attribute > implementations, though this does unfortunately not do what I'd > like. > > So what I'm doing now is keeping a session open on the gui's > thread. Which causes some problems when trying to modify said > instances from other threads and commiting the new values. > > How's this usually done is SA? Right now I've put a lock on the > gui thread's session and allow other threads to issue a commit on > that main thread if need be. I can't imagine this is the best way > to do it... > > Hope you're all having a great weekend, > > Christian > > 2009/2/8 <a...@svilendobrev.com>: > > afaiknow these traits are like my own static_types, i.e. > > descriptors holding metadata and applying it to attribute access. > > i have been combining SA with static_type for more than 2 years > > now, since SA 3.0. > > > > The approach i did in the beginning was to replace the object's > > __dict__ by something smart that is static_type-aware. When > > InstrumentionManager framework came into place, i did not find it > > any different, as SA still uses the __dict__ for data access. The > > difference is that now my __dict__ replacement is created once > > and not at every attr.access. > > > > i did suggest one patch about replaceing the SA's > > obj.__dict__.whatever usage with an explicit set of methods (so > > one knows what access to mime), and that was making SA 1-2% > > faster, but it wasn't accepted. > > > > basicaly now there's a thin layer on top of SA, then SA itself, > > then a thick layer underneath managing the data (the fake > > __dict__). > > > > declarative+traits... u'll end up where i was. dbcook.sf.net is > > doing that - since beginning. and it's switchable on/off. > > It all works well and stable, in project with 250-300 classes, > > although about 15% slower than without it (-:) > > > > The sa2static code: > > svn co > > http://dbcook.svn.sourceforge.net/svnroot/dbcook/trunk/dbcook/usa > >ge/static_type/ > > > > The static_type itself: > > svn co > > https://dbcook.svn.sourceforge.net/svnroot/dbcook/static_type > > > > whole dbcook: > > svn co https://dbcook.svn.sourceforge.net/svnroot/dbcook/trunk > > > > i have quite some experience fighting this field, ask if u want. > > > > ciao > > svilen > > www.svilendobrev.com > > > > On Sunday 08 February 2009 01:51:20 cputter wrote: > >> Hi guys and girls, > >> > >> I've recently discovered the joys of using sqlalchemy and would > >> love to using it together with Traits. A few months back there > >> was an attempt to integrate sqlalchemy into traits, though it > >> wasn't really comprehensive in exploiting all of sqlalchemy's > >> potential. > >> > >> So I'm trying to work on that and combine ext.Declarative with > >> traits. The basic idea is to use the DeclarativeMeta type to > >> generate Columns from Traits and pass those on for the > >> Declarative extension to do its magic. This would allow mixing > >> of sqlalchemy attributes and trait attributes in a single class > >> so that we could still make use of all the relational setup > >> sqlalchemy does in any case. > >> > >> Reading through several threads and looking at Elixir's SA > >> integration helped me a bit though I couldn't find any > >> documentation on how to implement the InstrumentationManager > >> interface. I'm assuming this would be essential for letting > >> Traits and SQLAlchemy play well together. > >> > >> There's still a lot of work to do, and I'm not really sure what > >> needs to be done for everything to work properly. Would really > >> appreciate it if someone could help me out. > >> > >> > >> Here's an example of how it's working at the moment, I'll add > >> the actual implementation at the end. > >> > >> ##################################### > >> class User(HasTraitsORM): > >> __tablename__ = 'users' > >> > >> id = Column('id', Integer, primary_key=True) > >> name = Str(sqldb=True) > >> > >> def _name_changed(self, old, new): > >> print 'Changed name from %s to %s.' % (old, new) > >> > >> def __repr__(self): > >> return '<User(%s, %s)>' % (self.id, self.name) > >> > >> people = ['John', 'Charls','Steve','Smith','Jane'] > >> > >> for per in people: > >> obj = User(name=per) > >> sess = sqlservice.Session() > >> sess.add(obj) > >> sess.commit() > >> sess.close() > >> print obj > >> > >> session = sqlservice.Session() > >> print '\nQuery all users\n' > >> for user in session.query(User).order_by(User.name).all(): > >> print user > >> > >> session.close() > >> > >> > >> ############################ > >> > >> Which spits out: > >> ############################### > >> Changed name from to John. > >> <User(users.id, )> > >> Changed name from to Charls. > >> <User(users.id, )> > >> Changed name from to Steve. > >> <User(users.id, Steve)> > >> Changed name from to Smith. > >> <User(users.id, Smith)> > >> Changed name from to Jane. > >> <User(users.id, Jane)> > >> > >> Query all users > >> > >> <User(2, Charls)> > >> <User(1, John)> > >> > >> ############################## > >> > >> Which is really strange behaviour. There's obviously something > >> wrong in my implementation of HasTraitsORM but why the different > >> results within the same loop??? Why add only two instances? > >> > >> Totally baffles me. > >> > >> Here's the rest of my code, hope somehow can help me out. It's > >> very messy, I've been hacking at it like crazy with no success > >> :-) > >> > >> Hope you're all having a great weekend. > >> -Chris > >> > >> > >> ############################## > >> > >> # Standard library imports. > >> import logging > >> > >> # Enthought library imports > >> from enthought.preferences.api import Preferences > >> from enthought.traits.api import \ > >> HasTraits, MetaHasTraits, Int, Str, Bool, Float, Any,\ > >> String, Enum, Python, \ > >> on_trait_change, TraitListObject > >> > >> # Package imports > >> import sqlalchemy > >> from sqlalchemy import Column, Integer > >> from sqlalchemy.schema import MetaData > >> from sqlalchemy.orm.interfaces import MapperProperty, > >> InstrumentationManager > >> from sqlalchemy.orm.attributes import get_attribute, > >> set_attribute, is_instrumented > >> from sqlalchemy.orm.collections import InstrumentedList, > >> collection_adapter > >> from sqlalchemy.ext.declarative import _as_declarative > >> > >> > >> # Setup a logger for this module. > >> logger = logging.getLogger(__name__) > >> > >> > >> TRAIT_MAPPING = { > >> Int : 'sqlalchemy.Integer', > >> Str : 'sqlalchemy.Text', > >> Enum : 'sqlalchemy.Text', > >> String : 'sqlalchemy.Text', > >> Float : 'sqlalchemy.Float', > >> Bool : 'sqlalchemy.Boolean', > >> } > >> > >> > >> class HasTraitsORMState(InstrumentationManager): > >> def __init__(self, cls): > >> self.states = {} > >> > >> def instrument_attribute(self, class_, key, attr): > >> pass > >> > >> def install_descriptor(self, class_, key, attr): > >> pass > >> > >> def uninstall_descriptor(self, class_, key, attr): > >> pass > >> > >> def instrument_collection_class(self, class_, key, > >> collection_class): > >> return ObjectCollection > >> > >> def get_instance_dict(self, class_, instance): > >> return instance.__dict__ > >> > >> def initialize_instance_dict(self, class_, instance): > >> instance.reset_traits() > >> > >> def initialize_collection(self, key, state, factory): > >> data = factory() > >> return ObjectCollectionAdapter(key, state, data), data > >> > >> def install_state(self, class_, instance, state): > >> self.states[id(instance)] = state > >> > >> def state_getter(self, class_): > >> def find(instance): > >> return self.states[id(instance)] > >> return find > >> > >> > >> class ObjectCollectionAdapter(object): > >> """ > >> An adapter for SQLAlchemy for TraitsListObject which is > >> the collection > >> we use for storing instances of classes within > >> attributes of other > >> classes. > >> > >> TODO: Think of a way to get this to behave like a > >> normal traited > >> attribute of objects. > >> At the moment collection attributes are set through the > >> mapper by SA > >> when using relations or backref. > >> > >> """ > >> def __init__(self, key, state, collection): > >> self.key = key > >> self.state = state > >> self.collection = collection > >> setattr(collection, '_sa_adapter', self) > >> > >> def unlink(self, data): > >> setattr(data, '_sa_adapter', None) > >> > >> def adapt_like_to_iterable(self, obj): > >> return iter(obj) > >> > >> def append_with_event(self, item, initiator=None): > >> self.collection.append(item, emit=initiator) > >> > >> def append_without_event(self, item): > >> self.collection.append(item, emit=False) > >> > >> def remove_with_event(self, item, initiator=None): > >> self.collection.remove(item, emit=initiator) > >> > >> def remove_without_event(self, item): > >> self.collection.remove(item, emit=False) > >> > >> def clear_with_event(self, initiator=None): > >> for item in list(self): > >> self.remove_with_event(item, initiator) > >> > >> def clear_without_event(self): > >> for item in list(self): > >> self.remove_without_event(item) > >> > >> def __iter__(self): > >> return iter(self.collection) > >> > >> def fire_append_event(self, item, initiator=None): > >> if initiator is not False and item is not None: > >> self.state.get_impl(self.key).fire_append_event > >> (self.state, item, > >> > >> initiator) > >> > >> def fire_remove_event(self, item, initiator=None): > >> if initiator is not False and item is not None: > >> self.state.get_impl(self.key).fire_remove_event > >> (self.state, item, > >> > >> initiator) > >> > >> def fire_pre_remove_event(self, initiator=None): > >> self.state.get_impl(self.key).fire_pre_remove_event > >> (self.state, > >> > >> initiator) > >> > >> class ObjectCollection(TraitListObject): > >> __emulates__ = list > >> > >> def __init__( self, trait, object, name, value ): > >> pass > >> > >> def __init__(self): > >> self.members = list() > >> def append(self, object, emit=None): > >> self.members.append(object) > >> #collection_adapter(self).fire_append_event(object, > >> emit) def remove(self, object, emit=None): > >> #collection_adapter(self).fire_pre_remove_event(object) > >> self.members.remove(object) > >> #collection_adapter(self).fire_remove_event(object, > >> emit) def __getitem__(self, index): > >> return self.members[index] > >> def __iter__(self): > >> return iter(self.members) > >> def __len__(self): > >> return len(self.members) > >> def __repr__(self): > >> if len(self.members) == 0: > >> return '[empty collection]' > >> s = '[' > >> for item in self.members[0:max(5, len(self.members))]: > >> s += '%s,' % item > >> s += '...]' > >> return s > >> > >> > >> class DeclarativeMetaTraits(MetaHasTraits): > >> pass > >> > >> # def __new__(cls, classname, bases, dict_): > >> # return MetaHasTraits.__new__(cls, classname, bases, > >> dict_) > >> > >> def __init__(cls, classname, bases, dict_): > >> > >> if '_decl_class_registry' in cls.__dict__: > >> super(DeclarativeMetaTraits, cls).__init__(cls, > >> classname, bases, dict_) > >> return > >> > >> if '__tablename__' not in dict_: > >> setattr(cls, '__tablename__', cls.__name__) > >> > >> # create sql columns from flagged traits > >> if '__class_traits__' in cls.__dict__: > >> traits = cls.__dict__['__class_traits__'] > >> for key, trait in traits.iteritems(): > >> if getattr( trait, 'sqldb' ): > >> args = getattr( trait, 'col_args' ) or () > >> kwargs = getattr( trait, 'col_kwargs' ) or > >> {} if 'name' not in kwargs: > >> kwargs['name'] = key > >> if 'type_' not in kwargs: > >> kwargs['type_'] = > >> eval(TRAIT_MAPPING[type (trait.trait_type)]) > >> > >> if getattr(trait, 'sqlpk'): > >> kwargs['primary_key'] = True > >> > >> c = Column(*args, **kwargs) > >> dict_[key] = c > >> setattr(cls, key, c) > >> > >> # Pass this along to SA's declarative mapper stuff > >> _as_declarative(cls, classname, dict_) > >> > >> super(DeclarativeMetaTraits, cls).__init__(cls, > >> classname, bases, dict_) > >> > >> > >> class HasTraitsORM(HasTraits): > >> __metaclass__ = DeclarativeMetaTraits > >> > >> # some SA hacking > >> __sa_instrumentation_manager__ = HasTraitsORMState > >> > >> _decl_class_registry = dict() > >> metadata = MetaData() > >> > >> # Any implicit traits added by SQLAlchemy are transient and > >> should not be > >> # copied through .clone_traits(), copy.copy(), or pickling. > >> _ = Python(transient=True) > >> > >> > >> # For some reason we need to override these, our superclass > >> screws things up > >> > >> def __getattr__(self, key): > >> if is_instrumented(self, key): > >> return get_attribute(self, key) > >> else: > >> try: > >> return getattr(self, key) > >> except: > >> pass > >> > >> def __setattr__(self, key, value): > >> if is_instrumented(self, key): > >> set_attribute(self, key, value) > >> else: > >> setattr(self, key, value) > >> > >> > >> ################################################# > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---