Hi svilen,
Thanks for your advice, going through your code helped a lot in
understanding how SA works.
I've gotten traited classes to behave like classes generated by SA's
declarative extension. After a lot of stepping through code I
realised the main problem was that SA removes values from an instance
__dict__ once they have been expired. Traits smartly (or not so
smartly) picks this up and on the next access to those attributes and
returns their default values. Whereas SA would normally refresh the
values from SQL. This caused some serious headaches, and really
strange behaviour.
My solution was to the overwrite the class's __getattribute__ with:
def __getattribute__(self, key):
try:
dict = super(HasTraitsORM, self).__getattribute__('__dict__')
if ('_sa_state' in dict):
state = dict['_sa_state']
if key in state.manager.keys():
return state.get_impl(key).get(state)
except Exception, e:
print "Exception in SA getattr for %s: %s" % (key, e)
pass
return super(HasTraitsORM, self).__getattribute__(key)
Which tries to first get an attribute from the instance's state (which
will issue SQL if needed), and if that fails pass the request along to
its super class where Traits will do it's magic if needed.
This seems to work in the tests if been trying. Most of the work is
done by SA declarative, and I've just added a thin layer on top to get
it to behave well with Traited classes and map traits to SA columns.
One issue I'm still having though is that after commits all attributes
get expired. If you close the session the instance was attached to
the data you just commited can't be accessed any more. This is also
the behaviour of plain declarative classes. This is rather
unintuitive I believe since once you've comitted an instance and
closed the session you can't use the data you just set any more. In my
case this means records being displayed in a gui get corrupted.
Though what's strange is that if you access the attributes after the
commit and before you close the session, and only then close the
session, the values remain in the instance's __dict__ and the class
behaves normally again. Is this intended for some reason? Is there a
workaround? Personally I don't think quantum physics should apply to
the bits of my programs and thus merely observing them should not
change their behaviour. From going through the source I found a
'dont_expire_missing' attribute on attribute implementations, though
this does unfortunately not do what I'd like.
So what I'm doing now is keeping a session open on the gui's thread.
Which causes some problems when trying to modify said instances from
other threads and commiting the new values.
How's this usually done is SA? Right now I've put a lock on the gui
thread's session and allow other threads to issue a commit on that
main thread if need be. I can't imagine this is the best way to do
it...
Hope you're all having a great weekend,
Christian
2009/2/8 <a...@svilendobrev.com>:
>
> afaiknow these traits are like my own static_types, i.e. descriptors
> holding metadata and applying it to attribute access.
> i have been combining SA with static_type for more than 2 years now,
> since SA 3.0.
>
> The approach i did in the beginning was to replace the object's
> __dict__ by something smart that is static_type-aware. When
> InstrumentionManager framework came into place, i did not find it any
> different, as SA still uses the __dict__ for data access. The
> difference is that now my __dict__ replacement is created once and
> not at every attr.access.
>
> i did suggest one patch about replaceing the SA's
> obj.__dict__.whatever usage with an explicit set of methods (so one
> knows what access to mime), and that was making SA 1-2% faster, but
> it wasn't accepted.
>
> basicaly now there's a thin layer on top of SA, then SA itself, then a
> thick layer underneath managing the data (the fake __dict__).
>
> declarative+traits... u'll end up where i was. dbcook.sf.net is doing
> that - since beginning. and it's switchable on/off.
> It all works well and stable, in project with 250-300 classes,
> although about 15% slower than without it (-:)
>
> The sa2static code:
> svn co
> http://dbcook.svn.sourceforge.net/svnroot/dbcook/trunk/dbcook/usage/static_type/
>
> The static_type itself:
> svn co https://dbcook.svn.sourceforge.net/svnroot/dbcook/static_type
>
> whole dbcook:
> svn co https://dbcook.svn.sourceforge.net/svnroot/dbcook/trunk
>
> i have quite some experience fighting this field, ask if u want.
>
> ciao
> svilen
> www.svilendobrev.com
>
> On Sunday 08 February 2009 01:51:20 cputter wrote:
>> Hi guys and girls,
>>
>> I've recently discovered the joys of using sqlalchemy and would
>> love to using it together with Traits. A few months back there was
>> an attempt to integrate sqlalchemy into traits, though it wasn't
>> really comprehensive in exploiting all of sqlalchemy's potential.
>>
>> So I'm trying to work on that and combine ext.Declarative with
>> traits. The basic idea is to use the DeclarativeMeta type to
>> generate Columns from Traits and pass those on for the Declarative
>> extension to do its magic. This would allow mixing of sqlalchemy
>> attributes and trait attributes in a single class so that we could
>> still make use of all the relational setup sqlalchemy does in any
>> case.
>>
>> Reading through several threads and looking at Elixir's SA
>> integration helped me a bit though I couldn't find any
>> documentation on how to implement the InstrumentationManager
>> interface. I'm assuming this would be essential for letting Traits
>> and SQLAlchemy play well together.
>>
>> There's still a lot of work to do, and I'm not really sure what
>> needs to be done for everything to work properly. Would really
>> appreciate it if someone could help me out.
>>
>>
>> Here's an example of how it's working at the moment, I'll add the
>> actual implementation at the end.
>>
>> #####################################
>> class User(HasTraitsORM):
>> __tablename__ = 'users'
>>
>> id = Column('id', Integer, primary_key=True)
>> name = Str(sqldb=True)
>>
>> def _name_changed(self, old, new):
>> print 'Changed name from %s to %s.' % (old, new)
>>
>> def __repr__(self):
>> return '<User(%s, %s)>' % (self.id, self.name)
>>
>> people = ['John', 'Charls','Steve','Smith','Jane']
>>
>> for per in people:
>> obj = User(name=per)
>> sess = sqlservice.Session()
>> sess.add(obj)
>> sess.commit()
>> sess.close()
>> print obj
>>
>> session = sqlservice.Session()
>> print '\nQuery all users\n'
>> for user in session.query(User).order_by(User.name).all():
>> print user
>>
>> session.close()
>>
>>
>> ############################
>>
>> Which spits out:
>> ###############################
>> Changed name from to John.
>> <User(users.id, )>
>> Changed name from to Charls.
>> <User(users.id, )>
>> Changed name from to Steve.
>> <User(users.id, Steve)>
>> Changed name from to Smith.
>> <User(users.id, Smith)>
>> Changed name from to Jane.
>> <User(users.id, Jane)>
>>
>> Query all users
>>
>> <User(2, Charls)>
>> <User(1, John)>
>>
>> ##############################
>>
>> Which is really strange behaviour. There's obviously something
>> wrong in my implementation of HasTraitsORM but why the different
>> results within the same loop??? Why add only two instances?
>>
>> Totally baffles me.
>>
>> Here's the rest of my code, hope somehow can help me out. It's
>> very messy, I've been hacking at it like crazy with no success :-)
>>
>> Hope you're all having a great weekend.
>> -Chris
>>
>>
>> ##############################
>>
>> # Standard library imports.
>> import logging
>>
>> # Enthought library imports
>> from enthought.preferences.api import Preferences
>> from enthought.traits.api import \
>> HasTraits, MetaHasTraits, Int, Str, Bool, Float, Any,\
>> String, Enum, Python, \
>> on_trait_change, TraitListObject
>>
>> # Package imports
>> import sqlalchemy
>> from sqlalchemy import Column, Integer
>> from sqlalchemy.schema import MetaData
>> from sqlalchemy.orm.interfaces import MapperProperty,
>> InstrumentationManager
>> from sqlalchemy.orm.attributes import get_attribute, set_attribute,
>> is_instrumented
>> from sqlalchemy.orm.collections import InstrumentedList,
>> collection_adapter
>> from sqlalchemy.ext.declarative import _as_declarative
>>
>>
>> # Setup a logger for this module.
>> logger = logging.getLogger(__name__)
>>
>>
>> TRAIT_MAPPING = {
>> Int : 'sqlalchemy.Integer',
>> Str : 'sqlalchemy.Text',
>> Enum : 'sqlalchemy.Text',
>> String : 'sqlalchemy.Text',
>> Float : 'sqlalchemy.Float',
>> Bool : 'sqlalchemy.Boolean',
>> }
>>
>>
>> class HasTraitsORMState(InstrumentationManager):
>> def __init__(self, cls):
>> self.states = {}
>>
>> def instrument_attribute(self, class_, key, attr):
>> pass
>>
>> def install_descriptor(self, class_, key, attr):
>> pass
>>
>> def uninstall_descriptor(self, class_, key, attr):
>> pass
>>
>> def instrument_collection_class(self, class_, key,
>> collection_class):
>> return ObjectCollection
>>
>> def get_instance_dict(self, class_, instance):
>> return instance.__dict__
>>
>> def initialize_instance_dict(self, class_, instance):
>> instance.reset_traits()
>>
>> def initialize_collection(self, key, state, factory):
>> data = factory()
>> return ObjectCollectionAdapter(key, state, data), data
>>
>> def install_state(self, class_, instance, state):
>> self.states[id(instance)] = state
>>
>> def state_getter(self, class_):
>> def find(instance):
>> return self.states[id(instance)]
>> return find
>>
>>
>> class ObjectCollectionAdapter(object):
>> """
>> An adapter for SQLAlchemy for TraitsListObject which is the
>> collection
>> we use for storing instances of classes within attributes
>> of other
>> classes.
>>
>> TODO: Think of a way to get this to behave like a normal
>> traited
>> attribute of objects.
>> At the moment collection attributes are set through the
>> mapper by SA
>> when using relations or backref.
>>
>> """
>> def __init__(self, key, state, collection):
>> self.key = key
>> self.state = state
>> self.collection = collection
>> setattr(collection, '_sa_adapter', self)
>>
>> def unlink(self, data):
>> setattr(data, '_sa_adapter', None)
>>
>> def adapt_like_to_iterable(self, obj):
>> return iter(obj)
>>
>> def append_with_event(self, item, initiator=None):
>> self.collection.append(item, emit=initiator)
>>
>> def append_without_event(self, item):
>> self.collection.append(item, emit=False)
>>
>> def remove_with_event(self, item, initiator=None):
>> self.collection.remove(item, emit=initiator)
>>
>> def remove_without_event(self, item):
>> self.collection.remove(item, emit=False)
>>
>> def clear_with_event(self, initiator=None):
>> for item in list(self):
>> self.remove_with_event(item, initiator)
>>
>> def clear_without_event(self):
>> for item in list(self):
>> self.remove_without_event(item)
>>
>> def __iter__(self):
>> return iter(self.collection)
>>
>> def fire_append_event(self, item, initiator=None):
>> if initiator is not False and item is not None:
>> self.state.get_impl(self.key).fire_append_event
>> (self.state, item,
>>
>> initiator)
>>
>> def fire_remove_event(self, item, initiator=None):
>> if initiator is not False and item is not None:
>> self.state.get_impl(self.key).fire_remove_event
>> (self.state, item,
>>
>> initiator)
>>
>> def fire_pre_remove_event(self, initiator=None):
>> self.state.get_impl(self.key).fire_pre_remove_event
>> (self.state,
>>
>> initiator)
>>
>> class ObjectCollection(TraitListObject):
>> __emulates__ = list
>>
>> def __init__( self, trait, object, name, value ):
>> pass
>>
>> def __init__(self):
>> self.members = list()
>> def append(self, object, emit=None):
>> self.members.append(object)
>> #collection_adapter(self).fire_append_event(object, emit)
>> def remove(self, object, emit=None):
>> #collection_adapter(self).fire_pre_remove_event(object)
>> self.members.remove(object)
>> #collection_adapter(self).fire_remove_event(object, emit)
>> def __getitem__(self, index):
>> return self.members[index]
>> def __iter__(self):
>> return iter(self.members)
>> def __len__(self):
>> return len(self.members)
>> def __repr__(self):
>> if len(self.members) == 0:
>> return '[empty collection]'
>> s = '['
>> for item in self.members[0:max(5, len(self.members))]:
>> s += '%s,' % item
>> s += '...]'
>> return s
>>
>>
>> class DeclarativeMetaTraits(MetaHasTraits):
>> pass
>>
>> # def __new__(cls, classname, bases, dict_):
>> # return MetaHasTraits.__new__(cls, classname, bases, dict_)
>>
>> def __init__(cls, classname, bases, dict_):
>>
>> if '_decl_class_registry' in cls.__dict__:
>> super(DeclarativeMetaTraits, cls).__init__(cls,
>> classname, bases, dict_)
>> return
>>
>> if '__tablename__' not in dict_:
>> setattr(cls, '__tablename__', cls.__name__)
>>
>> # create sql columns from flagged traits
>> if '__class_traits__' in cls.__dict__:
>> traits = cls.__dict__['__class_traits__']
>> for key, trait in traits.iteritems():
>> if getattr( trait, 'sqldb' ):
>> args = getattr( trait, 'col_args' ) or ()
>> kwargs = getattr( trait, 'col_kwargs' ) or {}
>> if 'name' not in kwargs:
>> kwargs['name'] = key
>> if 'type_' not in kwargs:
>> kwargs['type_'] = eval(TRAIT_MAPPING[type
>> (trait.trait_type)])
>>
>> if getattr(trait, 'sqlpk'):
>> kwargs['primary_key'] = True
>>
>> c = Column(*args, **kwargs)
>> dict_[key] = c
>> setattr(cls, key, c)
>>
>> # Pass this along to SA's declarative mapper stuff
>> _as_declarative(cls, classname, dict_)
>>
>> super(DeclarativeMetaTraits, cls).__init__(cls, classname,
>> bases, dict_)
>>
>>
>> class HasTraitsORM(HasTraits):
>> __metaclass__ = DeclarativeMetaTraits
>>
>> # some SA hacking
>> __sa_instrumentation_manager__ = HasTraitsORMState
>>
>> _decl_class_registry = dict()
>> metadata = MetaData()
>>
>> # Any implicit traits added by SQLAlchemy are transient and
>> should not be
>> # copied through .clone_traits(), copy.copy(), or pickling.
>> _ = Python(transient=True)
>>
>>
>> # For some reason we need to override these, our superclass
>> screws things up
>>
>> def __getattr__(self, key):
>> if is_instrumented(self, key):
>> return get_attribute(self, key)
>> else:
>> try:
>> return getattr(self, key)
>> except:
>> pass
>>
>> def __setattr__(self, key, value):
>> if is_instrumented(self, key):
>> set_attribute(self, key, value)
>> else:
>> setattr(self, key, value)
>>
>>
>> #################################################
>>
>>
>>
>
>
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~----------~----~----~----~------~----~------~--~---
