On Sunday 22 March 2009 21:17:15 Christiaan Putter wrote:
> Hi Svilen,
>
> Setting up tests is a very good idea, I'll start on some unit
> testing immediately. Definitely the best way to insure behaviour
> remains constant as you're working on the implementation, was just
> to lazy to do so up till now.
i'm sure if u do the tests first trying to cover as many use cases as
u can, u may even better understand what u really need... TDD of
sorts as first feedback to requirements-analisys.
> Speaking of laziness, I've noticed that setting lazy=False on a
> relation does not effect refresh operations on the relation, it
> only loads on reconstruction. Having cascade='all' doesn't change
> this either. Is their some setting that I'm missing?
dunno.
> Another little issue I'm trying to get fixed out is extending a
> class's attributes at run time, let's call it Parent. The need for
> this is allowing users to add custom fields (in a seperate table as
> the parent) at run time to Parent. My approach is letting the user
> set up a new class (ChildX), which inherits Base (not parent),
> with the attributes and methods that compute said attributes during
> run time. Then I compile that using exec, and and do some magic in
> its metaclass. Basically I add a relation to Parent with backref
> to ChildX and uselist=False on both sides. Enthought then allows
> me to add new traits to the Parent class that delegate their value
> the ChildX instance through the backref on Parent. This is
> basically just mapping the Parent class over several tables, with
> the ability to extend the class on the fly.
> It's mostly working. One problem though is that on reconstructing
> a Parent instance (or creating a new one) I have to have a
> mechanism that checks if all the Child classes have instances
> related to this record of Parent. Not a gig deal though.
>
> Is their a better way to do this? I'm doing some really ugly
> hacking to the Parent's mapper at run time, seeing as Mapper
> doesn't have a delete_property method.
>
> Should I try using normal inheritance instead? Will I be able to
> access ChildX.x from ChildY for instance? And what about changing
> ChildX's definition and running exec on it again?
i don't really get the reason of why u do all this.
wanna extend existing class runtime or want to add new class?
IMO just subclassing on-the-fly would be easier.
svil
> Perhaps someone has done something similar before and can give me
> some pointers.
>
> Thanks for all the advice up till now, it's been really useful.
>
> Have a great day,
> Christian
>
> 2009/3/22 <a...@svilendobrev.com>:
> > good on yer!
> > one thing that i sorta failed is making a clean set of tests for
> > this. if u aren't too far in the usage, save some major future
> > headaches and do yourself a favour, make such test set. All the
> > things u've tried has to be (simple) testcases - it will be the
> > _spec_ of how the thing works / what u want from it. Anyone else
> > on the same path could also use it then ;-)
> >
> > i hope that once u do that u'll get an idea of where exactly u
> > have failed and fix it.
> >
> > as of expiration, it also did me some trouble. i have not stepped
> > on that session-gone one but i can see how it happens. The
> > (expected) life-times of SA-touched things aren't that obvious,
> > and aren't documented. e.g. Objects living longer than session or
> > in an extreme case, the mappers, might be troublesome.
> >
> > what i can suggest is, put debug statements and run with
> > echo=True and watch what goes on behind and how SA-activities
> > intertwist with traits' /your ones.
> >
> > ciao
> > svilen
> >
> > On Sunday 22 March 2009 04:00:27 Christiaan Putter wrote:
> >> Hi svilen,
> >>
> >> Thanks for your advice, going through your code helped a lot in
> >> understanding how SA works.
> >>
> >> I've gotten traited classes to behave like classes generated by
> >> SA's declarative extension. After a lot of stepping through
> >> code I realised the main problem was that SA removes values from
> >> an instance __dict__ once they have been expired. Traits
> >> smartly (or not so smartly) picks this up and on the next access
> >> to those attributes and returns their default values. Whereas
> >> SA would normally refresh the values from SQL. This caused some
> >> serious headaches, and really strange behaviour.
> >>
> >> My solution was to the overwrite the class's __getattribute__
> >> with:
> >>
> >> def __getattribute__(self, key):
> >> try:
> >> dict = super(HasTraitsORM,
> >> self).__getattribute__('__dict__')
> >>
> >> if ('_sa_state' in dict):
> >> state = dict['_sa_state']
> >> if key in state.manager.keys():
> >> return state.get_impl(key).get(state)
> >>
> >> except Exception, e:
> >> print "Exception in SA getattr for %s: %s" % (key,
> >> e) pass
> >>
> >> return super(HasTraitsORM, self).__getattribute__(key)
> >>
> >>
> >> Which tries to first get an attribute from the instance's state
> >> (which will issue SQL if needed), and if that fails pass the
> >> request along to its super class where Traits will do it's magic
> >> if needed.
> >>
> >> This seems to work in the tests if been trying. Most of the
> >> work is done by SA declarative, and I've just added a thin layer
> >> on top to get it to behave well with Traited classes and map
> >> traits to SA columns.
> >>
> >> One issue I'm still having though is that after commits all
> >> attributes get expired. If you close the session the instance
> >> was attached to the data you just commited can't be accessed any
> >> more. This is also the behaviour of plain declarative classes.
> >> This is rather unintuitive I believe since once you've comitted
> >> an instance and closed the session you can't use the data you
> >> just set any more. In my case this means records being displayed
> >> in a gui get corrupted.
> >>
> >> Though what's strange is that if you access the attributes after
> >> the commit and before you close the session, and only then close
> >> the session, the values remain in the instance's __dict__ and
> >> the class behaves normally again. Is this intended for some
> >> reason? Is there a workaround? Personally I don't think quantum
> >> physics should apply to the bits of my programs and thus merely
> >> observing them should not change their behaviour. From going
> >> through the source I found a 'dont_expire_missing' attribute on
> >> attribute implementations, though this does unfortunately not do
> >> what I'd like.
> >>
> >> So what I'm doing now is keeping a session open on the gui's
> >> thread. Which causes some problems when trying to modify said
> >> instances from other threads and commiting the new values.
> >>
> >> How's this usually done is SA? Right now I've put a lock on the
> >> gui thread's session and allow other threads to issue a commit
> >> on that main thread if need be. I can't imagine this is the
> >> best way to do it...
> >>
> >> Hope you're all having a great weekend,
> >>
> >> Christian
> >>
> >> 2009/2/8 <a...@svilendobrev.com>:
> >> > afaiknow these traits are like my own static_types, i.e.
> >> > descriptors holding metadata and applying it to attribute
> >> > access. i have been combining SA with static_type for more
> >> > than 2 years now, since SA 3.0.
> >> >
> >> > The approach i did in the beginning was to replace the
> >> > object's __dict__ by something smart that is
> >> > static_type-aware. When InstrumentionManager framework came
> >> > into place, i did not find it any different, as SA still uses
> >> > the __dict__ for data access. The difference is that now my
> >> > __dict__ replacement is created once and not at every
> >> > attr.access.
> >> >
> >> > i did suggest one patch about replaceing the SA's
> >> > obj.__dict__.whatever usage with an explicit set of methods
> >> > (so one knows what access to mime), and that was making SA
> >> > 1-2% faster, but it wasn't accepted.
> >> >
> >> > basicaly now there's a thin layer on top of SA, then SA
> >> > itself, then a thick layer underneath managing the data (the
> >> > fake __dict__).
> >> >
> >> > declarative+traits... u'll end up where i was. dbcook.sf.net
> >> > is doing that - since beginning. and it's switchable on/off.
> >> > It all works well and stable, in project with 250-300 classes,
> >> > although about 15% slower than without it (-:)
> >> >
> >> > The sa2static code:
> >> > svn co
> >> > http://dbcook.svn.sourceforge.net/svnroot/dbcook/trunk/dbcook/
> >> >usa ge/static_type/
> >> >
> >> > The static_type itself:
> >> > svn co
> >> > https://dbcook.svn.sourceforge.net/svnroot/dbcook/static_type
> >> >
> >> > whole dbcook:
> >> > svn co https://dbcook.svn.sourceforge.net/svnroot/dbcook/trunk
> >> >
> >> > i have quite some experience fighting this field, ask if u
> >> > want.
> >> >
> >> > ciao
> >> > svilen
> >> > www.svilendobrev.com
> >> >
> >> > On Sunday 08 February 2009 01:51:20 cputter wrote:
> >> >> Hi guys and girls,
> >> >>
> >> >> I've recently discovered the joys of using sqlalchemy and
> >> >> would love to using it together with Traits. A few months
> >> >> back there was an attempt to integrate sqlalchemy into
> >> >> traits, though it wasn't really comprehensive in exploiting
> >> >> all of sqlalchemy's potential.
> >> >>
> >> >> So I'm trying to work on that and combine ext.Declarative
> >> >> with traits. The basic idea is to use the DeclarativeMeta
> >> >> type to generate Columns from Traits and pass those on for
> >> >> the Declarative extension to do its magic. This would allow
> >> >> mixing of sqlalchemy attributes and trait attributes in a
> >> >> single class so that we could still make use of all the
> >> >> relational setup sqlalchemy does in any case.
> >> >>
> >> >> Reading through several threads and looking at Elixir's SA
> >> >> integration helped me a bit though I couldn't find any
> >> >> documentation on how to implement the InstrumentationManager
> >> >> interface. I'm assuming this would be essential for letting
> >> >> Traits and SQLAlchemy play well together.
> >> >>
> >> >> There's still a lot of work to do, and I'm not really sure
> >> >> what needs to be done for everything to work properly. Would
> >> >> really appreciate it if someone could help me out.
> >> >>
> >> >>
> >> >> Here's an example of how it's working at the moment, I'll add
> >> >> the actual implementation at the end.
> >> >>
> >> >> #####################################
> >> >> class User(HasTraitsORM):
> >> >> __tablename__ = 'users'
> >> >>
> >> >> id = Column('id', Integer, primary_key=True)
> >> >> name = Str(sqldb=True)
> >> >>
> >> >> def _name_changed(self, old, new):
> >> >> print 'Changed name from %s to %s.' % (old, new)
> >> >>
> >> >> def __repr__(self):
> >> >> return '<User(%s, %s)>' % (self.id, self.name)
> >> >>
> >> >> people = ['John', 'Charls','Steve','Smith','Jane']
> >> >>
> >> >> for per in people:
> >> >> obj = User(name=per)
> >> >> sess = sqlservice.Session()
> >> >> sess.add(obj)
> >> >> sess.commit()
> >> >> sess.close()
> >> >> print obj
> >> >>
> >> >> session = sqlservice.Session()
> >> >> print '\nQuery all users\n'
> >> >> for user in session.query(User).order_by(User.name).all():
> >> >> print user
> >> >>
> >> >> session.close()
> >> >>
> >> >>
> >> >> ############################
> >> >>
> >> >> Which spits out:
> >> >> ###############################
> >> >> Changed name from to John.
> >> >> <User(users.id, )>
> >> >> Changed name from to Charls.
> >> >> <User(users.id, )>
> >> >> Changed name from to Steve.
> >> >> <User(users.id, Steve)>
> >> >> Changed name from to Smith.
> >> >> <User(users.id, Smith)>
> >> >> Changed name from to Jane.
> >> >> <User(users.id, Jane)>
> >> >>
> >> >> Query all users
> >> >>
> >> >> <User(2, Charls)>
> >> >> <User(1, John)>
> >> >>
> >> >> ##############################
> >> >>
> >> >> Which is really strange behaviour. There's obviously
> >> >> something wrong in my implementation of HasTraitsORM but why
> >> >> the different results within the same loop??? Why add only
> >> >> two instances?
> >> >>
> >> >> Totally baffles me.
> >> >>
> >> >> Here's the rest of my code, hope somehow can help me out.
> >> >> It's very messy, I've been hacking at it like crazy with no
> >> >> success
> >> >>
> >> >> :-)
> >> >>
> >> >> Hope you're all having a great weekend.
> >> >> -Chris
> >> >>
> >> >>
> >> >> ##############################
> >> >>
> >> >> # Standard library imports.
> >> >> import logging
> >> >>
> >> >> # Enthought library imports
> >> >> from enthought.preferences.api import Preferences
> >> >> from enthought.traits.api import \
> >> >> HasTraits, MetaHasTraits, Int, Str, Bool, Float, Any,\
> >> >> String, Enum, Python, \
> >> >> on_trait_change, TraitListObject
> >> >>
> >> >> # Package imports
> >> >> import sqlalchemy
> >> >> from sqlalchemy import Column, Integer
> >> >> from sqlalchemy.schema import MetaData
> >> >> from sqlalchemy.orm.interfaces import MapperProperty,
> >> >> InstrumentationManager
> >> >> from sqlalchemy.orm.attributes import get_attribute,
> >> >> set_attribute, is_instrumented
> >> >> from sqlalchemy.orm.collections import InstrumentedList,
> >> >> collection_adapter
> >> >> from sqlalchemy.ext.declarative import _as_declarative
> >> >>
> >> >>
> >> >> # Setup a logger for this module.
> >> >> logger = logging.getLogger(__name__)
> >> >>
> >> >>
> >> >> TRAIT_MAPPING = {
> >> >> Int : 'sqlalchemy.Integer',
> >> >> Str : 'sqlalchemy.Text',
> >> >> Enum : 'sqlalchemy.Text',
> >> >> String : 'sqlalchemy.Text',
> >> >> Float : 'sqlalchemy.Float',
> >> >> Bool : 'sqlalchemy.Boolean',
> >> >> }
> >> >>
> >> >>
> >> >> class HasTraitsORMState(InstrumentationManager):
> >> >> def __init__(self, cls):
> >> >> self.states = {}
> >> >>
> >> >> def instrument_attribute(self, class_, key, attr):
> >> >> pass
> >> >>
> >> >> def install_descriptor(self, class_, key, attr):
> >> >> pass
> >> >>
> >> >> def uninstall_descriptor(self, class_, key, attr):
> >> >> pass
> >> >>
> >> >> def instrument_collection_class(self, class_, key,
> >> >> collection_class):
> >> >> return ObjectCollection
> >> >>
> >> >> def get_instance_dict(self, class_, instance):
> >> >> return instance.__dict__
> >> >>
> >> >> def initialize_instance_dict(self, class_, instance):
> >> >> instance.reset_traits()
> >> >>
> >> >> def initialize_collection(self, key, state, factory):
> >> >> data = factory()
> >> >> return ObjectCollectionAdapter(key, state, data),
> >> >> data
> >> >>
> >> >> def install_state(self, class_, instance, state):
> >> >> self.states[id(instance)] = state
> >> >>
> >> >> def state_getter(self, class_):
> >> >> def find(instance):
> >> >> return self.states[id(instance)]
> >> >> return find
> >> >>
> >> >>
> >> >> class ObjectCollectionAdapter(object):
> >> >> """
> >> >> An adapter for SQLAlchemy for TraitsListObject which
> >> >> is the collection
> >> >> we use for storing instances of classes within
> >> >> attributes of other
> >> >> classes.
> >> >>
> >> >> TODO: Think of a way to get this to behave like a
> >> >> normal traited
> >> >> attribute of objects.
> >> >> At the moment collection attributes are set through
> >> >> the mapper by SA
> >> >> when using relations or backref.
> >> >>
> >> >> """
> >> >> def __init__(self, key, state, collection):
> >> >> self.key = key
> >> >> self.state = state
> >> >> self.collection = collection
> >> >> setattr(collection, '_sa_adapter', self)
> >> >>
> >> >> def unlink(self, data):
> >> >> setattr(data, '_sa_adapter', None)
> >> >>
> >> >> def adapt_like_to_iterable(self, obj):
> >> >> return iter(obj)
> >> >>
> >> >> def append_with_event(self, item, initiator=None):
> >> >> self.collection.append(item, emit=initiator)
> >> >>
> >> >> def append_without_event(self, item):
> >> >> self.collection.append(item, emit=False)
> >> >>
> >> >> def remove_with_event(self, item, initiator=None):
> >> >> self.collection.remove(item, emit=initiator)
> >> >>
> >> >> def remove_without_event(self, item):
> >> >> self.collection.remove(item, emit=False)
> >> >>
> >> >> def clear_with_event(self, initiator=None):
> >> >> for item in list(self):
> >> >> self.remove_with_event(item, initiator)
> >> >>
> >> >> def clear_without_event(self):
> >> >> for item in list(self):
> >> >> self.remove_without_event(item)
> >> >>
> >> >> def __iter__(self):
> >> >> return iter(self.collection)
> >> >>
> >> >> def fire_append_event(self, item, initiator=None):
> >> >> if initiator is not False and item is not None:
> >> >> self.state.get_impl(self.key).fire_append_event
> >> >> (self.state, item,
> >> >>
> >> >> initiator)
> >> >>
> >> >> def fire_remove_event(self, item, initiator=None):
> >> >> if initiator is not False and item is not None:
> >> >> self.state.get_impl(self.key).fire_remove_event
> >> >> (self.state, item,
> >> >>
> >> >> initiator)
> >> >>
> >> >> def fire_pre_remove_event(self, initiator=None):
> >> >> self.state.get_impl(self.key).fire_pre_remove_event
> >> >> (self.state,
> >> >>
> >> >> initiator)
> >> >>
> >> >> class ObjectCollection(TraitListObject):
> >> >> __emulates__ = list
> >> >>
> >> >> def __init__( self, trait, object, name, value ):
> >> >> pass
> >> >>
> >> >> def __init__(self):
> >> >> self.members = list()
> >> >> def append(self, object, emit=None):
> >> >> self.members.append(object)
> >> >> #collection_adapter(self).fire_append_event(object,
> >> >> emit) def remove(self, object, emit=None):
> >> >>
> >> >> #collection_adapter(self).fire_pre_remove_event(object)
> >> >> self.members.remove(object)
> >> >> #collection_adapter(self).fire_remove_event(object,
> >> >> emit) def __getitem__(self, index):
> >> >> return self.members[index]
> >> >> def __iter__(self):
> >> >> return iter(self.members)
> >> >> def __len__(self):
> >> >> return len(self.members)
> >> >> def __repr__(self):
> >> >> if len(self.members) == 0:
> >> >> return '[empty collection]'
> >> >> s = '['
> >> >> for item in self.members[0:max(5,
> >> >> len(self.members))]: s += '%s,' % item
> >> >> s += '...]'
> >> >> return s
> >> >>
> >> >>
> >> >> class DeclarativeMetaTraits(MetaHasTraits):
> >> >> pass
> >> >>
> >> >> # def __new__(cls, classname, bases, dict_):
> >> >> # return MetaHasTraits.__new__(cls, classname, bases,
> >> >> dict_)
> >> >>
> >> >> def __init__(cls, classname, bases, dict_):
> >> >>
> >> >> if '_decl_class_registry' in cls.__dict__:
> >> >> super(DeclarativeMetaTraits, cls).__init__(cls,
> >> >> classname, bases, dict_)
> >> >> return
> >> >>
> >> >> if '__tablename__' not in dict_:
> >> >> setattr(cls, '__tablename__', cls.__name__)
> >> >>
> >> >> # create sql columns from flagged traits
> >> >> if '__class_traits__' in cls.__dict__:
> >> >> traits = cls.__dict__['__class_traits__']
> >> >> for key, trait in traits.iteritems():
> >> >> if getattr( trait, 'sqldb' ):
> >> >> args = getattr( trait, 'col_args' ) or ()
> >> >> kwargs = getattr( trait, 'col_kwargs' )
> >> >> or {} if 'name' not in kwargs:
> >> >> kwargs['name'] = key
> >> >> if 'type_' not in kwargs:
> >> >> kwargs['type_'] =
> >> >> eval(TRAIT_MAPPING[type (trait.trait_type)])
> >> >>
> >> >> if getattr(trait, 'sqlpk'):
> >> >> kwargs['primary_key'] = True
> >> >>
> >> >> c = Column(*args, **kwargs)
> >> >> dict_[key] = c
> >> >> setattr(cls, key, c)
> >> >>
> >> >> # Pass this along to SA's declarative mapper stuff
> >> >> _as_declarative(cls, classname, dict_)
> >> >>
> >> >> super(DeclarativeMetaTraits, cls).__init__(cls,
> >> >> classname, bases, dict_)
> >> >>
> >> >>
> >> >> class HasTraitsORM(HasTraits):
> >> >> __metaclass__ = DeclarativeMetaTraits
> >> >>
> >> >> # some SA hacking
> >> >> __sa_instrumentation_manager__ = HasTraitsORMState
> >> >>
> >> >> _decl_class_registry = dict()
> >> >> metadata = MetaData()
> >> >>
> >> >> # Any implicit traits added by SQLAlchemy are transient
> >> >> and should not be
> >> >> # copied through .clone_traits(), copy.copy(), or
> >> >> pickling. _ = Python(transient=True)
> >> >>
> >> >>
> >> >> # For some reason we need to override these, our
> >> >> superclass screws things up
> >> >>
> >> >> def __getattr__(self, key):
> >> >> if is_instrumented(self, key):
> >> >> return get_attribute(self, key)
> >> >> else:
> >> >> try:
> >> >> return getattr(self, key)
> >> >> except:
> >> >> pass
> >> >>
> >> >> def __setattr__(self, key, value):
> >> >> if is_instrumented(self, key):
> >> >> set_attribute(self, key, value)
> >> >> else:
> >> >> setattr(self, key, value)
> >> >>
> >> >>
> >> >> #################################################
>
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~----------~----~----~----~------~----~------~--~---
