Hi Mike
I don’t need support very often, but when I do I know I can count on your clear
and concise responses to save the day.
Everything was SO close - all I needed to do was take my dynamically created
classes (SalesDocumentLine for example) and force them into the general
globals() and hey-presto - all fixed. ‘Knowing’ that the environment had a
global scope, and that it was thread safe eliminated so many other possible red
herrings. I could then focus on why my classes could not be ‘seen’.
Thanks once again. Best support on the interwebs. ;-)
> On 29 Sep 2016, at 11:48 PM, Mike Bayer <mike...@zzzcomputing.com> wrote:
>
>
>
> On 09/29/2016 01:38 AM, Warwick Prince wrote:
>> Hi Mike
>>
>> I would like a little insight into the session object, and the
>> declarative_base class.
>>
>> I have a process running many threads, where each thread may be
>> connected to potentially a different engine/database. If the database
>> connection between 2 or more threads is the same, then they will share
>> the same engine. However, they each have their own MetaData objects.
>>
>> There is a global sessionmaker() that has no binding at that time.
>> When each thread creates its OWN session, then it processes mySession =
>> Session(bind=myThreadsEngine).
>>
>> The Engines and MetaData part has worked perfectly for years, using
>> basic queries like Table(’some_table', threadMetaData,
>> autoload=True).select().execute().fetchall(). etc.
>>
>> I’ve started to use the ORM more now, and am using the relationships
>> between the objects. However, I’m hitting and issue that appears to
>> centre around some shared registry or class variables or something that
>> is causing a conflict.
>>
>> I’ve made it so each THREAD has is own Base =
>> declarative_base(metadata=theSessionsMetaData)
>>
>> Then, classes are mapped dynamically based on this new Base, and the
>> columns are autoload’ed. Again, this is working - sometimes. There’s
>> some random-like problem that mostly means it does not work when I do a
>> mySession.query(myMappedClassWithRelationships) and I get the following
>> exception being raised;
>
> so generating new classes in threads can be problematic because the registry
> of mappers is essentially global state. Initialization of mappers against
> each other, which is where your error here is, is mutexed and is overall
> thread-safe, but still, you need to make sure that all the things that your
> class needs to be used exist. Here, somewhere in your program you have a
> class called SalesDocumentLine, and that class has not been seen by your
> Python interpreter yet. That the problem only happens randomly in threads
> implies some kind of race condition which will make this harder to diagnose,
> but basically that name has to exist, if your mapping refers to it. You
> might want to play with the configure_mappers() call that will cause this
> initialization to occur at the point you tell it.
>
>
>
>
>>
>> File
>> "C:\Python27\lib\site-packages\dap-2.1.2-py2.7.egg\dap\db\dbutils.py",
>> line 323, in DAPDB_SetColumns
>> query = session.query(mappedClass).filter_by(**whereCriteria)
>> File "build\bdist.win32\egg\sqlalchemy\orm\session.py", line 1260, in
>> query
>> return self._query_cls(entities, self, **kwargs)
>> File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 110, in
>> __init__
>> self._set_entities(entities)
>> File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 120, in
>> _set_entities
>> self._set_entity_selectables(self._entities)
>> File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 150, in
>> _set_entity_selectables
>> ent.setup_entity(*d[entity])
>> File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 3446, in
>> setup_entity
>> self._with_polymorphic = ext_info.with_polymorphic_mappers
>> File "build\bdist.win32\egg\sqlalchemy\util\langhelpers.py", line 754,
>> in __get__
>> obj.__dict__[self.__name__] = result = self.fget(obj)
>> File "build\bdist.win32\egg\sqlalchemy\orm\mapper.py", line 1891, in
>> _with_polymorphic_mappers
>> configure_mappers()
>> File "build\bdist.win32\egg\sqlalchemy\orm\mapper.py", line 2768, in
>> configure_mappers
>> mapper._post_configure_properties()
>> File "build\bdist.win32\egg\sqlalchemy\orm\mapper.py", line 1708, in
>> _post_configure_properties
>> prop.init()
>> File "build\bdist.win32\egg\sqlalchemy\orm\interfaces.py", line 183,
>> in init
>> self.do_init()
>> File "build\bdist.win32\egg\sqlalchemy\orm\relationships.py", line
>> 1628, in do_init
>> self._process_dependent_arguments()
>> File "build\bdist.win32\egg\sqlalchemy\orm\relationships.py", line
>> 1653, in _process_dependent_arguments
>> setattr(self, attr, attr_value())
>> File
>> "build\bdist.win32\egg\sqlalchemy\ext\declarative\clsregistry.py", line
>> 293, in __call__
>> (self.prop.parent, self.arg, n.args[0], self.cls)
>> InvalidRequestError: When initializing mapper
>> Mapper|SalesDocument|rm_dt_documents, expression
>> 'SalesDocumentLine.parentID==SalesDocument.id' failed to locate a name
>> ("name 'SalesDocumentLine' is not defined"). If this is a class name,
>> consider adding this relationship() to the <class
>> 'dap.db.dbmanager.SalesDocument'> class after both dependent classes
>> have been defined.
>>
>> I understand what this is trying to tell me, however, the classes ARE
>> defined. Sometimes the code works perfectly, but mostly not. If I have
>> ONE Thread working and then start up another using exactly the same
>> code, then it will probably NOT work but more importantly, the one that
>> WAS working then dies with the same error. Clearly something somewhere
>> is shared - I just can’t find out what it is, or how I can separate the
>> code further.
>>
>> In summary;
>>
>> one global sessionmaker()
>> global Session=sessionmaker()
>> each thread (for the example here) shares an Engine
>> each thread has it’s OWN session from mySession = Session(bind=e)
>> each thread has it’s own Base created from
>> declarative_base(metadata=threadsMetaData)
>> I’m declaring two classes in this example. SalesDocument and
>> SalesDocumentLine. The relationships are set up and ‘can’ work on occasion.
>>
>> In that error, where exactly are they not ‘defined’. I’ve looked in
>> Base.decl_class_registry and both those names are there! Where else do
>> they need to be to be considered ‘declared'?
>>
>> Any pointers as to the error of my ways would be most appreciated. :-)
>>
>> Cheers
>> Warwick
>>
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "sqlalchemy" group.
>> To unsubscribe from this group and stop receiving emails from it, send
>> an email to sqlalchemy+unsubscr...@googlegroups.com
>> <mailto:sqlalchemy+unsubscr...@googlegroups.com>.
>> To post to this group, send email to sqlalchemy@googlegroups.com
>> <mailto:sqlalchemy@googlegroups.com>.
>> Visit this group at https://groups.google.com/group/sqlalchemy.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
