On 09/29/2016 01:38 AM, Warwick Prince wrote:
Hi Mike
I would like a little insight into the session object, and the
declarative_base class.
I have a process running many threads, where each thread may be
connected to potentially a different engine/database. If the database
connection between 2 or more threads is the same, then they will share
the same engine. However, they each have their own MetaData objects.
There is a global sessionmaker() that has no binding at that time.
When each thread creates its OWN session, then it processes mySession =
Session(bind=myThreadsEngine).
The Engines and MetaData part has worked perfectly for years, using
basic queries like Table(’some_table', threadMetaData,
autoload=True).select().execute().fetchall(). etc.
I’ve started to use the ORM more now, and am using the relationships
between the objects. However, I’m hitting and issue that appears to
centre around some shared registry or class variables or something that
is causing a conflict.
I’ve made it so each THREAD has is own Base =
declarative_base(metadata=theSessionsMetaData)
Then, classes are mapped dynamically based on this new Base, and the
columns are autoload’ed. Again, this is working - sometimes. There’s
some random-like problem that mostly means it does not work when I do a
mySession.query(myMappedClassWithRelationships) and I get the following
exception being raised;
so generating new classes in threads can be problematic because the
registry of mappers is essentially global state. Initialization of
mappers against each other, which is where your error here is, is
mutexed and is overall thread-safe, but still, you need to make sure
that all the things that your class needs to be used exist. Here,
somewhere in your program you have a class called SalesDocumentLine, and
that class has not been seen by your Python interpreter yet. That the
problem only happens randomly in threads implies some kind of race
condition which will make this harder to diagnose, but basically that
name has to exist, if your mapping refers to it. You might want to
play with the configure_mappers() call that will cause this
initialization to occur at the point you tell it.
File
"C:\Python27\lib\site-packages\dap-2.1.2-py2.7.egg\dap\db\dbutils.py",
line 323, in DAPDB_SetColumns
query = session.query(mappedClass).filter_by(**whereCriteria)
File "build\bdist.win32\egg\sqlalchemy\orm\session.py", line 1260, in
query
return self._query_cls(entities, self, **kwargs)
File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 110, in
__init__
self._set_entities(entities)
File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 120, in
_set_entities
self._set_entity_selectables(self._entities)
File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 150, in
_set_entity_selectables
ent.setup_entity(*d[entity])
File "build\bdist.win32\egg\sqlalchemy\orm\query.py", line 3446, in
setup_entity
self._with_polymorphic = ext_info.with_polymorphic_mappers
File "build\bdist.win32\egg\sqlalchemy\util\langhelpers.py", line 754,
in __get__
obj.__dict__[self.__name__] = result = self.fget(obj)
File "build\bdist.win32\egg\sqlalchemy\orm\mapper.py", line 1891, in
_with_polymorphic_mappers
configure_mappers()
File "build\bdist.win32\egg\sqlalchemy\orm\mapper.py", line 2768, in
configure_mappers
mapper._post_configure_properties()
File "build\bdist.win32\egg\sqlalchemy\orm\mapper.py", line 1708, in
_post_configure_properties
prop.init()
File "build\bdist.win32\egg\sqlalchemy\orm\interfaces.py", line 183,
in init
self.do_init()
File "build\bdist.win32\egg\sqlalchemy\orm\relationships.py", line
1628, in do_init
self._process_dependent_arguments()
File "build\bdist.win32\egg\sqlalchemy\orm\relationships.py", line
1653, in _process_dependent_arguments
setattr(self, attr, attr_value())
File
"build\bdist.win32\egg\sqlalchemy\ext\declarative\clsregistry.py", line
293, in __call__
(self.prop.parent, self.arg, n.args[0], self.cls)
InvalidRequestError: When initializing mapper
Mapper|SalesDocument|rm_dt_documents, expression
'SalesDocumentLine.parentID==SalesDocument.id' failed to locate a name
("name 'SalesDocumentLine' is not defined"). If this is a class name,
consider adding this relationship() to the <class
'dap.db.dbmanager.SalesDocument'> class after both dependent classes
have been defined.
I understand what this is trying to tell me, however, the classes ARE
defined. Sometimes the code works perfectly, but mostly not. If I have
ONE Thread working and then start up another using exactly the same
code, then it will probably NOT work but more importantly, the one that
WAS working then dies with the same error. Clearly something somewhere
is shared - I just can’t find out what it is, or how I can separate the
code further.
In summary;
one global sessionmaker()
global Session=sessionmaker()
each thread (for the example here) shares an Engine
each thread has it’s OWN session from mySession = Session(bind=e)
each thread has it’s own Base created from
declarative_base(metadata=threadsMetaData)
I’m declaring two classes in this example. SalesDocument and
SalesDocumentLine. The relationships are set up and ‘can’ work on occasion.
In that error, where exactly are they not ‘defined’. I’ve looked in
Base.decl_class_registry and both those names are there! Where else do
they need to be to be considered ‘declared'?
Any pointers as to the error of my ways would be most appreciated. :-)
Cheers
Warwick
--
You received this message because you are subscribed to the Google
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to sqlalchemy+unsubscr...@googlegroups.com
<mailto:sqlalchemy+unsubscr...@googlegroups.com>.
To post to this group, send email to sqlalchemy@googlegroups.com
<mailto:sqlalchemy@googlegroups.com>.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
