Re: [sqlalchemy] Custom Handler for "No such polymorphic_identity"
Here's the POC:
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.ext.declarative import declared_attr
Base = declarative_base()
class A(Base):
__tablename__ = 'a'
id = Column(Integer, primary_key=True)
data = Column(String)
type = Column(String)
__mapper_args__ = {
"polymorphic_on": type,
"polymorphic_identity": "a"
}
class MyRegistry(dict):
def _create_class(self, key):
if key == "b":
class B(A):
__mapper_args__ = {
"polymorphic_identity": "b"
}
return B.__mapper__
else:
raise KeyError(key)
def __missing__(self, key):
self[key] = val = self._create_class(key)
return val
A.__mapper__.polymorphic_map = MyRegistry(**A.__mapper__.polymorphic_map)
e = create_engine("sqlite://", echo=True)
with e.connect() as conn:
conn.execute(
"CREATE TABLE a (id INTEGER PRIMARY KEY, data VARCHAR, "
"type VARCHAR)")
conn.execute(
"INSERT INTO a (id, data, type) "
"VALUES (1, 'some object', 'b')")
s = Session(e)
print(s.query(A).all())
On Tue, Mar 12, 2019 at 9:38 AM Mike Bayer wrote:
>
> On Tue, Mar 12, 2019 at 2:05 AM Tolstov Sergey wrote:
> >
> > Addition.
> > If i create all possible classes i may get a low perfomance (rel to root
> > class)
>
> I understand the startup time issue, although again if your
> application runs long enough, all that time will ultimately have been
> taken.Once you do have all the classes loaded, I'm not sure how
> that affects performance going forward unless your server is running
> out of memory as a result. There should be no impact on performance
> if your application has 3 or 300 classes and mappers loaded in memory
> provided you have enough memory and provided these classes aren't
> spinning off some kind of background work by existing.
>
> Anyway, this is risky, because if you have a multithreaded /
> multi-green-thread application running lots of queries, and all of
> them are digging into your registry, creating classes and new mapper()
> objects on the fly, how are you guarding against two queries trying to
> access the same mapper() at the same time and both trying to create
> it? Are you mutexing around the whole get() operation or otherwise
> gating the production of the new class + mapper() (hurts performance
> also)? Additionally, a mapper needs to have the configure step
> called, which uses a mutex so should be threadsafe.
>
> Beyond that, this doesn't work from a mapping perspective, because the
> root mapper needs to know about the attributes on the subclass in
> order to load correctly. I can give you a POC that works only if you
> use single table inheritance and the subclasses have no columns on
> them, which is quite limited, see below. Otherwise, the loading
> system does not have the information it needs if new classes are added
> on the fly in the middle of the loading process.
>
> Bigger issue is that if you've built a datamodel that takes too long
> to start up because it has many hundreds of classes that usually arent
> needed, this may be a sign of a bigger architectural problem. If
> OTOH these polymorphic classes are generated anonymously, e.g. they
> really aren't business objects but just data containers, then that's
> not really how the polymorphic inheritance feature was meant to be
> used.
>
> >
> > --
> > SQLAlchemy -
> > The Python SQL Toolkit and Object Relational Mapper
> >
> > http://www.sqlalchemy.org/
> >
> > To post example code, please provide an MCVE: Minimal, Complete, and
> > Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> > description.
> > ---
> > You received this message because you are subscribed to the Google Groups
> > "sqlalchemy" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to sqlalchemy+unsubscr...@googlegroups.com.
> > To post to this group, send email to sqlalchemy@googlegroups.com.
> > Visit this group at https://groups.google.com/group/sqlalchemy.
> > For more options, visit https://groups.google.com/d/optout.
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable
Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
Re: [sqlalchemy] Custom Handler for "No such polymorphic_identity"
On Tue, Mar 12, 2019 at 2:05 AM Tolstov Sergey wrote: > > Addition. > If i create all possible classes i may get a low perfomance (rel to root > class) I understand the startup time issue, although again if your application runs long enough, all that time will ultimately have been taken.Once you do have all the classes loaded, I'm not sure how that affects performance going forward unless your server is running out of memory as a result. There should be no impact on performance if your application has 3 or 300 classes and mappers loaded in memory provided you have enough memory and provided these classes aren't spinning off some kind of background work by existing. Anyway, this is risky, because if you have a multithreaded / multi-green-thread application running lots of queries, and all of them are digging into your registry, creating classes and new mapper() objects on the fly, how are you guarding against two queries trying to access the same mapper() at the same time and both trying to create it? Are you mutexing around the whole get() operation or otherwise gating the production of the new class + mapper() (hurts performance also)? Additionally, a mapper needs to have the configure step called, which uses a mutex so should be threadsafe. Beyond that, this doesn't work from a mapping perspective, because the root mapper needs to know about the attributes on the subclass in order to load correctly. I can give you a POC that works only if you use single table inheritance and the subclasses have no columns on them, which is quite limited, see below. Otherwise, the loading system does not have the information it needs if new classes are added on the fly in the middle of the loading process. Bigger issue is that if you've built a datamodel that takes too long to start up because it has many hundreds of classes that usually arent needed, this may be a sign of a bigger architectural problem. If OTOH these polymorphic classes are generated anonymously, e.g. they really aren't business objects but just data containers, then that's not really how the polymorphic inheritance feature was meant to be used. > > -- > SQLAlchemy - > The Python SQL Toolkit and Object Relational Mapper > > http://www.sqlalchemy.org/ > > To post example code, please provide an MCVE: Minimal, Complete, and > Verifiable Example. See http://stackoverflow.com/help/mcve for a full > description. > --- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy+unsubscr...@googlegroups.com. > To post to this group, send email to sqlalchemy@googlegroups.com. > Visit this group at https://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
Re: [sqlalchemy] Custom Handler for "No such polymorphic_identity"
Addition. If i create all possible classes i may get a low perfomance (rel to root class) -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
Re: [sqlalchemy] Custom Handler for "No such polymorphic_identity"
Thanks for answer. Problem is too much loading time (~40 seconds) if i use standart declaration. To resolve this, i define classes when they called (write class factory), but on factory i cannot redefine func to get relationship value if return non-main class (such as child) Realization is def handle_assertion_error(self, err_info): result = re.match( "^No such polymorphic_identity '(.*)' is defined$", str(err_info)) if result.groups() is None: raise DBException(str(err_info)) for class_name in class_factory.all_classes_list: if (class_name.lower() == result.groups()[0]): getattr(class_factory, class_name) return With child of *sqlalchemy.orm.query.Query* class i override methods *all*, and *__getitem__ *and now query works But that is not good for orm objects such is *Organisation.Users*, need to use this handler for all queries I ask how can i pass handler to sqlalchemy? -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
Re: [sqlalchemy] Custom Handler for "No such polymorphic_identity"
On Mon, Mar 11, 2019 at 3:59 AM Tolstov Sergey wrote: > > Can i override handler for this error? > Example is - > > 1) defined only Parent class > 2) session.query(Parent) > -- > raised Exceptions > > Resolutions: > 1) try_except for all_queries (too much entry points) > 2) load all possible classes (low perfomance) > > How can i override /sqlalchemy/orm/loading.py > configure_subclass_mapper on runtime? If you are loading a Parent class, and the row indicates that it is in fact a Child class, then that mapper must be present.What would this custom handler do exactly, run a whole module import inside the loading process? I'm not sure that's threadsafe in any case.if your program runs long enough, all of those mappings would have been loaded in any case I'm not sure I understand what the problem is. > > -- > SQLAlchemy - > The Python SQL Toolkit and Object Relational Mapper > > http://www.sqlalchemy.org/ > > To post example code, please provide an MCVE: Minimal, Complete, and > Verifiable Example. See http://stackoverflow.com/help/mcve for a full > description. > --- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy+unsubscr...@googlegroups.com. > To post to this group, send email to sqlalchemy@googlegroups.com. > Visit this group at https://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
[sqlalchemy] Custom Handler for "No such polymorphic_identity"
Can i override handler for this error? Example is - > 1) defined only Parent class 2) session.query(Parent) -- > raised Exceptions Resolutions: 1) try_except for all_queries (too much entry points) 2) load all possible classes (low perfomance) How can i override /sqlalchemy/orm/loading.py configure_subclass_mapper on runtime? -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
