I implemented the changes you suggested, but I am running into errors with some polymorphic relations. Using a different theme, here is an example:
class AbstractVehicle(object): __tablename__ = "vehicles" id = Column(Integer, primary_key=True) name = Column(String, nullable=False) type = Column(String) __mapper_args__ = {"polymorphic_on": type} class AbstractCar(AbstractVehicle): __tablename__ = "cars" id = Column("id", Integer, primary_key=True) __mapper_args__ = {"polymorphic_identity": "car"} __table_args__ = ( ForeignKeyConstraint(["id"], ["vehicles.id"]), ) class AbstractCoupe(AbstractCar): __mapper_args__ = {"polymorphic_identity": "coupe"} Base = declarative_base() class Vehicle(AbstractVehicle, Base): pass class Car(AbstractCar, Vehicle): pass class Coupe(AbstractCoupe, Car): pass This resulted in the following error: File ".../model/vehicle.py", line 255, in <module> class Coupe(AbstractCoupe, Car): File ".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/api.py", line 55, in __init__ _as_declarative(cls, classname, cls.__dict__) File ".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 88, in _as_declarative _MapperConfig.setup_mapping(cls, classname, dict_) File ".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 103, in setup_mapping cfg_cls(cls_, classname, dict_) File ".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 131, in __init__ self._setup_table() File ".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 395, in _setup_table **table_kw) File ".../env/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 398, in __new__ "existing Table object." % key) InvalidRequestError: Table 'cars' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object. After adding "extend_existing=True" to the AbstractCoupe model, I received this error: File ".../model/vehicle.py", line 255, in <module> class Coupe(AbstractCoupe, Car): File ".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/api.py", line 55, in __init__ _as_declarative(cls, classname, cls.__dict__) File ".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 88, in _as_declarative _MapperConfig.setup_mapping(cls, classname, dict_) File ".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 103, in setup_mapping cfg_cls(cls_, classname, dict_) File ".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 135, in __init__ self._early_mapping() File ".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 138, in _early_mapping self.map() File ".../env/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 530, in map **self.mapper_args File "<string>", line 2, in mapper File ".../env/lib/python2.7/site-packages/sqlalchemy/orm/mapper.py", line 627, in __init__ self._configure_properties() File ".../env/lib/python2.7/site-packages/sqlalchemy/orm/mapper.py", line 1292, in _configure_properties self._adapt_inherited_property(key, prop, False) File ".../env/lib/python2.7/site-packages/sqlalchemy/orm/mapper.py", line 1514, in _adapt_inherited_property self._configure_property(key, prop, init=False, setparent=False) File ".../env/lib/python2.7/site-packages/sqlalchemy/orm/mapper.py", line 1541, in _configure_property prop.columns[0]) File ".../env/lib/python2.7/site-packages/sqlalchemy/sql/selectable.py", line 476, in corresponding_column if self.c.contains_column(column): File ".../env/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 754, in __get__ obj.__dict__[self.__name__] = result = self.fget(obj) File ".../env/lib/python2.7/site-packages/sqlalchemy/sql/selectable.py", line 553, in columns return self._columns.as_immutable() AttributeError: 'Table' object has no attribute '_columns' Is there a way to fix or work around this? Thanks, Angie On Wed, Jun 22, 2016 at 7:27 PM, Mike Bayer <mike...@zzzcomputing.com> wrote: > > > On 06/22/2016 04:53 PM, Angie Ellis wrote: > >> Mike, >> >> Thank you for the response! I do have control over the external library. >> I like your idea having both the abstract classes and concrete classes >> available in the library because I have use cases for both in my >> applications. However, I'm not quite sure how to configure the >> inheritance and declarative base. What am I doing wrong here? >> >> >> class AbstractFoo(object): >> __abstract__ = True >> __tablename__ = "foo" >> id = Column(Integer, primary_key=True) >> type = Column(String) >> foo_bar_id = Column(Integer, ForeignKey("foo_bar.id >> <http://foo_bar.id>")) >> foo_bar = relationship("FooBar", backref=backref("foo")) >> >> __mapper_args__ = {"polymorphic_on": type} >> >> >> class AbstractBar(object): >> __abstract__ = True >> __mapper_args__ = {"polymorphic_identity": "bar"} >> >> >> class AbstractFooBar(object): >> __abstract__ = True >> __tablename__ = "foo_bar" >> id = Column(Integer, primary_key=True) >> >> >> >> Base = declarative_base() >> >> >> class Foo(AbstractFoo, Base): >> pass >> >> >> class Bar(AbstractBar, Foo): >> pass >> >> >> class FooBar(AbstractFooBar, Base): >> pass >> >> >> Does AbstractBar need to inherit from AbstractFoo? Do the abstract >> classes need to inherit their own declarative base? Do I need to change >> the inheritance order? The classes Foo, Bar, and FooBar are not getting >> mapped. >> > > So the way you have it is that those are mixins, which is fine, and I > guess that's how we need it because you want to keep multiple Base's in > play at the same time. You don't need the __abstract__ keyword in that > case. So to map that looks like the following POC: > > from sqlalchemy import * > from sqlalchemy.orm import * > from sqlalchemy.ext.declarative import declarative_base > from sqlalchemy.ext.declarative import declared_attr > > > class AbstractFoo(object): > __tablename__ = "foo" > > id = Column(Integer, primary_key=True) > type = Column(String) > > @declared_attr > def foo_bar_id(cls): > return Column(Integer, ForeignKey("foo_bar.id")) > > @declared_attr > def foo_bar(cls): > return relationship("FooBar", backref=backref("foo")) > > __mapper_args__ = {"polymorphic_on": type} > > > class AbstractBar(AbstractFoo): > __mapper_args__ = {"polymorphic_identity": "bar"} > > > class AbstractFooBar(object): > __tablename__ = "foo_bar" > id = Column(Integer, primary_key=True) > > > Base = declarative_base() > > > class Foo(AbstractFoo, Base): > pass > > > class Bar(AbstractBar, Foo): > pass > > > class FooBar(AbstractFooBar, Base): > pass > > e = create_engine("sqlite://", echo=True) > Base.metadata.create_all(e) > s = Session(e) > > s.add(Bar(foo_bar=FooBar())) > > s.commit() > s.close() > > b1 = s.query(Foo).first() > print b1 > print b1.foo_bar > > > > > > >> Thanks, >> Angie >> >> On Fri, Jun 3, 2016 at 12:38 PM, Mike Bayer <mike...@zzzcomputing.com >> <mailto:mike...@zzzcomputing.com>> wrote: >> >> >> >> On 06/03/2016 02:44 PM, Angie E wrote: >> >> Rather than creating mixin classes that models inherit from, I >> have a >> use case that requires me to configure classes the other way >> around. The >> classes that would normally be mixin classes need to be the >> classes that >> inherit from the models as well as the class that model objects >> are >> created from. This is because the models and the mapper >> configurations >> are in an external library from the main repository. I need to >> pass in >> the host for the engine from the main repository to the models >> library >> before any of the models are loaded so they can load with the >> declarative base already configured. >> >> >> I hope you are using reflection for these models, otherwise there's >> no reason to have a depedency on the Engine for the model declaration. >> >> >> After the engine information is >> >> passed in, the session, Base class, and everything is created >> within a >> sort of base class that the models inherit from. Here is a >> simplified >> example: >> >> >> class SQLAlchemyBase(object): >> metadata = None >> Session = None >> Base = object >> sessionfactory = sessionmaker() >> >> def initialize(self, host): >> engine = create_engine(host) >> self.metadata = MetaData(bind=engine) >> self.Session = scoped_session(self.sessionfactory) >> self.Base = declarative_base(metadata=self.metadata) >> >> models = SQLAlchemyBase() >> >> (The models inherit from models.Base) >> >> >> So the SQLAlchemyBase will be imported into the main repository, >> the >> initialize method will be called, passing in the host for the >> engine, >> and the models can then be imported. >> >> >> Looks like no reflection taking place. I'd do away with the >> MetaData(bind=engine), and just have the engine as part of the >> sessionmaker(), and the sessionmaker() here also doesn't need to >> have anything to do with the "library", if the "library" is just >> defining model classes. >> >> The "bound metadata" pattern is highly discouraged in modern >> SQLAlchemy because it leads to this kind of confusion, e.g. that one >> needs an Engine in order to declare models. You don't. >> >> >> >> >> >> >> The main repository has its own >> >> classes with the same names as the models and have additional >> methods >> that a normal mixin class would have to extend functionality. >> However, I >> am unable to create model objects using the classes in the main >> repository because I can't get the mappers to play nice with this >> unusual inheritance that extends from the external models library. >> Additionally, in the models library, there are models that have >> multiple >> levels of inherited polymorphic relationships. Here is an >> example that >> is similar one of the more basic inherited polymorphic >> relationships: >> >> >> **Models Library** >> >> class Foo(models.Base): >> >> __tablename__ = "foo" >> id = Column(Integer, primary_key=True) >> type = Column(String) >> foo_bar_id = Column(Integer, ForeignKey("foo_bar.id >> <http://foo_bar.id>")) >> >> foo_bar = relationship(Foo, backref=backref("foos")) >> >> __mapper_args__ = {"polymorphic_on": type} >> >> >> class Bar(Foo): >> >> __mapper_args__ = {"polymorphic_identity": "bar"} >> >> >> class FooBar(models.Base): >> >> __tablename__ = "foo_bar" >> id = Column(Integer, primary_key=True) >> >> >> **Main Repository** >> >> from separate_library.models import models, Foo as BaseFoo, Bar as >> BaseBar, FooBar as BaseFooBar >> >> >> class Foo(BaseFoo): >> >> @classmethod >> def custom_create_method(cls, **kw): >> foo_obj = cls(**kw) >> models.session.add(foo_obj) >> models.session.flush() >> >> >> class Bar(BaseBar): >> pass >> >> >> class FooBar(BaseFooBar): >> pass >> >> >> The original error I was getting was something like this: >> >> "InvalidRequestError: One or more mappers failed to initialize - >> can't >> proceed with initialization of other mappers. Original >> exception was: >> Multiple classes found for path "Foo" in the registry of this >> declarative base. Please use a fully module-qualified path." >> >> >> that only happens if someone is using a string name inside of >> relationship(), like this: >> >> foobars = relationship("Foo") >> >> If the "library" is doing that, and you can't change it, easy >> solution, use a different name for your "Foo" that's also in the 3rd >> party library. Or declare your classes against a different base: >> >> Base = declarative_base() >> >> class Foo(other_package.Foo, Base): >> # ... >> >> >> class Bar(other_package.Bar, Base): >> # ... >> >> I'm not 100% sure that will work but there's probably a way to make >> that work. >> >> >> >> >> So I tried putting the full path in the relationships. Then it >> started >> giving me an error like this: >> >> >> oh, OK, then you're fine there. >> >> >> >> "FlushError: Attempting to flush an item of type <class >> 'main_module.models.Foo'> as a member of collection "FooBar.foos". >> Expected an object of type <class 'separate_library.models.Foo'> >> or a >> polymorphic subclass of this type. If <class >> 'main_module.models.Foo'> >> is a subclass of <class 'separate_library.models.Foo'>, >> configure mapper >> "Mapper|Foo|foos" to load this subtype polymorphically, or set >> enable_typechecks=False to allow any subtype to be accepted for >> flush. >> >> >> Well, when this relationship loads, you'd like it to return a list >> of main_module.models.Foo objects and not >> separate_library.models.Foo. Your model here needs to have a >> polymorphic identity assigned so that it can do this. I don't see >> those set up in your mappings, you need to put those in explcitly. >> If you're trying to re-use the same names that the superclass has, >> you might need to manipulate Foo.__mapper__.polymorphic_map >> directly, not sure. >> >> >> >> >> Essentially, the main problem is getting the classes in the main >> module >> to point to and act like the model classes. For example, when I >> try to >> create relationships, it says it expected an object of type >> 'separate_library.models.Foo' instead of 'main_module.models.Foo'. >> >> >> other than using a different name, there are ways to manipulate the >> registry that is doing this but they aren't publicly supported >> patterns, unless the easy "use a different Base" trick works out here. >> >> Additionally, in the polymorphic relationships, I can't get the >> polymorphic_identity to populate for the polymorphic_on column. >> For >> example, Bar in the main repository will have the 'type' column >> empty >> when the object is initially created. >> >> >> when you make a subclass like you are doing, you need to give it its >> own identity, so it at least needs to have "polymorphic_identity" >> set up locally. >> >> Overall there would be a question here, which is: do you have >> control over this external library? If so, I would design it to be >> used for subclassing by another declarative system. Assign names >> like "AbstractFoo" to the classes and additionally use the >> __abstract__ = True declarative flag to mark them as such, then use >> @declared_attr around __mapper_args__ to assign mapper arguments >> like polymorphic identity to subclasses properly. >> >> If you do *not* have control over this external library, then it is >> not intended to be used in this way; the most preferable way to go >> about this would be to get the maintainers of it to change it in >> order to allow it to be used in this way. An easy way for the >> external library to work like this would be that it supplies both >> the __abstract__ classes with nice names like AbstractFoo, then for >> its own purposes it supplies its own Foo / Bar concrete classes on >> top of a separate declarative Base, and you'd skip using that part. >> >> >> >> >> One idea I tried was to add a metaclass to the declarative base >> in the >> models library and modify the mappers in the __init__ method >> during >> their initialization. I made progress this way, but haven't >> gotten it to >> work completely. >> >> >> There's proabably ways to make this work with more tricks and >> metaclasses but they aren't supported on the SQLAlchemy end and they >> will be brittle, they will break when declarative changes how it >> does things internally. >> >> >> >> Sorry for the complex explanation, but this is a complex >> problem. I am >> not able to change anything about the models or the use case, >> unfortunately. I have to work within these constraints. >> >> >> If these are two different systems within your company, it would be >> a better investment to get the upstream system to work more >> flexibly, rather than invest efforts working around the system's >> limitations downstream. But, if the overlapping names and the >> polymorphic identities are the only problem you should be able to >> work around those. >> >> >> >> If anyone can >> >> offer ideas on how to configure the mappers for the classes in >> the main >> repository to act like the models in the model library, I would >> be very >> grateful. >> >> -- >> You received this message because you are subscribed to the Google >> Groups "sqlalchemy" group. >> To unsubscribe from this group and stop receiving emails from >> it, send >> an email to sqlalchemy+unsubscr...@googlegroups.com >> <mailto:sqlalchemy%2bunsubscr...@googlegroups.com> >> <mailto:sqlalchemy+unsubscr...@googlegroups.com >> <mailto:sqlalchemy%2bunsubscr...@googlegroups.com>>. >> To post to this group, send email to sqlalchemy@googlegroups.com >> <mailto:sqlalchemy@googlegroups.com> >> <mailto:sqlalchemy@googlegroups.com >> <mailto:sqlalchemy@googlegroups.com>>. >> Visit this group at https://groups.google.com/group/sqlalchemy. >> For more options, visit https://groups.google.com/d/optout. >> >> >> -- >> You received this message because you are subscribed to a topic in >> the Google Groups "sqlalchemy" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/sqlalchemy/u_CiTJIK9Gs/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> sqlalchemy+unsubscr...@googlegroups.com >> <mailto:sqlalchemy%2bunsubscr...@googlegroups.com>. >> To post to this group, send email to sqlalchemy@googlegroups.com >> <mailto:sqlalchemy@googlegroups.com>. >> Visit this group at https://groups.google.com/group/sqlalchemy. >> For more options, visit https://groups.google.com/d/optout. >> >> >> -- >> You received this message because you are subscribed to the Google >> Groups "sqlalchemy" group. >> To unsubscribe from this group and stop receiving emails from it, send >> an email to sqlalchemy+unsubscr...@googlegroups.com >> <mailto:sqlalchemy+unsubscr...@googlegroups.com>. >> To post to this group, send email to sqlalchemy@googlegroups.com >> <mailto:sqlalchemy@googlegroups.com>. >> Visit this group at https://groups.google.com/group/sqlalchemy. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to a topic in the > Google Groups "sqlalchemy" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/sqlalchemy/u_CiTJIK9Gs/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > sqlalchemy+unsubscr...@googlegroups.com. > To post to this group, send email to sqlalchemy@googlegroups.com. > Visit this group at https://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.