Re: [sqlalchemy] Large number of polymorphic subclasses

2017-05-01 Thread mike bayer
The selectinload feature revealed the need to reorganize most of the "loading_relationships" documentation to make adequate context for the new feature and overall improve a lot of crufty organization. This reorg of the docs somewhat awkwardly needed to be decoupled from the feature itself

Re: [sqlalchemy] Large number of polymorphic subclasses

2017-05-01 Thread damon
Hey Mike, I just saw the new selectinload feature, and am super excited about it! As far as I can tell, this is only for relationships, so I wanted to check what the plans are for an inheritance loader using selectinload as well, given that you initially mentioned them together and I can't

Re: [sqlalchemy] Large number of polymorphic subclasses

2017-04-04 Thread Mike Bayer
Reusing the original query is hard, and the case you have is due to mismatched entities that would have to be handled, probably by wrapping the original query in a subquery just like subquery eager loading does. The subquery eager loading feature took a few years to work out an enormous number of

Re: [sqlalchemy] Large number of polymorphic subclasses

2017-04-04 Thread damon
Hey Mike, Thanks for those -- seems to have helped those cases, though we're running into some pretty weird behavior with joins. Here's a simplified case that shows one of the issues we're running into (use _loader_from_cls from above): class W(Base): __tablename__ = 'w' id =

Re: [sqlalchemy] Large number of polymorphic subclasses

2017-04-04 Thread mike bayer
This repaste of the example contains two refinements to address each of these issues distinctly. We want to avoid re-entrant invocation of _loader_for_cls, so putting a flag into attributes handles that. Then there's the issue of same class coming in that we are already querying, we can

Re: [sqlalchemy] Large number of polymorphic subclasses

2017-04-03 Thread damon
Hey Mike, Looks like I spoke too soon -- a few more questions: Using the example code you posted, we're actually seeing 4 additional queries (one per result model), rather than the expected 3 (one per result model type). If you print context.query inside load_extra, I think it's clear why: -

Re: [sqlalchemy] Large number of polymorphic subclasses

2017-04-03 Thread damon
Thanks a ton for your help, Mike! We played around with it and are pretty happy with your solution using the load() event, so we'll be using that moving forward. Damon On Wednesday, March 29, 2017 at 2:40:39 PM UTC-7, Mike Bayer wrote: > > I have a working version of both loading

Re: [sqlalchemy] Large number of polymorphic subclasses

2017-03-29 Thread mike bayer
I have a working version of both loading relationships via IN as well as loading joined inheritance subclasses via IN, including your great idea that extra eager loaders should continue to work for the subclass loaders. I've only tested it with one scenario so far and both patches have a

Re: [sqlalchemy] Large number of polymorphic subclasses

2017-03-23 Thread mike bayer
On 03/23/2017 02:40 PM, mike bayer wrote: On 03/23/2017 12:53 PM, da...@benchling.com wrote: Hey Mike, Thanks for the quick response! For developers that are pretty familiar with the SQLAlchemy API, but not so much the internals, would implementing the subqueryloads to contribute to SA be

Re: [sqlalchemy] Large number of polymorphic subclasses

2017-03-23 Thread mike bayer
On 03/23/2017 12:53 PM, da...@benchling.com wrote: Hey Mike, Thanks for the quick response! For developers that are pretty familiar with the SQLAlchemy API, but not so much the internals, would implementing the subqueryloads to contribute to SA be a reasonable endeavor? Could you ballpark

Re: [sqlalchemy] Large number of polymorphic subclasses

2017-03-23 Thread damon
Hey Mike, Thanks for the quick response! For developers that are pretty familiar with the SQLAlchemy API, but not so much the internals, would implementing the subqueryloads to contribute to SA be a reasonable endeavor? Could you ballpark how much time how long it might take for us to do it?

Re: [sqlalchemy] Large number of polymorphic subclasses

2017-03-22 Thread mike bayer
On 03/22/2017 02:17 PM, da...@benchling.com wrote: Hey all, We were wondering if you had any advice on having a large (~10) number of polymorphic subclasses for a single base class. Using with_polymorphic: '*' causes SQLAlchemy to joinedload all subclasses like this: SELECT ... FROM

[sqlalchemy] Large number of polymorphic subclasses

2017-03-22 Thread damon
Hey all, We were wondering if you had any advice on having a large (~10) number of polymorphic subclasses for a single base class. Using with_polymorphic: '*' causes SQLAlchemy to joinedload all subclasses like this: SELECT ... FROM base_table LEFT OUTER JOIN sub_table_1 ON base_table.id =