On Thursday, May 4, 2017 at 12:16:20 AM UTC-4, David Chanin wrote:
>
> How do objects get registered into the aggressive loader? Does it happen
> automatically when they're initially loaded via query or cache? Ideally we
> wanted to group items together when they're loaded and do bulk lazy loading
> on that group rather than on all models in the session, but couldn't figure
> out a good way to accomplish that. Ex if we run users =
> session.query(User).limit(10).all() then we'd just want to do bulk lazy
> loading within the users from that query rather than all users that happen
> to be in the session, but it seems like we'd need to do some pretty
> invasive changes to SQLAlchemy to keep track of which models were initially
> loaded together.
>
Our implementation (which is a dirty hack) is loosely this:
class Students(Base):
aggresive_load = ['school', 'report_cards', 'classes']
class Classes(Base):
aggresive_load = ['teacher', 'students']
agLoader = AggressiveLoader(dbSession)
students = session.query(Students).limit(10).all()
agLoader.register(students)
agLoader.load()
The `register` function adds the students at loaded.
The `load` function inspects all the loaded objects and derives the
potential primary keys/relationships to load.
To put this in context, the use-case for our concern is an multiple
newspaper view. One page had hundreds of Articles, from dozens of
Publications, with tend of Authors; this could happen in multiple context
zones.
* Non-lazyload -- 4000+ sql statements
* lazyloading -- 1200+ sql statements
* aggressive loading like this -- 100+ statements
We specifically avoided your intended approach of 'bulk on the group',
because we wanted to defer all the unloaded primary-key candidates from
multiple queries into a single select. In the newspaper view above, there
could be 30 content zones that have a relationship to the "Author" table --
with overlap; our goal was to aggregate all 30 SELECTs that might return
900 rows into a single SELECT that returns only the distinct rows-- perhaps
100.
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable
Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
