On Jul 8, 2011, at 6:17 PM, Ben Chess wrote: > Hmm, bummer. > > We're pretty dependent on detached mode. We eagerload a bunch of > things for the purposes of storing them in memcache, and things don't > go well when they're not fully traversable.
OK so you're caching. Actually when I worked with caching a lot I found that I was looking for the objects to exist without their related collections, then I'd use the Beaker recipe so that the collections etc. were cached separately as they were lazy loaded. It allowed the cache to contain individual collections of things more discretely, rather than several large trees tailored towards some specific loading scenario. The more granular level collections allowed the retrieval over memcached to not spend time loading things that weren't needed for a particular view. > I've been delving into > the mapper.py code site you mention. It's a lot of new code to me, > but I agree it doesn't seem trivial. Also seems like a potential > performance hit, but could be something that'd be enabled with an > option. > > One thing I still don't understand though is why the problem only > happens when we first eagerly load the completely unrelated "d_row"? its the "a_obj.c_rows" that loads all the C's which then eagerload their A, the A of course is already in the Session but it does the join and populates each C.a_row. > > > On Thu, Jul 7, 2011 at 6:39 PM, Michael Bayer <mike...@zzzcomputing.com> > wrote: >> >> On Jul 7, 2011, at 7:04 PM, Ben Chess wrote: >> >>> I've hit a problem where eagerload() fails to load in a relation of a >>> relation when lazy='joined' is involved. It's easiest just to show >>> the test. It fails in 0.7.1, and an equivalent test also fails in >>> 0.6.8. >>> >>> http://pastebin.com/ruq6SM1z >>> >>> Basically, A has relations to B, C, and D. C's relationship to A is a >>> lazy='joined'. >>> >>> First load A, eagerloading 'd_row' >>> Then reference A.c_row, causing it to load C. >>> >>> Then, separately, load C, eagerloading 'a_row.b_row'. >>> At this point, I expunge_all() and demonstrate that b_row was not >>> attached to C.a_row. >>> >>> This does not occur if C's relationship to A is lazy='select'. >>> Weirdly, this also does not occur if the initial load of A does not >>> eagerload 'd_row'. I'm not sure why that should affect anything. >> >> This will make it pass: >> >> assert 'a_row' in a_obj.c_rows[0].__dict__ >> session.expire_all() >> >> c_obj = >> session.query(C).options(eagerload_all('a_row.b_row')).filter_by(id=1).one() >> session.expunge_all() >> >> assert c_obj.a_row.b_row >> >> note after load #1, c_obj is already in the Session, and c_obj.a_row is >> already populated (looking in __dict__ is always the way to see if something >> is already loaded). This is because of the lazy=False on C.a_row. >> >> Then what happens in the load, and it occurs on line 2587 of mapper.py in >> the current tip, we get the C object already in the identity map during the >> second load. We say, OK C do you have any attributes that aren't populated >> which we can pull from this row ? C says, "nope". C.a_row is already >> there. This process currently doesn't descend further into the objects >> attached to C.a_row so the rest of the columns are thrown away. >> >> It was actually somewhat of an innovation around 0.5 or so when I actually >> got the thing to populate "unloaded" attributes on objects that were >> otherwise loaded and might even have pending changes, which was a big step >> forward at that time, I didn't take on trying to figure out if eagers could >> keep on going into the graph and find deeper attributes that aren't loaded. >> >> If you have an opinion on this, let me know, right now I feel like its in an >> OK place considering the tradeoff of digging way down into a graph which may >> be unnecessary for those rows that were already loaded, many-to-ones are >> usually not an issue since they pull from the identity map. If the >> issue is you're going for "detached" behavior, I generally don't recommend >> relying heavily on object graphs that are fully traversable in the detached >> state unless you're doing some kind of offline caching. Of course, if >> there were a patch to that area of code that successfully kept the traversal >> going deeper into already loaded nodes based on the current eagers present, >> I'm open to evaluating it, though it doesn't seem like a quick tweak at the >> moment. >> >> Nice test though, if you're interested in helping with tests/patches we're >> always looking for help. >> >> >> >> >> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "sqlalchemy" group. >>> To post to this group, send email to sqlalchemy@googlegroups.com. >>> To unsubscribe from this group, send email to >>> sqlalchemy+unsubscr...@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/sqlalchemy?hl=en. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "sqlalchemy" group. >> To post to this group, send email to sqlalchemy@googlegroups.com. >> To unsubscribe from this group, send email to >> sqlalchemy+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/sqlalchemy?hl=en. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to sqlalchemy@googlegroups.com. > To unsubscribe from this group, send email to > sqlalchemy+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
