Re: [sqlalchemy] [Q][SA 0.7.9] yield_per() vs. joinedload() and subqueryload()

2013-06-05 Thread Ladislav Lenart
Hello. I have one more question about my approach to WindowedRangeQuery: * Suppose a complex query q with join(...), filter(...) and options(...). * I need to create q2 from q such that: * It has no options. * Can I reset the options with q = q.options(None)? * It has select with the

Re: [sqlalchemy] [Q][SA 0.7.9] yield_per() vs. joinedload() and subqueryload()

2013-06-05 Thread Ladislav Lenart
Hello. I've tried an experiment to verify that yield_per() with subqueryload() behaves as badly as you described, but according to my practical observation, it issues ONE subqueryload() and everything behaves as I would expect. It emits two SELECTs, one to fetch the objects and the second one to

Re: [sqlalchemy] [Q][SA 0.7.9] yield_per() vs. joinedload() and subqueryload()

2013-06-05 Thread Ladislav Lenart
Hello. One more note. I've just tried the below experiment with joinedload() instead of subqueryload() and that does NOT work just as you expected. One contact is returned several times and the first occurrences have incomplete phones. However my experiments with subqueryload() suggest that it

Re: [sqlalchemy] [Q][SA 0.7.9] yield_per() vs. joinedload() and subqueryload()

2013-06-05 Thread Michael Bayer
the Query emitted by subqueryload does not use yield_per. so if your total result set is 1000 rows, and the total rows represented by all the collections is 1, the first time that query is emitted, 1 rows will be fully loaded and processed into memory at once. This would occur

Re: [sqlalchemy] [Q][SA 0.7.9] yield_per() vs. joinedload() and subqueryload()

2013-06-05 Thread Ladislav Lenart
Ok, so yield_per() is useless when subqueryload() is used. Thank you, Ladislav Lenart On 5.6.2013 16:27, Michael Bayer wrote: the Query emitted by subqueryload does not use yield_per. so if your total result set is 1000 rows, and the total rows represented by all the collections is

Re: [sqlalchemy] [Q][SA 0.7.9] yield_per() vs. joinedload() and subqueryload()

2013-06-04 Thread Michael Bayer
On Jun 4, 2013, at 8:18 AM, Ladislav Lenart lenart...@volny.cz wrote: Hello. I have a hard time to understand the following comment for Query.yield_per(): Yield only ``count`` rows at a time. WARNING: use this method with caution; if the same instance is present in

Re: [sqlalchemy] [Q][SA 0.7.9] yield_per() vs. joinedload() and subqueryload()

2013-06-04 Thread Ladislav Lenart
Hello. assuming cls.foo is a many-to-one, it will produce the correct result, but will be far worse in terms of memory and performance, as the subqueryload() call will be invoked for each distinct batch of 50 rows, across the *full* result set. So if your result has 1000 rows, and the

Re: [sqlalchemy] [Q][SA 0.7.9] yield_per() vs. joinedload() and subqueryload()

2013-06-04 Thread Michael Bayer
On Jun 4, 2013, at 10:45 AM, Ladislav Lenart lenart...@volny.cz wrote: Hello. assuming cls.foo is a many-to-one, it will produce the correct result, but will be far worse in terms of memory and performance, as the subqueryload() call will be invoked for each distinct batch of 50 rows,

Re: [sqlalchemy] [Q][SA 0.7.9] yield_per() vs. joinedload() and subqueryload()

2013-06-04 Thread Ladislav Lenart
Hello. You will then get the wrong results. The docstring tries to explain this - a joinedload uses a JOIN. For each cls instance, there are many rows, one for each bar. If you cut off the results in the middle of populating that collection, the collection is incomplete, you'll see the

Re: [sqlalchemy] [Q][SA 0.7.9] yield_per() vs. joinedload() and subqueryload()

2013-06-04 Thread Michael Bayer
On Jun 4, 2013, at 11:41 AM, Ladislav Lenart lenart...@volny.cz wrote: Hello. You will then get the wrong results. The docstring tries to explain this - a joinedload uses a JOIN. For each cls instance, there are many rows, one for each bar. If you cut off the results in the middle of

Re: [sqlalchemy] [Q][SA 0.7.9] yield_per() vs. joinedload() and subqueryload()

2013-06-04 Thread Ladislav Lenart
Hello. OK, but with yield_per() you want to use eagerloading also, so yield_per() not fast enough either, I guess No. I use yield_per() on complex queries with join(), filter() and both joinedload() and subqueryload(). It is possible that they sometimes returns wrong results because of

Re: [sqlalchemy] [Q][SA 0.7.9] yield_per() vs. joinedload() and subqueryload()

2013-06-04 Thread Michael Bayer
On Jun 4, 2013, at 12:28 PM, Ladislav Lenart lenart...@volny.cz wrote: Absolutely, you should do whatever you have to in order to get the range you want, in fact the recipe even says this... Ok. What I want to do is basically the following: * Suppose a complex query q with join(...),

Re: [sqlalchemy] [Q][SA 0.7.9] yield_per() vs. joinedload() and subqueryload()

2013-06-04 Thread Ladislav Lenart
On 4.6.2013 18:49, Michael Bayer wrote: On Jun 4, 2013, at 12:28 PM, Ladislav Lenart lenart...@volny.cz wrote: Absolutely, you should do whatever you have to in order to get the range you want, in fact the recipe even says this... Ok. What I want to do is basically the following: *