On 13/11/09 10:04 PM, Hans Pikkemaat wrote:
I ran some tests using 3.0b with SQLTemplate in combination with
prefetching and found
a possible new problem.
It seems that when running the query in eg 1 minute, it takes about 2
minutes before cayenne
has constructed the prefetched objects.
My query produces 2.5 million records. The query will take about 30
minutes. Construction
of the objects will then take an extra hour.
Just to be clear about what you are doing:
* Cayenne 3.0 beta 1
* SQL template query
* prefeching across to-many join
* paging on
You expect the first query (which gets hollow objects) to NOT include the
prefetch JOIN, but when fetching a page of results, it should use the prefetch.
Cayenne is constructing the first query which includes the JOIN and that makes
it take 30 minutes in your database to return 2.5 million records.
Is that correct?
My opinions:
1. Does prefetch really help here anyway? You are only getting (say) 100
records at a time, so the extra queries to follow the relations may not be that
significant.
2. Do you really want to fetch 2.5 million rows? If so, I assume this is not a
user interface :-) Perhaps Cayenne (or any ORM for that matter) is not the
best way to batch process that many rows.
Ari
--
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
