Re: findAll() memory exhaustion
Ah! That's new! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php -~--~~~~--~~--~--~---
findAll() memory exhaustion
Hi,
I am using findAll() to retrieve about 7500 records. My current PHP
setup is such that this fails with an out of memory error.
So I tried this:
$conditions = null;
$fields = null;
$order = null;
$page = 0;
$limit = 200;
$recursive = null;
while( 1 ) {
echo pre$page/pre;
$allTests = null;
$allTests =
$this-Test-findAll($conditions,$fields,$order,$limit,$page,$recursive);
$this-set('data', count($allTests) );
echo pre;echo count($allTests);echo /pre;
if( count($allTests) ==0 ) {
break;
}
$page++;
}
and this too runs out of memory. It works OK for a few 'pages' worth
($page can get as high as about 12 before memory is exhausted).
It seems that memory is being consumed by findAll() somewhere along the
line.
My DEBUG level is set to 2 so caching of queries should not be
happening, should it?
Any ideas?
Allan
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups Cake
PHP group.
To post to this group, send email to cake-php@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at http://groups.google.com/group/cake-php
-~--~~~~--~~--~--~---
Re: findAll() memory exhaustion
Hey Alan, why you'd ever want to pull 7500 records at once in the normal operation of an app is quite beyond me, but I know for certain things, I have pulled somewhere around 10,000 records at once with no issues. You might want to check your memory settings in php.ini. I think the default is only about 8mb or so, so that might be the problem. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php -~--~~~~--~~--~--~---
Re: findAll() memory exhaustion
nate wrote: Hey Alan, why you'd ever want to pull 7500 records at once in the normal operation of an app is quite beyond me, but I know for certain things, I have pulled somewhere around 10,000 records at once with no issues. i have found a way round this, using a direct SQL query() but ... You might want to check your memory settings in php.ini. I think the default is only about 8mb or so, so that might be the problem. Yup, the default memory is about 8M and is used up, I could up that to 32M etc etc BUT the point is by paginating I should be able to avoid having to change this. Is there something in findAll() that's caching the data somewhere? Allan --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php -~--~~~~--~~--~--~---
Re: findAll() memory exhaustion
Yes, query results are cached (somewhere in DboSource I believe). But if you're paging results, why are you doing it by reading the entire resultset into memory? Model::findAll( ) has the $limit and $page parameters so you don't have to. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php -~--~~~~--~~--~--~---
Re: findAll() memory exhaustion
nate wrote: Yes, query results are cached (somewhere in DboSource I believe). But if you're paging results, why are you doing it by reading the entire resultset into memory? Model::findAll( ) has the $limit and $page parameters so you don't have to. Ah, I think I may not have explained my situation clearly enough, sorry. I tried first to read the entire resultset into memory and that failed so I then started to use the $limit and $page parameters and to my surprise found myself *still* running out of memory. The code fragment I pasted in the original posting was supposed to demonstrate my use of findAll( ..., $limit, $page ) and I guess I was asking for advice as to whether the code fragment was valid (is this how findAll() is expected to be used) or not. If it is valid then something is wrong 'cos it looks to me that I'm not using great amounts of memory using this approach. Allan --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php -~--~~~~--~~--~--~---
Re: findAll() memory exhaustion
One thing I missed completely the first time around: if you set $recursive to null, it just uses the default level of recursion, which is 1. To disable pulling of associated model data, set $recursive to 0. To disable all associations, incuding LEFT JOINs, set $recursive to -1. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php -~--~~~~--~~--~--~---
