Re: Memory Leak in Model::Save()
Thank you, Dr. Loboto.
Your solution appears to have helped me a lot!
During a simple test, it did not report any memory leak.
As I've spent these few days running my applications, there has been
no reports of any memory leak thus far.
I truly appreciate the community at CakePHP!
Thank you so much everyone!
On Apr 11, 11:07 am, "Dr. Loboto" wrote:
> With debug < 2 and cacheQueries = false nothing is saved and this
> source of "leaks" is eliminated.
>
> On Apr 9, 2:01 am, Ryan Schmidt wrote:
>
>
>
>
>
>
>
> > On Apr 8, 2011, at 07:18, ProFire wrote:
>
> > > Well, whether it'smemoryleaks or not, I'm experiencing amemory
> > > exhaustion.
> > > The code would look like this:
>
> > > $memorySize = memory_get_usage();// X bytes
> > > $this->model->save($this->data);
> > > $memorySize = memory_get_usage();// X + 2000 bytes
> > > $this->model->create();
> > > $memorySize = memory_get_usage();// X + 2000 bytes
>
> > > I ran this test a few times by putting it in a function and looping
> > > it.
> > > The results are pretty consistent.
> > > Each Model::Save() increasesmemoryby about 2000 bytes.
> > > Even after I call Model::Create(), thememorydoes not reduce.
>
> > > Is there a way to clear thememory?
> > > Does anybody also experience a somewhat similar issue? where by each
> > > Model::Save results in an increase ofmemory?
>
> > This topic has come up before; read this thread:
>
> >http://groups.google.com/group/cake-php/browse_thread/thread/3f4cfb80...
>
> > The guess there is that CakePHP saves the query (and/or information about
> > it) somewhere inmemory, possibly for debugging purposes, and that CakePHP
> > simply didn't expect you to be doing so many queries at once that this
> > extra debugging information would exhaustmemory. Perhaps you can identify
> > where in the core files this is happening ("look in dbo_source for
> > cacheMethod"), and see if it can be reduced or eliminated.
>
> > Alternately, perhaps you can rewrite your process so it doesn't have to do
> > so many queries in one request.
--
Our newest site for the community: CakePHP Video Tutorials
http://tv.cakephp.org
Check out the new CakePHP Questions site http://ask.cakephp.org and help others
with their CakePHP related questions.
To unsubscribe from this group, send email to
cake-php+unsubscr...@googlegroups.com For more options, visit this group at
http://groups.google.com/group/cake-php
Re: Memory Leak in Model::Save()
With debug < 2 and cacheQueries = false nothing is saved and this
source of "leaks" is eliminated.
On Apr 9, 2:01 am, Ryan Schmidt wrote:
> On Apr 8, 2011, at 07:18, ProFire wrote:
>
>
>
>
>
>
>
>
>
> > Well, whether it's memory leaks or not, I'm experiencing a memory
> > exhaustion.
> > The code would look like this:
>
> > $memorySize = memory_get_usage();// X bytes
> > $this->model->save($this->data);
> > $memorySize = memory_get_usage();// X + 2000 bytes
> > $this->model->create();
> > $memorySize = memory_get_usage();// X + 2000 bytes
>
> > I ran this test a few times by putting it in a function and looping
> > it.
> > The results are pretty consistent.
> > Each Model::Save() increases memory by about 2000 bytes.
> > Even after I call Model::Create(), the memory does not reduce.
>
> > Is there a way to clear the memory?
> > Does anybody also experience a somewhat similar issue? where by each
> > Model::Save results in an increase of memory?
>
> This topic has come up before; read this thread:
>
> http://groups.google.com/group/cake-php/browse_thread/thread/3f4cfb80...
>
> The guess there is that CakePHP saves the query (and/or information about it)
> somewhere in memory, possibly for debugging purposes, and that CakePHP simply
> didn't expect you to be doing so many queries at once that this extra
> debugging information would exhaust memory. Perhaps you can identify where in
> the core files this is happening ("look in dbo_source for cacheMethod"), and
> see if it can be reduced or eliminated.
>
> Alternately, perhaps you can rewrite your process so it doesn't have to do so
> many queries in one request.
--
Our newest site for the community: CakePHP Video Tutorials
http://tv.cakephp.org
Check out the new CakePHP Questions site http://ask.cakephp.org and help others
with their CakePHP related questions.
To unsubscribe from this group, send email to
cake-php+unsubscr...@googlegroups.com For more options, visit this group at
http://groups.google.com/group/cake-php
Re: Memory Leak in Model::Save()
On Apr 8, 2011, at 07:18, ProFire wrote:
> Well, whether it's memory leaks or not, I'm experiencing a memory
> exhaustion.
> The code would look like this:
>
> $memorySize = memory_get_usage();// X bytes
> $this->model->save($this->data);
> $memorySize = memory_get_usage();// X + 2000 bytes
> $this->model->create();
> $memorySize = memory_get_usage();// X + 2000 bytes
>
> I ran this test a few times by putting it in a function and looping
> it.
> The results are pretty consistent.
> Each Model::Save() increases memory by about 2000 bytes.
> Even after I call Model::Create(), the memory does not reduce.
>
> Is there a way to clear the memory?
> Does anybody also experience a somewhat similar issue? where by each
> Model::Save results in an increase of memory?
This topic has come up before; read this thread:
http://groups.google.com/group/cake-php/browse_thread/thread/3f4cfb807b5efb08/8878cfa997fb5a98
The guess there is that CakePHP saves the query (and/or information about it)
somewhere in memory, possibly for debugging purposes, and that CakePHP simply
didn't expect you to be doing so many queries at once that this extra debugging
information would exhaust memory. Perhaps you can identify where in the core
files this is happening ("look in dbo_source for cacheMethod"), and see if it
can be reduced or eliminated.
Alternately, perhaps you can rewrite your process so it doesn't have to do so
many queries in one request.
--
Our newest site for the community: CakePHP Video Tutorials
http://tv.cakephp.org
Check out the new CakePHP Questions site http://ask.cakephp.org and help others
with their CakePHP related questions.
To unsubscribe from this group, send email to
cake-php+unsubscr...@googlegroups.com For more options, visit this group at
http://groups.google.com/group/cake-php
Re: Memory Leak in Model::Save()
On 4/8/2011 3:18 PM, ProFire wrote: Is there a way to clear the memory? Does anybody also experience a somewhat similar issue? where by each Model::Save results in an increase of memory? You can't control memory in PHP. I'd try different combos of max_execution_time[1] and memory_limit[2]. However I don't think this will help you much with the huge data sets you've mentioned. Another approach will be to split you data into blocks of n size and save it using Model::saveAll($data). HTH. [1] http://www.php.net/manual/en/info.configuration.php#ini.max-execution-time [2] http://www.php.net/manual/en/ini.core.php#ini.memory-limit P.S. Providing a DB schema and some code would help everyone to better understand your situation. -- Our newest site for the community: CakePHP Video Tutorials http://tv.cakephp.org Check out the new CakePHP Questions site http://ask.cakephp.org and help others with their CakePHP related questions. To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php
Re: Memory Leak in Model::Save()
Well, whether it's memory leaks or not, I'm experiencing a memory exhaustion. The code would look like this: $memorySize = memory_get_usage();// X bytes $this->model->save($this->data); $memorySize = memory_get_usage();// X + 2000 bytes $this->model->create(); $memorySize = memory_get_usage();// X + 2000 bytes I ran this test a few times by putting it in a function and looping it. The results are pretty consistent. Each Model::Save() increases memory by about 2000 bytes. Even after I call Model::Create(), the memory does not reduce. Is there a way to clear the memory? Does anybody also experience a somewhat similar issue? where by each Model::Save results in an increase of memory? On Apr 8, 12:56 pm, Alex Ciobanu wrote: > On 4/8/2011 7:47 AM, keymaster wrote:> I think Miles is pointing out there is > a bug in php interpreter which > > can lose memory under certain circumstances, and through our coding we > > can trigger that bug. > > > So, yes, it appears possible that memory can leak, and that by > > changing our code, we can stop it. > > > Pretty scary, isn't it. > > Yeah, this can get nasty :-) -- Our newest site for the community: CakePHP Video Tutorials http://tv.cakephp.org Check out the new CakePHP Questions site http://ask.cakephp.org and help others with their CakePHP related questions. To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php
Re: Memory Leak in Model::Save()
On 4/8/2011 7:47 AM, keymaster wrote: I think Miles is pointing out there is a bug in php interpreter which can lose memory under certain circumstances, and through our coding we can trigger that bug. So, yes, it appears possible that memory can leak, and that by changing our code, we can stop it. Pretty scary, isn't it. Yeah, this can get nasty :-) -- Our newest site for the community: CakePHP Video Tutorials http://tv.cakephp.org Check out the new CakePHP Questions site http://ask.cakephp.org and help others with their CakePHP related questions. To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php
Re: Memory Leak in Model::Save()
I think Miles is pointing out there is a bug in php interpreter which can lose memory under certain circumstances, and through our coding we can trigger that bug. So, yes, it appears possible that memory can leak, and that by changing our code, we can stop it. Pretty scary, isn't it. -- Our newest site for the community: CakePHP Video Tutorials http://tv.cakephp.org Check out the new CakePHP Questions site http://ask.cakephp.org and help others with their CakePHP related questions. To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php
Re: Memory Leak in Model::Save()
On 4/7/2011 8:37 PM, Miles J wrote: @Alex - Yes you can have memory leaks in PHP. http://www.google.com/search?q=memory+leaks+in+php&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a Miles, please point me to calloc/malloc PHP equivalent so I can test memory leaking in PHP. -- Our newest site for the community: CakePHP Video Tutorials http://tv.cakephp.org Check out the new CakePHP Questions site http://ask.cakephp.org and help others with their CakePHP related questions. To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php
Re: Memory Leak in Model::Save()
@Alex - Yes you can have memory leaks in PHP. http://www.google.com/search?q=memory+leaks+in+php&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a On Apr 7, 3:59 am, Alex Ciobanu wrote: > On 4/4/2011 6:38 PM, ProFire wrote:> I was very careful not to have any > memory leak. > > You don't control memory in PHP, you can't have memory > leaks.http://goo.gl/nm6OZ -- Our newest site for the community: CakePHP Video Tutorials http://tv.cakephp.org Check out the new CakePHP Questions site http://ask.cakephp.org and help others with their CakePHP related questions. To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php
Re: Memory Leak in Model::Save()
On 4/4/2011 6:38 PM, ProFire wrote: I was very careful not to have any memory leak. You don't control memory in PHP, you can't have memory leaks. http://goo.gl/nm6OZ -- Our newest site for the community: CakePHP Video Tutorials http://tv.cakephp.org Check out the new CakePHP Questions site http://ask.cakephp.org and help others with their CakePHP related questions. To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php
Re: Memory Leak in Model::Save()
I am not really experienced, but this looks like it is not up to you. It might be that it is not your problem, but servers or cakes.. However I don't think cake was made for generating such large data.. What is the purpose of your project I mean as a whole project. Is it like a web site, or is it console application ? If it is not web oriented php might not be t amount of the perfect solution for you.. Maybe c++ with that data or C# etc.. You could also maybe test to see if it is MySql database problem, CakePHP problem or your server (I'm guessing apache) problem. You can make a script in let's say Java which will make 50 million entries to the database, and see if it will break as well. If that works fine, then you can see if there is a problem with apache (maybe it makes these 2000 b when you do a http://request) by making php script (without usage of cakephp framework, so new project) do a 50 million saves in a row with http request and at last test cake with a 50 million saves with and without http request (by a for loop, $this- >model->save). You will get an idea then where the problem lies, and maybe try to work around the issue, not to use that memory leaker etc... Milos On Apr 5, 5:13 am, "Dr. Tarique Sani" wrote: > Don't really know about the memory leak but I would not use a http request > for generating the data > > If you have to use PHP or CakePHP create a console script and run it via a > cron or some other queue system > > Tarique > > On Mon, Apr 4, 2011 at 9:08 PM, ProFire wrote: > > Hello Fellow Bakers! > > > I have developed an application to generate a family tree. > > In one of my models, I have attached Tree Behaviour. > > > -- > > = > PHP for E-Biz:http://sanisoft.com > = -- Our newest site for the community: CakePHP Video Tutorials http://tv.cakephp.org Check out the new CakePHP Questions site http://ask.cakephp.org and help others with their CakePHP related questions. To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php
Re: Memory Leak in Model::Save()
Don't really know about the memory leak but I would not use a http request for generating the data If you have to use PHP or CakePHP create a console script and run it via a cron or some other queue system Tarique On Mon, Apr 4, 2011 at 9:08 PM, ProFire wrote: > Hello Fellow Bakers! > > I have developed an application to generate a family tree. > In one of my models, I have attached Tree Behaviour. > > -- = PHP for E-Biz: http://sanisoft.com = -- Our newest site for the community: CakePHP Video Tutorials http://tv.cakephp.org Check out the new CakePHP Questions site http://ask.cakephp.org and help others with their CakePHP related questions. To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php
Memory Leak in Model::Save()
Hello Fellow Bakers! I have developed an application to generate a family tree. In one of my models, I have attached Tree Behaviour. However, during my development, I encountered a memory exhaustion. I noticed that in some of my data sets, I have more than 1 million population to generate. The users of my application are fine with the length of time to generate the data, as long as they get the data. An email notification is done whenever a generation is complete. Each data set = 1 http request I was very careful not to have any memory leak. Every variable created was unset once it's no longer in use. I unbind all unnecessary models during the query and also specially select only fields I'll be using. Finally, I used xdebug to monitor my memory usage by breaking points whenever I invoke memory_get_usage(). I noticed that whenever I do a $this->model->save(), the memory usage increases by 2000bytes on average. Even after I did a $this->model->create(), the memory usage did not reduce. While this amount is considerably low, during huge data sets with many iterations, these bytes really add up. I need advice on these areas: 1) I want to identify where memory leak is. May it be MySQL or CakePHP or my own mistake. 2) I want to clear the memory whenever I do a Model::Save(). 3) I suspect a memory leak happens on Model::Find() too. I have previously encountered a similar situation with my Financial App (Stock Data) at http://groups.google.com/group/cake-php/browse_thread/thread/3f4cfb807b5efb08/dce96997c3c3eb96?lnk=gst&q=memory+leak#dce96997c3c3eb96 I appreciate the many advices provided in that post and it has assisted me to look into other work arounds. Thanks for the fantastic community here at CakePHP I look forward to everyone's response again! =D -- Our newest site for the community: CakePHP Video Tutorials http://tv.cakephp.org Check out the new CakePHP Questions site http://ask.cakephp.org and help others with their CakePHP related questions. To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php
