Hi,
> I use a single http request.
Ah and that is your problem. PHP and web servers in general are not
designed to execute long running tasks.
I do batch processing in cakephp using a javascript timer that invokes
an ajax call to cake every 2 seconds. Cake then executes a small portion
of the enormous batch task and logs that. Since this is the type of task
you don't want to rush it works very well for me.
It is sure is a matter of taste whether or not you think such a solution
is pretty and I don't know whether or not it is appropriate in your case.
Regards,
Maurits
On 02/22/2011 01:07 AM, ProFire wrote:
I use a single http request.
On Feb 21, 11:32 pm, Maurits van der Schee<maur...@vdschee.nl> wrote:
Hi,
Do you use a single http request or multiple invoked by a javascript timer?
Regards,
Maurits
On 02/21/2011 02:49 PM, ProFire wrote:
Hi fellowcakephpers,
I've been a long time user ofCakePHPand I've been satisfied since I
first tried it. In fact, there's no more turning back for me ever> since I
started it. No framework matchesCakePHPwhen it comes to ease
of development.
However, very recently, I've encountered a problem I can't figure out
the source. I'm dealing with Financial Data and being in finance, my
application often have to crunch huge sets of data. I've always been
very careful with how my application has handled the data as the data
involve is huge and I could run into a memory leak if I don't clear
those unused variables.
This year, I was tasked to run a very heavy simulation on the
financial data that involves possible 100 million mysql queries in a
single run. As such, I'm very prepared to let the simulation run over
a period of 1 month. However, within 2 days, the application threw a
memory exhausted error. What really puzzled me was I had been very
careful not to store any unused data in memory.
In my algorithm, after I query the data, I store them in a temporary
variable. At its final usage, I unset the variable despite knowing
that the next iteration the data will be overwritten. That's just to
be sure. After each round of simulation, the variables used are stored
in the database. After the last $model->save(), I clear every variable
used in the simulation, even if the data will be overwritten at the
next iteration.
All other persistent data throughout the iteration are either
integers, floats or unchanging arrays. As such, there's no way these
persistent data could be the cause of memory leak.
I've debugged as much as I could to pin point the source of the memory
leak in my controllers and models, but without any luck. I ran a
smaller simulation and monitor the memory size each iteration, I
noticed that the memory either stays the same or gets bigger.
I still put my faith inCakePHPand I need expert advise on where this
memory leak could be.
--
Our newest site for the community: CakePHP Video Tutorials http://tv.cakephp.org
Check out the new CakePHP Questions site http://ask.cakephp.org and help others with their CakePHP related questions.
To unsubscribe from this group, send email to
cake-php+unsubscr...@googlegroups.com For more options, visit this group at
http://groups.google.com/group/cake-php
