I totally agree with other guys about demonizing it and making the process being able to pause, maybe restart the computer and contine from where you have paused.
So I am writting to express wory about a precision of your results. I am not an old cake user and I haven't had a chance to do complex math functions in cake, but having in mind you have so many results you need to go through a simple 0.0000000000000001 mistake could multiply to something huge. Esepecialy if you have numbers with a lot of figures eg 123456789123.123456789123456789 Now in some cases floating numbers here just start to loose last figures and you don't even get that reported about that. I think it's cool you are trying to do this in cake because you will proove that anything is possible but I would without a hesitation check the results in math lab - software which actually is programmed , optimased and tested for such calculations and also does them more quickly. Also I believe - not sure - that you can "rent" supercomputers with matlab or smth like that like cloud computing and get your job done even quicker. I hope i didn't bother you too much :) All the best, Milos On Feb 22, 7:51 am, Maurits van der Schee <maur...@vdschee.nl> wrote: > Hi, > > > I use a single http request. > > Ah and that is your problem. PHP and web servers in general are not > designed to execute long running tasks. > > I do batch processing in cakephp using a javascript timer that invokes > an ajax call to cake every 2 seconds. Cake then executes a small portion > of the enormous batch task and logs that. Since this is the type of task > you don't want to rush it works very well for me. > > It is sure is a matter of taste whether or not you think such a solution > is pretty and I don't know whether or not it is appropriate in your case. > > Regards, > > Maurits > > On 02/22/2011 01:07 AM, ProFire wrote: > > > I use a single http request. > > > On Feb 21, 11:32 pm, Maurits van der Schee<maur...@vdschee.nl> wrote: > >> Hi, > > >> Do you use a single http request or multiple invoked by a javascript timer? > > >> Regards, > > >> Maurits > > >> On 02/21/2011 02:49 PM, ProFire wrote: > > >>> Hi fellowcakephpers, > > >>> I've been a long time user ofCakePHPand I've been satisfied since I > >>> first tried it. In fact, there's no more turning back for me ever> since > >>> I started it. No framework matchesCakePHPwhen it comes to ease > >>> of development. > > >>> However, very recently, I've encountered a problem I can't figure out > >>> the source. I'm dealing with Financial Data and being in finance, my > >>> application often have to crunch huge sets of data. I've always been > >>> very careful with how my application has handled the data as the data > >>> involve is huge and I could run into a memory leak if I don't clear > >>> those unused variables. > > >>> This year, I was tasked to run a very heavy simulation on the > >>> financial data that involves possible 100 million mysql queries in a > >>> single run. As such, I'm very prepared to let the simulation run over > >>> a period of 1 month. However, within 2 days, the application threw a > >>> memory exhausted error. What really puzzled me was I had been very > >>> careful not to store any unused data in memory. > > >>> In my algorithm, after I query the data, I store them in a temporary > >>> variable. At its final usage, I unset the variable despite knowing > >>> that the next iteration the data will be overwritten. That's just to > >>> be sure. After each round of simulation, the variables used are stored > >>> in the database. After the last $model->save(), I clear every variable > >>> used in the simulation, even if the data will be overwritten at the > >>> next iteration. > > >>> All other persistent data throughout the iteration are either > >>> integers, floats or unchanging arrays. As such, there's no way these > >>> persistent data could be the cause of memory leak. > > >>> I've debugged as much as I could to pin point the source of the memory > >>> leak in my controllers and models, but without any luck. I ran a > >>> smaller simulation and monitor the memory size each iteration, I > >>> noticed that the memory either stays the same or gets bigger. > > >>> I still put my faith inCakePHPand I need expert advise on where this > >>> memory leak could be. -- Our newest site for the community: CakePHP Video Tutorials http://tv.cakephp.org Check out the new CakePHP Questions site http://ask.cakephp.org and help others with their CakePHP related questions. To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php
