Thanks Daniel for the response, i am using PHP 5.3.2-1ubuntu4.5 with Suhosin-Patch (cli) (built: Sep 17 2010 13:41:55), read some bug reports related to circula reference and the garbage colector, but they suld be fixed for 5.3.
OnDemand hydration did't help, i am currently doing it, i am procesing batches of 100 objects inside a method and i can currently hold the objects but i only can call this method about 18 times until it run out of memory. Incresing memory of the server may be a valid option but is not sustentable, so i like to try other options. gc_collect_cycles may help a little but didn't change the situation. The only options is to implement a Fork/Subtask (Following the sample provided by Rich). It's a shame since i could do this on propel with any problem :(. 2010/11/16 Daniel Lohse <annismcken...@googlemail.com>: > That's unfortunate because you really have few options — I know because I've > been there just this week. > > So, array hydration is out of the question, fine, let's see what other > options you have: > > 1. Increase memory of the server (sorry, but it's one option) > > 2. Use the latest PHP 5.3.x and try to call "gc_collect_cycles();" after each > iteration — this will slow down your script some but could help. > Unfortunately, while garbage collection has certainly improved with PHP 5.3, > you can't rely on this because PHP is not forced to garbage collect and > sometimes can't; tricky stuff. > > 3. Fork another process after, say, 1000 objects. Each PHP process has its > own memory limit. I know of one project that uses this — you'll have to go > look for it, though. The update-search-index task of the Apostrophe CMS is > using this technique to work around this problem, you can find the source on > svn.apostrophenow.org. Sorry for not being more specific. :( > > 4. Only process a given amount of objects per call and work your way through > the queue, flagging objects you've finished processing along the way. > > 5. Try to use the OnDemand object hydrator and see if that helps. > > If you already tried all options I listed above (and failed ;-)), you'll need > to tell us more about the process, about why they don't/won't work — maybe we > can then come up with a solution. > > > I feel your pain, cheers, Daniel > > Sent from my iPad > > On Nov 16, 2010, at 8:48 PM, "Luciano A. Andrade" <andrade.luci...@gmail.com> > wrote: > >> I am tyring to make a batch processing for synchronization of two >> databases, and i am running on limitation of the memory usage. I like >> to be able to process 30000 objects, i can't use arrays for the >> hydration, since i have some logic i need to execute. I only need one >> of this object at any time, but it looks like theres no way to >> liberate the used objects, so memory usage grow rapidly. Any one has >> be able to free some memory, i am using >> $object->free(true); >> unset($object); >> >> so i run out of ideas. Any one? >> >> -- >> If you want to report a vulnerability issue on symfony, please send it to >> security at symfony-project.com >> >> You received this message because you are subscribed to the Google >> Groups "symfony users" group. >> To post to this group, send email to symfony-users@googlegroups.com >> To unsubscribe from this group, send email to >> symfony-users+unsubscr...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/symfony-users?hl=en > > -- > If you want to report a vulnerability issue on symfony, please send it to > security at symfony-project.com > > You received this message because you are subscribed to the Google > Groups "symfony users" group. > To post to this group, send email to symfony-users@googlegroups.com > To unsubscribe from this group, send email to > symfony-users+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/symfony-users?hl=en > -- If you want to report a vulnerability issue on symfony, please send it to security at symfony-project.com You received this message because you are subscribed to the Google Groups "symfony users" group. To post to this group, send email to symfony-users@googlegroups.com To unsubscribe from this group, send email to symfony-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/symfony-users?hl=en
