Re: [PHP] free allocated memory: HOW ?
Richard Lynch wrote: On Fri, April 13, 2007 12:54 am, Arthur Erdös wrote: It can if you're trying to process a borked image... I've had imagecreatefromjpeg() eat memory up to almost 50x the size of the image before finally deciding it can't handle it and crapping out. That was *after* running it through getimagesize() with no problem at all. okay - good point - but in this case the OP is reading in an html template file, just a string of 20.7Kb, what could go wrong there? this is probably one issue... the template is an HTML template containing images like http://www.brainguide.com/xxx"/>. How does PHP handle this stuff? I suppose that it is just a string and the image at the URL is not really read, or am I wrong? The newsletter does not have any attachements. Now you're not making any sense at all... Either your newsletter has the images in it, or it doesn't. Embedded images are different to images with a href. To embed images, you need to read them in to memory and attach them (much the same as a regular attachment). Referencing a href like that doesn't do anything, it's just part of the string. -- Postgresql & php tutorials http://www.designmagick.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
On Fri, April 13, 2007 12:49 am, Arthur Erdös wrote:
> $abonents = $personAction->findAbonents($start,
> $offset);
> $size = $personAction->countFoundRows();
> $this->logger->debug("Found " . $size . "
> abonents");
> $loops = ceil($size/self::PROCESS_ABONENTS);
> $this->logger->debug("Need " . $loops . " loops at " .
> self::PROCESS_ABONENTS . " abonents per loop");
> for($i = 0; $i < $loops; $i++){
> $start = $i*self::PROCESS_ABONENTS;
> $abonents = $personAction->findAbonents($start,
> self::PROCESS_ABONENTS);
What happened to the other $abonents you set about 6 lines ago?...
> // unset($text, $lvo);
> $text = null;
> $lvo = null;
> sleep(1);
Do the unset() as well on each var, I think.
--
Some people have a "gift" link here.
Know what I want?
I want you to buy a CD from some indie artist.
http://cdbaby.com/browse/from/lynch
Yeah, I get a buck. So?
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
On Fri, April 13, 2007 12:54 am, Arthur Erdös wrote: >>> It can if you're trying to process a borked image... >>> >>> I've had imagecreatefromjpeg() eat memory up to almost 50x the size >>> of >>> the image before finally deciding it can't handle it and crapping >>> out. >>> That was *after* running it through getimagesize() with no problem >>> at all. >> >> okay - good point - but in this case the OP is reading in an html >> template file, >> just a string of 20.7Kb, what could go wrong there? >> > > this is probably one issue... the template is an HTML template > containing images like http://www.brainguide.com/xxx"/>. How > does PHP handle this stuff? I suppose that it is just a string and the > image at the URL is not really read, or am I wrong? > > The newsletter does not have any attachements. Now you're not making any sense at all... Either your newsletter has the images in it, or it doesn't. I suspect that it's set up to take that newsletter *AND* all the images, and bundle that up into a nice *BIG* HTML Enhanced (cough, cough) email, and send that out... If you don't free up that template, that could be a large chunk of RAM right there. -- Some people have a "gift" link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
Roman Neuhauser wrote: > # [EMAIL PROTECTED] / 2007-04-12 18:17:34 +0200: >> 3-4 seconds is dead slow if you ask me - a script like this should be >> capable of >> making the average mailserver go completely apeshit assuming you'd be >> mailing the >> newsletters out directly after creating them [rather than storing them in a >> db] (i.e. >> pumping out lots of emails a second] ... I know this because I have a qmail >> server >> that has trouble keeping up with my massmailer scripts. > > switch po Postfix, one problem fewer for you. yeah or exim, only I can't because it's one of the darn plesk servers - and changing out the mail server is not something I can do. > -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
> Reading in the template should only happen once and take roughly as much > memory as the file is big. > > If the file is 20k, memory should go up roughly by that much. > > If it's going up a lot more, something is going wrong (either you're > doing something wrong or you're not just reading the template - you're > doing something else along with that). > i completely agree with you ;) i will check if the template is really the reason for the big jump with logging the memory usage each line. apart from that, i am still wondering why the memory usage grows in each loop for about 30 megs... -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
Arthur Erdös wrote:
Am Freitag, den 13.04.2007, 16:03 +1000 schrieb Chris:
ok, the complete workflow is a little bit complicated. we are using a
workflow engine and the newsletter generator is one step of three. the
first cleans the statistics data, the second generates the new data and
the third is the one which generates the mails.
Put this in between each line and see where your memory is jumping the most:
error_log('in file ' . __FILE__ . ' at line ' . __LINE__ . '; memory
usage is ' . number_format((memory_get_usage() / 1024), 4) . ' kb');
Once you've worked out the biggest jumps, fix them, then work on the
next one and so on.
It's going to be a lot quicker doing something like that over asking us
to help you work out thousands of lines of code.
sure, i just posted the code because i was asked to ;)
i figured out the biggest jump (after reading the template) but could
not fix it yet -.-
Reading in the template should only happen once and take roughly as much
memory as the file is big.
If the file is 20k, memory should go up roughly by that much.
If it's going up a lot more, something is going wrong (either you're
doing something wrong or you're not just reading the template - you're
doing something else along with that).
--
Postgresql & php tutorials
http://www.designmagick.com/
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
# [EMAIL PROTECTED] / 2007-04-13 07:49:43 +0200: > > > Just post your source already. > > > > ok, the complete workflow is a little bit complicated. we are using a > workflow engine and the newsletter generator is one step of three. the > first cleans the statistics data, the second generates the new data and > the third is the one which generates the mails. > > here is the concerning method: aren't you suffering from fetching huge result sets? just a guess. -- How many Vietnam vets does it take to screw in a light bulb? You don't know, man. You don't KNOW. Cause you weren't THERE. http://bash.org/?255991 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
Am Freitag, den 13.04.2007, 16:03 +1000 schrieb Chris:
> > ok, the complete workflow is a little bit complicated. we are using a
> > workflow engine and the newsletter generator is one step of three. the
> > first cleans the statistics data, the second generates the new data and
> > the third is the one which generates the mails.
>
> Put this in between each line and see where your memory is jumping the most:
>
> error_log('in file ' . __FILE__ . ' at line ' . __LINE__ . '; memory
> usage is ' . number_format((memory_get_usage() / 1024), 4) . ' kb');
>
> Once you've worked out the biggest jumps, fix them, then work on the
> next one and so on.
>
>
> It's going to be a lot quicker doing something like that over asking us
> to help you work out thousands of lines of code.
>
sure, i just posted the code because i was asked to ;)
i figured out the biggest jump (after reading the template) but could
not fix it yet -.-
thats why I asked a general question concerning php memory handling.
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
# [EMAIL PROTECTED] / 2007-04-12 18:17:34 +0200: > 3-4 seconds is dead slow if you ask me - a script like this should be capable > of > making the average mailserver go completely apeshit assuming you'd be mailing > the > newsletters out directly after creating them [rather than storing them in a > db] (i.e. > pumping out lots of emails a second] ... I know this because I have a qmail > server > that has trouble keeping up with my massmailer scripts. switch po Postfix, one problem fewer for you. -- How many Vietnam vets does it take to screw in a light bulb? You don't know, man. You don't KNOW. Cause you weren't THERE. http://bash.org/?255991 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
Arthur Erdös wrote: It can if you're trying to process a borked image... I've had imagecreatefromjpeg() eat memory up to almost 50x the size of the image before finally deciding it can't handle it and crapping out. That was *after* running it through getimagesize() with no problem at all. okay - good point - but in this case the OP is reading in an html template file, just a string of 20.7Kb, what could go wrong there? this is probably one issue... the template is an HTML template containing images like http://www.brainguide.com/xxx"/>. How does PHP handle this stuff? I suppose that it is just a string and the image at the URL is not really read, or am I wrong? It's just part of the string of content unless you do something special with it. -- Postgresql & php tutorials http://www.designmagick.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
ok, the complete workflow is a little bit complicated. we are using a
workflow engine and the newsletter generator is one step of three. the
first cleans the statistics data, the second generates the new data and
the third is the one which generates the mails.
Put this in between each line and see where your memory is jumping the most:
error_log('in file ' . __FILE__ . ' at line ' . __LINE__ . '; memory
usage is ' . number_format((memory_get_usage() / 1024), 4) . ' kb');
Once you've worked out the biggest jumps, fix them, then work on the
next one and so on.
It's going to be a lot quicker doing something like that over asking us
to help you work out thousands of lines of code.
--
Postgresql & php tutorials
http://www.designmagick.com/
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
It can if you're trying to process a borked image... I've had imagecreatefromjpeg() eat memory up to almost 50x the size of the image before finally deciding it can't handle it and crapping out. That was *after* running it through getimagesize() with no problem at all. okay - good point - but in this case the OP is reading in an html template file, just a string of 20.7Kb, what could go wrong there? this is probably one issue... the template is an HTML template containing images like http://www.brainguide.com/xxx"/>. How does PHP handle this stuff? I suppose that it is just a string and the image at the URL is not really read, or am I wrong? The newsletter does not have any attachements. Ed --PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
> Just post your source already.
>
ok, the complete workflow is a little bit complicated. we are using a
workflow engine and the newsletter generator is one step of three. the
first cleans the statistics data, the second generates the new data and
the third is the one which generates the mails.
here is the concerning method:
/**
* This method generates the monthly newsletter with the
statistics
block for the specified period.
*
* @param NewsletterValue $vo Initialized NewsletterValue object
* @param Integer $dateFrom Timestamp representing the start
date of
the period
* @param Integer $dateTo Timestamp representing the end date
of the
period
*/
private final function generateMonthlyNewsletter($vo, $dateFrom,
$dateTo){
$this->logger->debug("Generating monthly newsletter
with statistics
blocks.");
// initialize variable for the abonents
$abonents = null;
// read the newsletter template
$asset = $vo->getAsset();
$theData = self::readTemplate("../../../" .
$this->templatePath .
$asset->getFileName());
$this->logger->debug("successfully read template,
MEMORY USAGE: " .
memory_get_usage()/1048576 . " MB");
// iterate over the abonents and generate customized
newsletter
content
$counter = 0;
$start = 0;
$offset = self::PROCESS_ABONENTS;
// get the first newsletter abonents
$personAction = new PersonEPBAction();
$abonents = $personAction->findAbonents($start,
$offset);
$size = $personAction->countFoundRows();
$this->logger->debug("Found " . $size . "
abonents");
$loops = ceil($size/self::PROCESS_ABONENTS);
$this->logger->debug("Need " . $loops . " loops at " .
self::PROCESS_ABONENTS . " abonents per loop");
for($i = 0; $i < $loops; $i++){
$start = $i*self::PROCESS_ABONENTS;
$abonents = $personAction->findAbonents($start,
self::PROCESS_ABONENTS);
$this->logger->debug("Loop " . $i . ", MEMORY
USAGE: " .
memory_get_usage()/1048576 . " MB");
foreach($abonents as $person){
$this->logger->debug("Generating mail
for " .
$person->getPersonId() . ", " . $person->getFullName());
// replace custom person fields
$text =
self::replaceTemplateVars($theData, $person, $dateFrom,
$dateTo);
// add statistics
$text =
self::addStatisticsBlocks($text, $person);
// add the text to the
person_newsletter table
$lvo = new PersonNewsletterLightValue();
$lvo->setNewsletterIdFk($vo->getNewsletterId());
$lvo->setPersonIdFk($person->getPersonId());
$lvo->setMimeTypeIdFk(2);
// text/html TODO: refactor!
$lvo->setName("brainGuide AG"); //
TODO: refactor!
$lvo->setEmail("[EMAIL PROTECTED]"); //
TODO: refactor!
$lvo->setMailto($person->getEmailBusiness());
$lvo->setSubject($vo->getSubject());
$lvo->setText($text);
$lvo->setTstamp(time());
$lvo->setSent(0);
$lvo->setViewed(0);
self::savePersonNewsletter($lvo);
// unset variables
// unset($text, $lvo);
$text = null;
$lvo = null;
sleep(1);
}
}
}
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
Arthur Erdös wrote:
Hello all,
is there a way to free memory allocated by variables in PHP?? This is a
very important issue concerning long running scripts...
I have a script that generates > 5000 Newsletters and when the script
finishes it uses 1.8 GB (!!) of RAM. Although I am using unset() to
clean up variables (tried with $var = null too).
I'd place stuff like:
error_log('in file ' . __FILE__ . ' at line ' . __LINE__ . '; memory
usage is ' . (memory_get_usage() / 1024) . ' kb');
all over the place and work out where it's jumping, then work out why
it's jumping so much.
Work out your biggest jumps first (they will have the most benefit),
then work out smaller jumps and slowly work down the chain until it
stays at a reasonable level.
Also as Richard pointed out, if you're including images and/or
attachments in a newsletter, that increases memory a lot. You need to
base64_encode the image/attachment before it will work in an email
program - and encoding it takes up around an extra 30% over the original
data (see http://php.net/base64_encode).
So what starts out as a 100k image will most likely end up as almost
130k+ in memory.
--
Postgresql & php tutorials
http://www.designmagick.com/
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
On Thu, April 12, 2007 11:34 am, Arthur Erdös wrote: > no email goes out, the mails are stored in the database. http://mysql_free_result/ There may be something similar for the INSERT statements, or maybe there's nothing there to free up... But I suspect that you are keeping a *TON* of MySQL data in your PHP script. Of course, if you aren't getting rid of the $newsletter itself, that would also be the problem. Just post your source already. -- Some people have a "gift" link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
[combining responses]
On Thu, April 12, 2007 8:13 am, Arthur Erdös wrote:
> any ideas what i am doing wrong and where i [ab]use php? ^^
Not showing us source code is your biggest mistake... :-)
On Thu, April 12, 2007 9:34 am, Zoltán Németh wrote:
>> site/company I think we are okay on that front. oh but I loved the
>> way the HP ad on 'his' homepage almost ate all
>> my RAM before firefox finally offered to kill the script that was
>> going nuts ('because Flash was running slowly' ...
>> yeah, not to mention the rest of my machine) ... seems the OP really
>> has a problem with memory consumption ;-).
>>
>
> AdBlock Plus killed it for me without any single notice ;)
Not installing Flash at all every again has worked well for me. :-)
--
Some people have a "gift" link here.
Know what I want?
I want you to buy a CD from some indie artist.
http://cdbaby.com/browse/from/lynch
Yeah, I get a buck. So?
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
On Thu, April 12, 2007 6:40 am, Arthur Erdös wrote: > is there a way to free memory allocated by variables in PHP?? This is > a > very important issue concerning long running scripts... > > I have a script that generates > 5000 Newsletters and when the script > finishes it uses 1.8 GB (!!) of RAM. Although I am using unset() to > clean up variables (tried with $var = null too). > > I've read sth about php not giving the allocated memory back to the OS > until a script finishes, is that right?? Is it possible to free the > memory "by hand"? PHP garbage collector CAN free up some unused memory, if your code makes it patently clear that you will no longer be using that data. And if php.ini settings are reasonable. You also need to be clear about how you measured 1.8 G -- What you are looking at may be a record of how much RAM PHP used at the maximum, for some particularly long/nasty Newsletter with attachments and photos or something... -- Some people have a "gift" link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
On Apr 12, 2007, at 3:04 PM, Jochem Maas wrote: Edward Vermillion wrote: On Apr 12, 2007, at 11:17 AM, Jochem Maas wrote: Arthur Erdös wrote: int memory_get_usage ( [bool $real_usage] ) Returns the amount of memory, in bytes, that's currently being allocated to your PHP script. [snip] php will not turn a 20.7Kb file into 400+ Megs of consumed memory - unless your read and store it in memory 2 odd times. It can if you're trying to process a borked image... I've had imagecreatefromjpeg() eat memory up to almost 50x the size of the image before finally deciding it can't handle it and crapping out. That was *after* running it through getimagesize() with no problem at all. okay - good point - but in this case the OP is reading in an html template file, just a string of 20.7Kb, what could go wrong there? I got the impression that the OP hadn't really narrowed down where the memory gobbling portion of the script was and was just looking at an overall memory usage of the total process. Then the mention of newsletter and I immediately thought image. Although a later post I read made it sound like a memory leak in the code, maybe a variable that's holding data for some operation, or an object that's created each iteration, that's not being emptied or reused or destroyed before the next go-round. Leading to a slow consumption of memory over time. Ed -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
Arthur Erdös wrote: >> don't OMFG me - I can't read your cmdline from here you know, >> and I'm not the one with a completely borked script/system/whatever. >> > > a big sorry again for the missunderstanding!! ;) I did not omfg YOU! I > did omfg ME when I saw that the script eats much more memory as > memory_get_usage() returned!! so please excuse me if my wording was > wrong, i'm not a native english speaker oO ah I see. (guessing from my name you can probably assume I'm not native either) (I'm not a speaker either - I'm missing a magnet and a cone. ;-) ... >>> // read the newsletter template >>> $fh = fopen($file, 'r'); >>> $theData = fread($fh, filesize($file)); >>> fclose($fh); >>> return $theData; >> which can more succinctly be written as: >> >> return file_get_contents($file); >> > > I'll try to change fread to file_get_contents()... > > anyhow, each generation of 100 mails eats approx 30 more megs :( this is just not right - I can't believe the problem is not in your code somehow. time to do some simple tests: to start with write a test script that *only* reads in the relevant file and see what happens (checking memory usage before and after you read it in). ... > > no email goes out, the mails are stored in the database. yes so you said, but your script should be capable of generating them so fast that if you *were* to send them out straight away the mail server would get *really* annoyed .. at one every 3-4 seconds the mail server would laugh at you. if you catch my drift. > -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
Edward Vermillion wrote: > > On Apr 12, 2007, at 11:17 AM, Jochem Maas wrote: > >> Arthur Erdös wrote: int memory_get_usage ( [bool $real_usage] ) Returns the amount of memory, in bytes, that's currently being allocated to your PHP script. it returns the number of *bytes* !!! >>> >>> sure, and memory_get_usage()/1048576 returns the number of megs, or am i >>> wrong? o_O >>> >> exactly how much ram does the server in question have that you can >> allow a single script to schlock up almost 2gigs ?? > dev machine 4 GB, webserver 8 GB of RAM does running top in the cmdline confirm that the script is really eating 100's of Megs? >>> >>> OMFG, YES it does... top shows the script eating 658 megs right after >>> reading the template (~125 megs before) >> >> don't OMFG me - I can't read your cmdline from here you know, >> and I'm not the one with a completely borked script/system/whatever. >> >> php will not turn a 20.7Kb file into 400+ Megs of consumed memory - >> unless your read >> and store it in memory 2 odd times. >> > > It can if you're trying to process a borked image... > > I've had imagecreatefromjpeg() eat memory up to almost 50x the size of > the image before finally deciding it can't handle it and crapping out. > That was *after* running it through getimagesize() with no problem at all. okay - good point - but in this case the OP is reading in an html template file, just a string of 20.7Kb, what could go wrong there? > > Ed > --PHP General Mailing List (http://www.php.net/) > To unsubscribe, visit: http://www.php.net/unsub.php > -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
Arthur Erdös wrote: Arthur Erdös wrote: ok, its not really fast, but it runs on my development machine (AMD64, 4 GB of RAM, Ubuntu). mysql on the same machine, database with 175 tables and 2,3 GB data. that could probably slow down scripts, don't you agree? no. you have 4 F'ing gigs of ram on your desktop and it's a freakin' DB - they are supposed to handle *massive* ammounts of data ... you might consider analysing the queries you have and figuring out what an INDEX is for. I suppose the database is not the problem, the indexes and foreign keys are all set correctly. we have the mysql enterprise version and had a mysql consultant here some time ago and our database is well designed. Ok, so when this database was "well designed" by the "mysql consultant" did you tell them about all the queries you would be running against it? If not, your statement oozes a lack of understanding concerning indexes and general database optimization. If that's the case, start Googling. Now. Before you write any more code. -Stut -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
On Apr 12, 2007, at 11:17 AM, Jochem Maas wrote: Arthur Erdös wrote: int memory_get_usage ( [bool $real_usage] ) Returns the amount of memory, in bytes, that's currently being allocated to your PHP script. it returns the number of *bytes* !!! sure, and memory_get_usage()/1048576 returns the number of megs, or am i wrong? o_O exactly how much ram does the server in question have that you can allow a single script to schlock up almost 2gigs ?? dev machine 4 GB, webserver 8 GB of RAM does running top in the cmdline confirm that the script is really eating 100's of Megs? OMFG, YES it does... top shows the script eating 658 megs right after reading the template (~125 megs before) don't OMFG me - I can't read your cmdline from here you know, and I'm not the one with a completely borked script/system/whatever. php will not turn a 20.7Kb file into 400+ Megs of consumed memory - unless your read and store it in memory 2 odd times. It can if you're trying to process a borked image... I've had imagecreatefromjpeg() eat memory up to almost 50x the size of the image before finally deciding it can't handle it and crapping out. That was *after* running it through getimagesize() with no problem at all. Ed -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
> Arthur Erdös wrote: > > ok, its not really fast, but it runs on my development machine (AMD64, 4 > > GB of RAM, Ubuntu). mysql on the same machine, database with 175 tables > > and 2,3 GB data. that could probably slow down scripts, don't you agree? > > no. you have 4 F'ing gigs of ram on your desktop and it's a freakin' DB - > they are > supposed to handle *massive* ammounts of data ... you might consider > analysing the > queries you have and figuring out what an INDEX is for. I suppose the database is not the problem, the indexes and foreign keys are all set correctly. we have the mysql enterprise version and had a mysql consultant here some time ago and our database is well designed. (enterprise version only on webserver but anyway...) -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
> don't OMFG me - I can't read your cmdline from here you know, > and I'm not the one with a completely borked script/system/whatever. > a big sorry again for the missunderstanding!! ;) I did not omfg YOU! I did omfg ME when I saw that the script eats much more memory as memory_get_usage() returned!! so please excuse me if my wording was wrong, i'm not a native english speaker oO Far from it! I am greatly thankfull for people like you and the community here! > php will not turn a 20.7Kb file into 400+ Megs of consumed memory - unless > your read > and store it in memory 2 odd times. > > which still leaves the question as to what is taking 125 Megs of RAM *before* > you > even start processing. > > > > > the read method does following: > > > > // read the newsletter template > > $fh = fopen($file, 'r'); > > $theData = fread($fh, filesize($file)); > > fclose($fh); > > return $theData; > > which can more succinctly be written as: > > return file_get_contents($file); > I'll try to change fread to file_get_contents()... anyhow, each generation of 100 mails eats approx 30 more megs :( > > > >>> the size of the template is 20.7 Kb > >> no way in hell that 20.7Kb turns into 270 Megs if you read the file into > >> php. > >> > >> are you experiencing severe load on the server whilst the script is > >> running? > >> and/or is the script actually very, very slow? > > > > well, the script is not very, very slow... it takes ~3-4 seconds to > > generate each newsletter (including database querys for the customized > > data) > > 3-4 seconds is dead slow if you ask me - a script like this should be capable > of > making the average mailserver go completely apeshit assuming you'd be mailing > the > newsletters out directly after creating them [rather than storing them in a > db] (i.e. > pumping out lots of emails a second] ... I know this because I have a qmail > server > that has trouble keeping up with my massmailer scripts. > no email goes out, the mails are stored in the database. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
Arthur Erdös wrote: > ok, its not really fast, but it runs on my development machine (AMD64, 4 > GB of RAM, Ubuntu). mysql on the same machine, database with 175 tables > and 2,3 GB data. that could probably slow down scripts, don't you agree? no. you have 4 F'ing gigs of ram on your desktop and it's a freakin' DB - they are supposed to handle *massive* ammounts of data ... you might consider analysing the queries you have and figuring out what an INDEX is for. > > >> On Thu, 2007-04-12 at 17:59 +0200, Arthur Erdös wrote: are you experiencing severe load on the server whilst the script is running? and/or is the script actually very, very slow? >>> well, the script is not very, very slow... it takes ~3-4 seconds to >>> generate each newsletter (including database querys for the customized >>> data) >> I would consider that slow :| >> >> Cheers, >> Rob. > -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
Arthur Erdös wrote: >> int memory_get_usage ( [bool $real_usage] ) >> >> Returns the amount of memory, in bytes, that's currently being allocated to >> your PHP script. >> >> it returns the number of *bytes* !!! > > sure, and memory_get_usage()/1048576 returns the number of megs, or am i > wrong? o_O > exactly how much ram does the server in question have that you can allow a single script to schlock up almost 2gigs ?? >>> dev machine 4 GB, webserver 8 GB of RAM >> does running top in the cmdline confirm that the script is really eating >> 100's of Megs? >> > > OMFG, YES it does... top shows the script eating 658 megs right after > reading the template (~125 megs before) don't OMFG me - I can't read your cmdline from here you know, and I'm not the one with a completely borked script/system/whatever. php will not turn a 20.7Kb file into 400+ Megs of consumed memory - unless your read and store it in memory 2 odd times. which still leaves the question as to what is taking 125 Megs of RAM *before* you even start processing. > > the read method does following: > > // read the newsletter template > $fh = fopen($file, 'r'); > $theData = fread($fh, filesize($file)); > fclose($fh); > return $theData; which can more succinctly be written as: return file_get_contents($file); > >>> the size of the template is 20.7 Kb >> no way in hell that 20.7Kb turns into 270 Megs if you read the file into php. >> >> are you experiencing severe load on the server whilst the script is running? >> and/or is the script actually very, very slow? > > well, the script is not very, very slow... it takes ~3-4 seconds to > generate each newsletter (including database querys for the customized > data) 3-4 seconds is dead slow if you ask me - a script like this should be capable of making the average mailserver go completely apeshit assuming you'd be mailing the newsletters out directly after creating them [rather than storing them in a db] (i.e. pumping out lots of emails a second] ... I know this because I have a qmail server that has trouble keeping up with my massmailer scripts. > -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
ok, its not really fast, but it runs on my development machine (AMD64, 4 GB of RAM, Ubuntu). mysql on the same machine, database with 175 tables and 2,3 GB data. that could probably slow down scripts, don't you agree? > On Thu, 2007-04-12 at 17:59 +0200, Arthur Erdös wrote: > > > > > > are you experiencing severe load on the server whilst the script is > > > running? > > > and/or is the script actually very, very slow? > > > > well, the script is not very, very slow... it takes ~3-4 seconds to > > generate each newsletter (including database querys for the customized > > data) > > I would consider that slow :| > > Cheers, > Rob. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
On Thu, 2007-04-12 at 17:59 +0200, Arthur Erdös wrote: > > > > are you experiencing severe load on the server whilst the script is running? > > and/or is the script actually very, very slow? > > well, the script is not very, very slow... it takes ~3-4 seconds to > generate each newsletter (including database querys for the customized > data) I would consider that slow :| Cheers, Rob. -- .. | InterJinn Application Framework - http://www.interjinn.com | :: | An application and templating framework for PHP. Boasting | | a powerful, scalable system for accessing system services | | such as forms, properties, sessions, and caches. InterJinn | | also provides an extremely flexible architecture for | | creating re-usable components quickly and easily. | `' -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
> int memory_get_usage ( [bool $real_usage] ) > > Returns the amount of memory, in bytes, that's currently being allocated to > your PHP script. > > it returns the number of *bytes* !!! sure, and memory_get_usage()/1048576 returns the number of megs, or am i wrong? o_O > > > > >> exactly how much ram does the server in question have that you can > >> allow a single script to schlock up almost 2gigs ?? > > > > dev machine 4 GB, webserver 8 GB of RAM > > does running top in the cmdline confirm that the script is really eating > 100's of Megs? > OMFG, YES it does... top shows the script eating 658 megs right after reading the template (~125 megs before) the read method does following: // read the newsletter template $fh = fopen($file, 'r'); $theData = fread($fh, filesize($file)); fclose($fh); return $theData; > > > > the size of the template is 20.7 Kb > > no way in hell that 20.7Kb turns into 270 Megs if you read the file into php. > > are you experiencing severe load on the server whilst the script is running? > and/or is the script actually very, very slow? well, the script is not very, very slow... it takes ~3-4 seconds to generate each newsletter (including database querys for the customized data) -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
Arthur Erdös wrote: > answering Jochem: > >> WTF - 386 megs for an html file??? somehow I doubt this is >> the correct number ... are you reading the memory consumption >> correctly? > > when the newsletter generation starts memory_get_usage() says ~96 MB in > use. So approx 290 megs more are used after reading the 20.7 Kb template > with fread()! I am reading the memory consumption with > memory_get_usage(), so I dont know what should be wrong with that... int memory_get_usage ( [bool $real_usage] ) Returns the amount of memory, in bytes, that's currently being allocated to your PHP script. it returns the number of *bytes* !!! > >> exactly how much ram does the server in question have that you can >> allow a single script to schlock up almost 2gigs ?? > > dev machine 4 GB, webserver 8 GB of RAM does running top in the cmdline confirm that the script is really eating 100's of Megs? > >> if the values are customized I can't fathom why you would generate >> 100 emails per loop. > > sorry, my fault, I meant I am generating 100 emails "once", sleep(2), > then the next 100, and so on... In my case (50xx mails - that results in > 51 "loops" a 100 mails). > so that's 10,000+ loops (iterations) with a sleep() performed every 100'th iteration. > > answering TG: > >> Couple of questions come to mind... >> >> 1. How big is this template that you're loading? Not sure why almost >> any HTML file would take up 300+ meg of memory when loaded into a >> variable. Are you loading images and other content in as well or just >> the HTML file? > > the size of the template is 20.7 Kb no way in hell that 20.7Kb turns into 270 Megs if you read the file into php. are you experiencing severe load on the server whilst the script is running? and/or is the script actually very, very slow? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
hihi, sorry for that! The Ads come from our Ad-Seller (or how is this
called in english) and we have nearly no influence on what kind of ads
are delivered ;)
Am Donnerstag, den 12.04.2007, 16:18 +0200 schrieb Jochem Maas:
> [EMAIL PROTECTED] wrote:
> > Couple of questions come to mind...
> >
>
> ...
>
> > 4. Please tell us these 'newsletters' don't contain information on "male
> > enhancement products" or "govt seized puppies"..hah
>
> this spring to my mind as well - but given his domainname (brainguide), which
> points to a seemingly respectable
> site/company I think we are okay on that front. oh but I loved the way the HP
> ad on 'his' homepage almost ate all
> my RAM before firefox finally offered to kill the script that was going nuts
> ('because Flash was running slowly' ...
> yeah, not to mention the rest of my machine) ... seems the OP really has a
> problem with memory consumption ;-).
>
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
answering Jochem: > WTF - 386 megs for an html file??? somehow I doubt this is > the correct number ... are you reading the memory consumption > correctly? when the newsletter generation starts memory_get_usage() says ~96 MB in use. So approx 290 megs more are used after reading the 20.7 Kb template with fread()! I am reading the memory consumption with memory_get_usage(), so I dont know what should be wrong with that... > exactly how much ram does the server in question have that you can > allow a single script to schlock up almost 2gigs ?? dev machine 4 GB, webserver 8 GB of RAM > if the values are customized I can't fathom why you would generate > 100 emails per loop. sorry, my fault, I meant I am generating 100 emails "once", sleep(2), then the next 100, and so on... In my case (50xx mails - that results in 51 "loops" a 100 mails). answering TG: Couple of questions come to mind... 1. How big is this template that you're loading? Not sure why almost any HTML file would take up 300+ meg of memory when loaded into a variable. Are you loading images and other content in as well or just the HTML file? the size of the template is 20.7 Kb 2. Replacing the placeholders with actual data increases memory that much? Damn.. maybe I'm just ignorant of how PHP maintains things in memory, but that seems kind of nuts too 3. Do you have to store each newsletter in memory at each loop or can you send them out then move to the next one without storing them in an array or whatever you're doing? I write each newsletter to a mysql database then move to the next 4. Please tell us these 'newsletters' don't contain information on "male enhancement products" or "govt seized puppies"..hah No, they don't ;) -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
2007. 04. 12, csütörtök keltezéssel 16.18-kor Jochem Maas ezt írta:
> [EMAIL PROTECTED] wrote:
> > Couple of questions come to mind...
> >
>
> ...
>
> > 4. Please tell us these 'newsletters' don't contain information on "male
> > enhancement products" or "govt seized puppies"..hah
>
> this spring to my mind as well - but given his domainname (brainguide), which
> points to a seemingly respectable
> site/company I think we are okay on that front. oh but I loved the way the HP
> ad on 'his' homepage almost ate all
> my RAM before firefox finally offered to kill the script that was going nuts
> ('because Flash was running slowly' ...
> yeah, not to mention the rest of my machine) ... seems the OP really has a
> problem with memory consumption ;-).
>
AdBlock Plus killed it for me without any single notice ;)
greets
Zoltán Németh
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
[EMAIL PROTECTED] wrote:
> Couple of questions come to mind...
>
...
> 4. Please tell us these 'newsletters' don't contain information on "male
> enhancement products" or "govt seized puppies"..hah
this spring to my mind as well - but given his domainname (brainguide), which
points to a seemingly respectable
site/company I think we are okay on that front. oh but I loved the way the HP
ad on 'his' homepage almost ate all
my RAM before firefox finally offered to kill the script that was going nuts
('because Flash was running slowly' ...
yeah, not to mention the rest of my machine) ... seems the OP really has a
problem with memory consumption ;-).
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
When you free memory you are only marking memory as available for reclaiming, but it doesn't mean it will actually reclaim it at all. The garbage collector usually runs at a very low priority and if you don't give your process a break, it will never catch up. One thing I would try is to insert a call to sleep() within the loop and see if that gets the garbage collector to do some catching up (and if it does, please let us know). Another thing would be to have a master process shelling out processes to process a template one at a time. It might either be either through exec() or using curl to request Apache to deal with handling the processes. The end of each individual process would free all its memory and, after all, that is the way PHP is meant to run and has been highly optimized to do. Using some shared memory to communicate whatever needs to be shared would avoid too much repetition. If the templates to be processed are large, perhaps you can get some external utility to process it far more efficiently. Nevertheless, I wouldn't try any of that until I exahusted all that can be done on the programming side. Are you using file_get_contents to read whole chunks at a time? Is it really necesary? Can you manage to read a line at a time? If having the whole template in memory is required, can you break it up into an array with explode() and operate on each line at a time or whatever unit you have? Or better yeat, have it read a line at a time into an array and avoid the explode(). Memory gets a better chance of being reused if it is split into many smaller chunks. Using square brackets to access and specially modify individual characters as if the string is an array of characters works on the string right in place, avoiding copying and trashing the memory, though it might not be faster. Anyway, these are no more than random thoughts and nothing beats a good application design, something that nowadays, with so much memory and powerful processors tends to be overlooked. Someone famous once said that 64kbytes was all the memory anyone would ever need. Satyam - Original Message - From: "Arthur Erdös" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]>; "Jochem Maas" <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Sent: Thursday, April 12, 2007 3:13 PM Subject: Re: [PHP] free allocated memory: HOW ? I am using PHP version 5.1.2. on my dev machine. The same problem with PHP 5.2.1 on webserver. I am generating mails based on a HTML Template. If you look at the appended screenshot you'll see that after reading the template with fread() memory_get_usage() says ~386 MB in use... Then the placeholders in the read template are replaced by customized values within a loop. The size of the used memory grows permanently in each loop by approx 30 MB. any ideas what i am doing wrong and where i [ab]use php? ^^ Jochem Maas schrieb: Arthur Erdös wrote: Hello all, is there a way to free memory allocated by variables in PHP?? This is a very important issue concerning long running scripts... this is a recurrent problem - not much can be done about it AFAIK ... I'd be very glad to be proved wrong. I have a script that generates > 5000 Newsletters and when the script finishes it uses 1.8 GB (!!) of RAM. Although I am using unset() to clean up variables (tried with $var = null too). okay this does suggest your doing something else very wrong - I personally have code that regularly generates upto 20,000 individualised newsletters without going anywhere near this ammount of memory - granted I've hit my memory limit a couple of times and had to jack it up to 128Megs (in practice the script grab just over 64Megs when it's doing it's worst). what version of php are you [ab]using? I've read sth about php not giving the allocated memory back to the OS until a script finishes, is that right?? Is it possible to free the memory "by hand"? Any suggestion is appreciated! thx in advance -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.5.446 / Virus Database: 269.2.0/757 - Release Date: 11/04/2007 17:14 -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
Couple of questions come to mind... 1. How big is this template that you're loading? Not sure why almost any HTML file would take up 300+ meg of memory when loaded into a variable. Are you loading images and other content in as well or just the HTML file? 2. Replacing the placeholders with actual data increases memory that much? Damn.. maybe I'm just ignorant of how PHP maintains things in memory, but that seems kind of nuts too 3. Do you have to store each newsletter in memory at each loop or can you send them out then move to the next one without storing them in an array or whatever you're doing? 4. Please tell us these 'newsletters' don't contain information on "male enhancement products" or "govt seized puppies"..hah -TG = = = Original message = = = I am using PHP version 5.1.2. on my dev machine. The same problem with PHP 5.2.1 on webserver. I am generating mails based on a HTML Template. After reading the template with fread() memory_get_usage() says ~386 MB in use... Then the placeholders in the read template are replaced by customized values within a loop. The size of the used memory grows permanently in each loop by approx 30 MB (100 newsletters are generated per loop). any ideas what i am doing wrong and where i [ab]use php? ^^ Jochem Maas schrieb: > Arthur Erd~s wrote: >> Hello all, >> >> is there a way to free memory allocated by variables in PHP?? This is a >> very important issue concerning long running scripts... > > this is a recurrent problem - not much can be done about it AFAIK ... I'd > be very glad to be proved wrong. > >> I have a script that generates > 5000 Newsletters and when the script >> finishes it uses 1.8 GB (!!) of RAM. Although I am using unset() to >> clean up variables (tried with $var = null too). > > okay this does suggest your doing something else very wrong - I personally > have code that regularly generates upto 20,000 individualised newsletters > without > going anywhere near this ammount of memory - granted I've hit my memory limit > a couple of times and had to jack it up to 128Megs (in practice the script > grab just over > 64Megs when it's doing it's worst). > > what version of php are you [ab]using? > >> I've read sth about php not giving the allocated memory back to the OS >> until a script finishes, is that right?? Is it possible to free the >> memory "by hand"? >> >> Any suggestion is appreciated! >> >> thx in advance >> -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php ___ Sent by ePrompter, the premier email notification software. Free download at http://www.ePrompter.com. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
Zoltán Németh wrote: > 2007. 04. 12, csütörtök keltezéssel 15.52-kor Jochem Maas ezt írta: >> Arthur Erdös wrote: >>> I am using PHP version 5.1.2. on my dev machine. The same problem with >>> PHP 5.2.1 on webserver. >>> >>> I am generating mails based on a HTML Template. After reading the >>> template with fread() memory_get_usage() says ~386 MB in use... >> WTF - 386 megs for an html file??? somehow I doubt this is >> the correct number ... are you reading the memory consumption >> correctly? >> >> exactly how much ram does the server in question have that you can >> allow a single script to schlock up almost 2gigs ?? >> >>> Then the placeholders in the read template are replaced by customized >>> values within a loop. The size of the used memory grows permanently in >>> each loop by approx 30 MB (100 newsletters are generated per loop). >> if the values are customized I can't fathom why you would generate >> 100 emails per loop. >> >>> any ideas what i am doing wrong and where i [ab]use php? ^^ >> no not really - but it's rather ironic that you sent the same email/reply 3 >> times. >> >> with showing code there is nothing anybody can do to help. > > I think you meant "without showing code" in the above line ;) I don't think I didn't ;-) sorry for the typooh. > > greets > Zoltán Németh > > -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
2007. 04. 12, csütörtök keltezéssel 15.52-kor Jochem Maas ezt írta: > Arthur Erdös wrote: > > I am using PHP version 5.1.2. on my dev machine. The same problem with > > PHP 5.2.1 on webserver. > > > > I am generating mails based on a HTML Template. After reading the > > template with fread() memory_get_usage() says ~386 MB in use... > > WTF - 386 megs for an html file??? somehow I doubt this is > the correct number ... are you reading the memory consumption > correctly? > > exactly how much ram does the server in question have that you can > allow a single script to schlock up almost 2gigs ?? > > > > > Then the placeholders in the read template are replaced by customized > > values within a loop. The size of the used memory grows permanently in > > each loop by approx 30 MB (100 newsletters are generated per loop). > > if the values are customized I can't fathom why you would generate > 100 emails per loop. > > > > > any ideas what i am doing wrong and where i [ab]use php? ^^ > > no not really - but it's rather ironic that you sent the same email/reply 3 > times. > > with showing code there is nothing anybody can do to help. I think you meant "without showing code" in the above line ;) greets Zoltán Németh > -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
Arthur Erdös wrote: > I am using PHP version 5.1.2. on my dev machine. The same problem with > PHP 5.2.1 on webserver. > > I am generating mails based on a HTML Template. After reading the > template with fread() memory_get_usage() says ~386 MB in use... WTF - 386 megs for an html file??? somehow I doubt this is the correct number ... are you reading the memory consumption correctly? exactly how much ram does the server in question have that you can allow a single script to schlock up almost 2gigs ?? > > Then the placeholders in the read template are replaced by customized > values within a loop. The size of the used memory grows permanently in > each loop by approx 30 MB (100 newsletters are generated per loop). if the values are customized I can't fathom why you would generate 100 emails per loop. > > any ideas what i am doing wrong and where i [ab]use php? ^^ no not really - but it's rather ironic that you sent the same email/reply 3 times. with showing code there is nothing anybody can do to help. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
I am using PHP version 5.1.2. on my dev machine. The same problem with PHP 5.2.1 on webserver. I am generating mails based on a HTML Template. After reading the template with fread() memory_get_usage() says ~386 MB in use... Then the placeholders in the read template are replaced by customized values within a loop. The size of the used memory grows permanently in each loop by approx 30 MB (100 newsletters are generated per loop). any ideas what i am doing wrong and where i [ab]use php? ^^ > Arthur Erdös wrote: > > Hello all, > > > > is there a way to free memory allocated by variables in PHP?? This is a > > very important issue concerning long running scripts... > > this is a recurrent problem - not much can be done about it AFAIK ... I'd > be very glad to be proved wrong. > > > > > I have a script that generates > 5000 Newsletters and when the script > > finishes it uses 1.8 GB (!!) of RAM. Although I am using unset() to > > clean up variables (tried with $var = null too). > > okay this does suggest your doing something else very wrong - I personally > have code that regularly generates upto 20,000 individualised newsletters > without > going anywhere near this ammount of memory - granted I've hit my memory limit > a couple of times and had to jack it up to 128Megs (in practice the script > grab just over > 64Megs when it's doing it's worst). > > what version of php are you [ab]using? > > > > > I've read sth about php not giving the allocated memory back to the OS > > until a script finishes, is that right?? Is it possible to free the > > memory "by hand"? > > > > Any suggestion is appreciated! > > > > thx in advance > > > -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
I am using PHP version 5.1.2. on my dev machine. The same problem with PHP 5.2.1 on webserver. I am generating mails based on a HTML Template. After reading the template with fread() memory_get_usage() says ~386 MB in use... Then the placeholders in the read template are replaced by customized values within a loop. The size of the used memory grows permanently in each loop by approx 30 MB (100 newsletters are generated per loop). any ideas what i am doing wrong and where i [ab]use php? ^^ Jochem Maas schrieb: Arthur Erdös wrote: Hello all, is there a way to free memory allocated by variables in PHP?? This is a very important issue concerning long running scripts... this is a recurrent problem - not much can be done about it AFAIK ... I'd be very glad to be proved wrong. I have a script that generates > 5000 Newsletters and when the script finishes it uses 1.8 GB (!!) of RAM. Although I am using unset() to clean up variables (tried with $var = null too). okay this does suggest your doing something else very wrong - I personally have code that regularly generates upto 20,000 individualised newsletters without going anywhere near this ammount of memory - granted I've hit my memory limit a couple of times and had to jack it up to 128Megs (in practice the script grab just over 64Megs when it's doing it's worst). what version of php are you [ab]using? I've read sth about php not giving the allocated memory back to the OS until a script finishes, is that right?? Is it possible to free the memory "by hand"? Any suggestion is appreciated! thx in advance -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
I am using PHP version 5.1.2. on my dev machine. The same problem with PHP 5.2.1 on webserver. I am generating mails based on a HTML Template. If you look at the appended screenshot you'll see that after reading the template with fread() memory_get_usage() says ~386 MB in use... Then the placeholders in the read template are replaced by customized values within a loop. The size of the used memory grows permanently in each loop by approx 30 MB. any ideas what i am doing wrong and where i [ab]use php? ^^ Jochem Maas schrieb: Arthur Erdös wrote: Hello all, is there a way to free memory allocated by variables in PHP?? This is a very important issue concerning long running scripts... this is a recurrent problem - not much can be done about it AFAIK ... I'd be very glad to be proved wrong. I have a script that generates > 5000 Newsletters and when the script finishes it uses 1.8 GB (!!) of RAM. Although I am using unset() to clean up variables (tried with $var = null too). okay this does suggest your doing something else very wrong - I personally have code that regularly generates upto 20,000 individualised newsletters without going anywhere near this ammount of memory - granted I've hit my memory limit a couple of times and had to jack it up to 128Megs (in practice the script grab just over 64Megs when it's doing it's worst). what version of php are you [ab]using? I've read sth about php not giving the allocated memory back to the OS until a script finishes, is that right?? Is it possible to free the memory "by hand"? Any suggestion is appreciated! thx in advance -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] free allocated memory: HOW ?
Arthur Erdös wrote: > Hello all, > > is there a way to free memory allocated by variables in PHP?? This is a > very important issue concerning long running scripts... this is a recurrent problem - not much can be done about it AFAIK ... I'd be very glad to be proved wrong. > > I have a script that generates > 5000 Newsletters and when the script > finishes it uses 1.8 GB (!!) of RAM. Although I am using unset() to > clean up variables (tried with $var = null too). okay this does suggest your doing something else very wrong - I personally have code that regularly generates upto 20,000 individualised newsletters without going anywhere near this ammount of memory - granted I've hit my memory limit a couple of times and had to jack it up to 128Megs (in practice the script grab just over 64Megs when it's doing it's worst). what version of php are you [ab]using? > > I've read sth about php not giving the allocated memory back to the OS > until a script finishes, is that right?? Is it possible to free the > memory "by hand"? > > Any suggestion is appreciated! > > thx in advance > -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] free allocated memory: HOW ?
Hello all, is there a way to free memory allocated by variables in PHP?? This is a very important issue concerning long running scripts... I have a script that generates > 5000 Newsletters and when the script finishes it uses 1.8 GB (!!) of RAM. Although I am using unset() to clean up variables (tried with $var = null too). I've read sth about php not giving the allocated memory back to the OS until a script finishes, is that right?? Is it possible to free the memory "by hand"? Any suggestion is appreciated! thx in advance -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
