RE : RE : Memory consumption
Witch version of ActivePerl do you have (I have the 5.6 633)? What results do you get exactly (time for the loop, memory consumption at its end, result of the undef) ? thanks > maybe it's your configuration, your environnment is > not good, we have old pc than yours and it's ok > > --- Thomas Drugeon <[EMAIL PROTECTED]> a écrit : > > I > ran this script on a PII 350, 128mo, with both > > activePerl and > > Siemens > > > the loop takes about 2minutes to completes (!), > > eating up to 10mo at > > its > > > end (witch is normal), and memory is released with > > the undef. > > > > By 10mo, I meant 10M ! > > I'm sorry, this is a french habit (octet in stead of > > byte) > > > > Thomas > > > > ___ > > Perl-Win32-Users mailing list > > [EMAIL PROTECTED] > > To unsubscribe: > http://listserv.ActiveState.com/mailman/mysubs > > ___ > Do You Yahoo!? -- Une adresse @yahoo.fr gratuite et en français ! > Yahoo! Mail : http://fr.mail.yahoo.com > ___ > Perl-Win32-Users mailing list > [EMAIL PROTECTED] > To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs ___ Perl-Win32-Users mailing list [EMAIL PROTECTED] To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re: Memory consumption
At 17:39 2002-10-15 -0700, Mike Gossland wrote: >I used Win32::SystemInfo to show free memory available. I was surprised to >see how much memory was consumed on each pass. By cutting out sections of >code, I was able to see exactly what was consuming the memory. I found >that LWP ate up a bunch, and substitutions ate up more. Sounds weird. perl -v says what? Could you post a minimal piece of code to see if we can duplicate this? I run spiders to fetch a LOT of urls from dmoz.org, and I don't have a general problem with this. Perhaps it's only noticed when there are huge files being fetched? ( I do restart the program after n thousand urls though, because the memory of a large scalar can't be returned to the system, only reused by perl. One problem I had was with HTML::Parse, where I had to break circular references myself in order to release the memory of a parsed document. ) >During my tests on Windows, I found that up to 1 MB was being consumed >each time through the loop! I found that my output results file was close >to 1MB at this time. I was updating the results file by slurping the whole >thing into memory, doing a substitution and >writing it back out again. By changing it to read one line at a time, >while writing to a different filehandle, the consumption dropped to almost >zero. Okay, maybe my first method was wasteful, but I didn't expect the >memory to be used just once! What does the s/// look like? >I was very surprised to see that when I finished with the memory (by it >going out of scope, or being undefined, or set to ""), it wasn't being >returned for re-use, but that's the way it worked. One thing I noticed with Windows perl is that it seems like memory is effectively released when minimizing the console window. At least that's what it looks like in the Task Manager. Don't know the relation to the internals of perl's garbage collection etc, or if it's just something reported by the Task manager. It happens with other programs as well it seems. /J -- --- -- -- -- -- - - - Johan LindströmSourcerer @ Boss Casinos [EMAIL PROTECTED] Latest bookmark: "SourceForge.net Project Info - Perl Direct Conn..." http://sourceforge.net/projects/pdcc/ dmoz (1 of 30): /Arts/Music/Styles/Rock/ 44 ___ Perl-Win32-Users mailing list [EMAIL PROTECTED] To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Memory consumption
Someone on another non-windows perl list wrote this: >Incidentally, I hate to call this stuff "memory leaks", since it's usually >just normal growth as memory gets used. Calling it "leaks" implies that >memory is being wasted as a result of mistakes with pointers or some such. >When people hear "memory leaks" they go running for stuff like Apache::Leak, >thinking they can find some problem and fix it, which is usually not the case. This prompts me to relate a recent experience and ask for comments. I have an application that runs on Windows with ActiveState Perl. It loops through a bunch of records and for each record it invokes LWP to fetch some data, parses the results, and saves some text in a file that grows with each iteration. With my small tests, everything went fine. When I delivered it to the customer, he tried it out and everything looked fine, until he tried it on a file with 3000 records. It died after the 450th with an Out of memory! error, and I was able to get an Out of memory! error on my box on the same file too. I looked into the source, thinking there was some reference count that wasn't going to zero and this was responsible for using it up. Not so. I used Win32::SystemInfo to show free memory available. I was surprised to see how much memory was consumed on each pass. By cutting out sections of code, I was able to see exactly what was consuming the memory. I found that LWP ate up a bunch, and substitutions ate up more. I'd never really worried about memory much before, and never had any problems on Linux where my perl software usually runs. In fact the very same program ran perfectly well on a Linux box with 1/4 the memory. During my tests on Windows, I found that up to 1 MB was being consumed each time through the loop! I found that my output results file was close to 1MB at this time. I was updating the results file by slurping the whole thing into memory, doing a substitution and writing it back out again. By changing it to read one line at a time, while writing to a different filehandle, the consumption dropped to almost zero. Okay, maybe my first method was wasteful, but I didn't expect the memory to be used just once! I was very surprised to see that when I finished with the memory (by it going out of scope, or being undefined, or set to ""), it wasn't being returned for re-use, but that's the way it worked. Is this just a Windows thing (Windows XP and ME)? Is this a normal thing that everyone always lives with? I've got routines on the Linux box that can run for weeks without running out of memory. Any comments? Mike Gossland ___ Perl-Win32-Users mailing list [EMAIL PROTECTED] To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
