BlankI wrote a program that converts 5 gigs of emails stored in mysql to
phisical messages on disk (resulting in approximately 10 gigs). The program
consumes way to much memory although I've wrote it in a very clean way (use
strict, no globals, use of udef $var; to help free memory). I start it
te
> I wrote a program that converts 5 gigs of emails stored in mysql to
> phisical messages on disk (resulting in approximately 10 gigs). The
program
> consumes way to much memory although I've wrote it in a very clean way
(use
> strict, no globals, use of udef $var; to help free memory). I start it
[EMAIL PROTECTED] (Perrin Harkins) wrote:
>It sounds like you have some data structure which you are adding to and
>never clearing out. One thing to be careful of is the MySQL DBI driver.
>The last time I used it, it fetched all returned rows into memory. With a
>large result set, that could tak
> > Why is the memory consumption so high ? According to my calculations it
> > should take less than 100 kilobytes.
>
> What are you basing that on?
I use a single object (lightweight) which I construct and then just call
its method in a loop, once for each message. If I don't use globals, use
> I use a single object (lightweight) which I construct and then just call
> its method in a loop, once for each message. If I don't use globals, use
> strict, undef $SomeVar I'm doing the right thing. All the "my" variables
> should dissapear once the method call is over.
It sounds like you're
[EMAIL PROTECTED] (Perrin Harkins) wrote:
>> I guess that DBI gets all records if you use fetchall_array|hashref
>> and that it is else OK.
>
>No, the behavior that I saw was when fetching a row at a time. Behind the
>scenes, DBI would fetch ALL the rows into RAM and then iterate over them.
>This
>[EMAIL PROTECTED] (Perrin Harkins) wrote:
>>> I guess that DBI gets all records if you use fetchall_array|hashref
>>> and that it is else OK.
>>
>>No, the behavior that I saw was when fetching a row at a time. Behind the
>>scenes, DBI would fetch ALL the rows into RAM and then iterate over the
Paul DuBois wrote:
> mysql_store_result/mysql_use_result determine whether or not the server
> sends all the rows to the client at once, or one at a time as the client
> asks for them. mysql_store_result is therefore more memory intensive
> for the client (which must hold the entire result set).
At 2:55 PM +1000 8/31/01, Jeremy Howard wrote:
>Paul DuBois wrote:
>> mysql_store_result/mysql_use_result determine whether or not the server
>> sends all the rows to the client at once, or one at a time as the client
>> asks for them. mysql_store_result is therefore more memory intensive
>>
