excessive memory usage not being released?

[Zachary Buckholz]

I have a program which submits websites to search engines etc. It currently
submits to just under 10,000 and updates a text file as it runs. When it's
done it imports the data from the text file into a Microsoft SQL database
that I connect to using FreeTDS and the DBD::Sybase.

What I noticed is that each time through the process of submitting a website
to the list of search engines the memory usage of the program increases by 1
meg. Since the program also uses Parallel Fork Manager each child process
also increases it's memory usage each loop.

I believe I have isolated the problem to the sub below, but I can't see any
reason why anything in the sub below would not release the memory.

By the time I have submitted 7 websites to the 10,000 sites the memory of
each forked child process is just under 10 megs.

Also the forking happens during the submission process and I wait for ALL
child processes before attempting to open the log file and insert it into
SQL.
Then once the SQL update is completed it loops back to the next website to
submit.

Any ideas?

Thanks
zack


##########################################
sub update_member_data {

        my $dbh = DBI->connect($data_source, $username, $password)
                || die "Can't connect: $DBI::errstr";

        my $member_id   = $_[0];

        my $log_file = "submit_data/$$member_id.log";

        open(LOG,"<$log_file") or die "Unable to open $$member_id.log\n";

        while(<LOG>) {
        chomp;

                my ($type, $date, $resp_code, $site) = split(/\t/);

                my $sql_statement = sprintf "INSERT INTO
Member_submissions_detailed_Results (Member_id, Type, Date_submitted, Resul
t_code, Page_submitted) VALUES
('$$member_id','$type','$date','$resp_code','$site')";

                my $sth = $dbh->prepare($sql_statement)
                        || die "Can't prepare the SQL statement:
$DBI::errstr";

                $sth->execute || die "Can't execute the SQL statement:
$DBI::errstr $member_id";

                $sth->finish;
        }
        close(LOG);

        my $sql_statement2 = "UPDATE members_submissions
                             SET Last_date_submitted = getdate(),
Submission_count = Isnull(Submission_count,0) + 1
                             WHERE member_id = $$member_id";

        my $sth_mem = $dbh->prepare($sql_statement)
                || die "Can't prepare the SQL statement: $DBI::errstr";

        $sth_mem->execute || die "Can't execute the SQL statement:
$DBI::errstr $member_id";

        $sth_mem->finish;

        $dbh->disconnect;
}

##########################################