memory leak

[Curt Russell Crandall]

I'm having a problem with my program having its memory usage go out of
control.  I'm working on Solairs 2.5.1 and 2.6 with several Perl builds:

Perl 5.00404 w/ DBI 1.13 & DBD::Sybase 0.21
Perl 5.00404 w/ DBI 1.14 & DBD::Sybase 0.91
Perl 5.6.0   w/ DBI 1.14 & DBD::Sybase 0.91


My program is somewhat OO in design.  I loop through a set of files and
instantiate an object for each.  In that object, I instantiate 2 more
objects that hold the input and output data structures.  These structures
are created using Class::Struct and I'm using the array form.
For each file, I loop through each line parsing the contents and moving
the contents into the input structure.  I then map that to the output
structure.

For some of the files, I may need to insert a record into a database table
if a parameter comes up missing.  For these files, the memory usage grows
larger with each line.


Since Sybase can't have a database handle with multiple statement handles
with AutoCommit off, I'm creating 2 database handles.  The one with
AutoCommit off is used for selects and for the insert mentioned
above.  The other handle is used to insert the output structure into the
database... I'm caching every 100 transactions between commits.

my %attrs = (RaiseError=>0, PrintError=>1, AutoCommit=>1, ChopBlanks=>1);
my $dbconn = "dbi:Sybase:server=$env{DB_SERVER}";

my $dbh1 = DBI->connect($dbconn, $env{DB_USER}, $env{DB_PASSWD}, \%attrs);

$attrs{ChopBlanks} = 0;
my $dbh2 = DBI->connect($dbconn, $env{DB_USER}, $env{DB_PASSWD}, \%attrs);


These 2 database handles are stuffed into attributes of the main object:
$self->{dbh} and $self->{dbh_ins}, respectively.


Now I prepare the statement and place that in an object attribute:

$self->{fa_ins} = $self->{dbh}->prepare( <<EOF
INSERT INTO blah
(foo, bar)
values (?, ?)
EOF;


In the piece of the program that does the processing:

sub process_file
 {
     my $self = shift;  ### main object

     open(IN, "<$self->{file}");

     while (defined(my $record = <IN>))
      {
            my @data = split("\t", $record);
            $self->{inObj}->init(@data);      ### Populate sub-object

            $self->{inObj}->val();            ### validate sub-object
            $self->{inObj}->map();            ### map input to output

            $self->write();                   ### write to database
      }
 }


The problem happens in the method map:

if ((!defined($master->{outObj}->cusip)) &&
(!defined($href->{alias_name})))
 {
     $master->{fa_ins}->execute($master->{source_id}, $self->fund_name);
     return($DBI::errstr) if ($DBI::err);
 }

where $master refers to the main object in this method.

This chunk of code should conditionally insert a line into a db table and
immediately commit since the AutoCommit is enabled.  The correct actions
are taking place, however, each time a line is mapped and this command
executed, the amount of memory and, to a lesser degree, cpu resources are
consumed in increasing amounts.  By the time I hit about 17000 lines, my
memory usage is around 2GB.

Can anyone guess as to what I am doing wrong, or is this possibly a quirk
with the Sybase driver?

Thanks,
Curt Crandall