ANJAN PURKAYASTHA wrote:
>
> ok here is the problem. i have a limited amount of memory.
>
> i need to open a large file (~600 megs). i have to open a filehandle, read
> and process each line and write the output result into another file (which
> soon grows to about 300 megs.
> so,
Always
use strict;
use warnings;
at the start of your program.
> open (IN, "averylargefile");
> open (OUT, "anotherverylargefile");
You have opened both files for input, you should always check the status of the
open to make sure it succeeded, and it is best practice to use lexical file
handles and the three-parameter form of open.
open my $in, '<', 'averylargefile' or die $!;
open my $out, '>', 'anotherverylargefile' or die $!;
> while (<IN>){
while (<$in>) {
> chomp;
> process line here....
>
> write OUT ("$abcd\n");
There is no write function.
print $out "$abcd\n";
> }
>
> close (IN);
> close (OUT);
There is no real need to explicitly close an input file handle unless you are
opening many files in your program, and you should always check the status when
closing an output file to make sure it succeeded.
close $out or die $!;
> Now in order to ensure that all this remains within my quota, i need to
> delete each line i have read from the input file after i have written my
> processed output to the output file.
> this reduction in input file size along with a simultaneous increase in the
> output file size will ensure that i do not overshoot the quota.
>
> does anyone have pointers on how to accomplish this in perl.
Having annotated your program, I shall suggest an entirely different approach!
Is your disk quota really as low as 1GB? I would have thought it would be best
to simply delete the input file once processing completes. An alternative is to
edit the file in place using Tie::File as in the program below.
HTH,
Rob
use strict;
use warnings;
use Tie::File;
use Fcntl;
tie my @file, 'Tie::File', 'averylargefile', mode => O_RDWR
or die $!;
foreach my $record (@file) {
# modify $record as required
}
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/
