On Thu, 2008-08-21 at 15:23 -0400, ANJAN PURKAYASTHA wrote:
> ok here is the problem. i have a limited amount of memory.
>
> i need to open a large file (~600 megs). i have to open a filehandle, read
> and process each line and write the output result into another file (which
> soon grows to about 300 megs.
> so,
>
> open (IN, "averylargefile");
> open (OUT, "anotherverylargefile");
>
> while (<IN>){
> chomp;
> process line here....
>
> write OUT ("$abcd\n");
> }
>
> close (IN);
> close (OUT);
>
> Now in order to ensure that all this remains within my quota, i need to
> delete each line i have read from the input file after i have written my
> processed output to the output file.
> this reduction in input file size along with a simultaneous increase in the
> output file size will ensure that i do not overshoot the quota.
>
> does anyone have pointers on how to accomplish this in perl.
>
> tia,
> anjan
>
Go out and buy yourself another disk. Seriously.
LOOP:
Read the first line and process it.
Read the second line and write were the first was.
Read the third and write at the end of the second.
Continue to the end of file.
Note that if any lines are longer than the previous, you have to read
more than one to make room to write it.
Truncate the file after the last write.
Rewind the file and repeat from LOOP.
See:
`perldoc -f open` and search for '<+'
`perldoc -f seek`
`perldoc -f tell`
`perldoc -f truncate`
--
Just my 0.00000002 million dollars worth,
Shawn
"Where there's duct tape, there's hope."
Cross Time Cafe
"Perl is the duct tape of the Internet."
Hassan Schroeder, Sun's first webmaster
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/
