On Thu, 24 May 2018 20:27:23 +0530
SATYABRATA KARAN <satyabratakara...@gmail.com> wrote:
> Hello,
> Does anyone guide me how to handle a huge file (>50GB) in perl?
Your code should be looping line by line, and shouldn't use much
memory, depending of course on how long the lines are, and what you're
doing with them.
It will fall down if the file doesn't contain lines, or if $/ doesn't
match the idea of lines in the file.
You haven't shown what you're actually doing with each line you process
- if you're keeping it in memory, then yes, you'll probably soon run
out of space.
Do you still get problems if you simplify your code to e.g.:
my $lines;
open my $FILE, "<", "Log_file.txt";
while (<$FILE>) {
$lines++;
}
close $FILE;
print "Saw $lines lines\n";
If so - look at what you're doing with each line (the "some task" bit
of your psuedocode) and see if it's copying each line somewhere or
something.
If not - most likely you need to look at the value of $/ and make sure
it makes sense for the line endings in that file.
Cheers
Dave P
--
To unsubscribe, e-mail: beginners-unsubscr...@perl.org
For additional commands, e-mail: beginners-h...@perl.org
http://learn.perl.org/
