Re: maximum file size for while(FILE) loop?
On Sat, 2007-01-20 at 09:31 +1100, Ken Foskey wrote: What's exactly the difference between: ++$lines; and $lines++; ? Nothing in this context. What about other contexts? David. -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] http://learn.perl.org/
Re: maximum file size for while(FILE) loop?
David Moreno Garza am Sonntag, 21. Januar 2007 07:50:
On Sat, 2007-01-20 at 09:31 +1100, Ken Foskey wrote:
What's exactly the difference between:
++$lines and $lines++; ?
Nothing in this context.
What about other contexts?
Hi David
#!/usr/bin/perl
use strict;
use warnings;
{ # preincrement
my (%h, $i);
$h{++$i}='hi';
print keys %h, , $i\n;
}
{ # postincrement
my (%h, $i);
$h{$i++}='hi';
print keys %h, , $i\n;
}
__END__
1, 1
0, 1
The difference is the order of read current value (used as hash key value)
and increment current value (done by ++ operator).
There's no difference between standalone ++$lines and $lines++ because only
increment takes place, and the result value is not used in the same
expression.
See also perldoc perlop, Auto-increment and Auto-decrement.
Hope this helps!
Dani
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/
Re: maximum file size for while(FILE) loop? - maybe HASH problem?
On 1/19/07, Bertrand Baesjou [EMAIL PROTECTED] wrote:
While running my script it seems to use around a gigabyte of memory
(there is 1GB of RAM and 1GB of swap in the system), might this be the
problem?
If you're running low on memory, unless you're working on an
inherintly large problem, your algorithm is probably wasting some
memory.
foreach $line (INFILE) {
$somestorage{$linecounter}=$value;
$linecounter++;
}
Well, that builds a big hash for nothing. Unless you're trying to waste memory?
print $linecounter;
You should probably put a newline at the end of your output.
system(pwd) == 0 or die system failed: $?;
5198365system failed: 0 at ./sample1.pl line 22.
You're trying to run the command pwd, which seems to have failed.
The value of $? is zero though, which would normally indicate success.
Perhaps it's zero because the command couldn't be executed at all?
(Maybe low memory?) Does the pwd command normally work from
system()? (You're not comparing this pwd to the shell built-in, are
you?)
Hope this helps!
--Tom Phoenix
Stonehenge Perl Training
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/
maximum file size for while(FILE) loop?
Hi,
I am trying to read data from a file, I do this by using the while
(FILE){ $line} construction.
However with files with a size of roughly bigger than 430MB it seems to
crash the script :S Syntax seems all fine (perl -wc - syntax OK).
I was thinking that maybe it was running to the end of a 32 bit counter
(but that would be 536 MB right?)? Can anybody offer an other solution
to work with such large files and perl?
Tnx,
Bertrand
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/
Re: maximum file size for while(FILE) loop?
On Fri, 2007-01-19 at 13:16 +0100, Bertrand Baesjou wrote:
Hi,
I am trying to read data from a file, I do this by using the while
(FILE){ $line} construction.
However with files with a size of roughly bigger than 430MB it seems to
crash the script :S Syntax seems all fine (perl -wc - syntax OK).
I was thinking that maybe it was running to the end of a 32 bit counter
(but that would be 536 MB right?)? Can anybody offer an other solution
to work with such large files and perl?
No idea, a little script sample might be good.
--
Ken Foskey
FOSS developer
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/
Re: maximum file size for while(FILE) loop?
Bertrand Baesjou wrote:
Hi,
I am trying to read data from a file, I do this by using the while
(FILE){ $line} construction.
However with files with a size of roughly bigger than 430MB it seems to
crash the script :S Syntax seems all fine (perl -wc - syntax OK).
How does your script crash? What are the symptoms?
I was thinking that maybe it was running to the end of a 32 bit counter
(but that would be 536 MB right?)? Can anybody offer an other solution
to work with such large files and perl?
People have read files of several gigabytes with Perl. The problem is more
likely to lie with what you do with the data once you have read it. To prove
this for yourself, set this code against the same file:
my $lines = 0;
while (FILE) {
++$lines;
}
print $lines;
and I am pretty sure that won't crash.
Then try to simplify your code by removing stuff from the loop until the problem
goes away. The last thing you removed contains the cause of the crash.
If you need to post again please give us comprehensive details of the crash.
HTH,
Rob
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/
Re: maximum file size for while(FILE) loop? - maybe HASH problem?
Ken Foskey wrote:
On Fri, 2007-01-19 at 13:16 +0100, Bertrand Baesjou wrote:
Hi,
I am trying to read data from a file, I do this by using the while
(FILE){ $line} construction.
However with files with a size of roughly bigger than 430MB it seems to
crash the script :S Syntax seems all fine (perl -wc - syntax OK).
I was thinking that maybe it was running to the end of a 32 bit counter
(but that would be 536 MB right?)? Can anybody offer an other solution
to work with such large files and perl?
No idea, a little script sample might be good.
While running my script it seems to use around a gigabyte of memory
(there is 1GB of RAM and 1GB of swap in the system), might this be the
problem?
The script below gives the error:
#!/usr/local/bin/perl
##
#
use POSIX;
my $inputFile = $ARGV[0]; #file we are
reading from
my $outputFile = ./tmp/overall-memory.dat;
my %somestorage;
my $linecounter = 0;
my $value=40;
my $bool = 0;
open(INFILE, $inputFile) or die(Could not open log
file.);# open for input
open(OUTFILE, $outputFile);
foreach $line (INFILE) {
$somestorage{$linecounter}=$value;
$linecounter++;
}
close(INFILE);
close(OUTFILE);
print $linecounter;
system(pwd) == 0 or die system failed: $?;
#wc -l samples/1169209055.trcxml
5198365 samples/1169209055.trcxml
# ./sample1.pl samples/1169209055.trcxml
5198365system failed: 0 at ./sample1.pl line 22.
Any ideas how to solve my problem?
Tnx!
Bertrand
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/
Re: maximum file size for while(FILE) loop? - maybe HASH problem?
On Fri, Jan 19, 2007 at 03:17:19PM +0100, Bertrand Baesjou wrote:
foreach $line (INFILE) {
See, this isn't a while loop, as you have in the subject.
That is the cause of your problems.
--
Paul Johnson - [EMAIL PROTECTED]
http://www.pjcj.net
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/
Re: maximum file size for while(FILE) loop?
On Fri, 2007-01-19 at 13:24 +, Rob Dixon wrote: ++$lines; What's exactly the difference between: ++$lines; and $lines++; ? David. -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] http://learn.perl.org/
Re: maximum file size for while(FILE) loop?
On Fri, 2007-01-19 at 16:21 -0600, David Moreno Garza wrote: On Fri, 2007-01-19 at 13:24 +, Rob Dixon wrote: ++$lines; What's exactly the difference between: ++$lines; and $lines++; ? Nothing in this context. It does make a difference if you are 'using' the value see sample and try it: $lines = 10; $value = ++$lines; print first value is it 10 or 11? $value\n $value = $lines++; print second value is it 11 or 12? $value\n -- Ken Foskey FOSS developer -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] http://learn.perl.org/
Re: maximum file size for while(FILE) loop?
David Moreno Garza wrote: On Fri, 2007-01-19 at 13:24 +, Rob Dixon wrote: ++$lines; What's exactly the difference between: ++$lines; and $lines++; ? In void context they are both the same because perl optimizes $lines++ to ++$lines. John -- Perl isn't a toolbox, but a small machine shop where you can special-order certain sorts of tools at low cost and in short order. -- Larry Wall -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] http://learn.perl.org/
