Arantxa Otegi wrote:
> I have memory problems programming with perl: "out of memory!"
>
> I have to process a lot of xml files which are in different directories
> (more than 20000 files in 110 directories). The files are quite small
> (almost all of them are smaller than 100KB).
> Here is some code:
>
> #### code
> #####################################################################################
>
> use strict;
> use Expansion;
>
> die "Usage: doc_expansion.pl sarrera_dir irteera_dir hizkuntza_kodea
> %synset_expansioa" if ! $#ARGV == 3;
> my $sarrera_dir = $ARGV[0];
> my $irteera_dir = $ARGV[1];
> my $hizk = $ARGV[2];
> my $portzentaia = $ARGV[3];
That is usually written as:
die "Usage: $0 sarrera_dir irteera_dir hizkuntza_kodea %synset_expansioa"
if @ARGV != 4;
my ( $sarrera_dir, $irteera_dir, $hizk, $portzentaia ) = @ARGV;
> my %variantak = Expansion::irakurri_variantak($hizk);
>
> my @dirs;
> my $dir;
> my @fitxategiak;
> my $fitx;
> my $irt_fitx;
> my @fitx_zenb;
> my $fitx_zenb;
>
> $| = 1;
>
> opendir(DIR,$sarrera_dir)|| die "ezin da ireki DIR: $sarrera_dir";
You should include the $! variable in the error message so you know *why* it
failed:
opendir DIR, $sarrera_dir or die "ezin da ireki DIR: $sarrera_dir: $!";
> @dirs = readdir(DIR);
my @dirs = readdir(DIR);
> closedir(DIR);
>
> foreach $dir (@dirs){
foreach my $dir (@dirs){
> next if $dir =~ /\.$/;
>
> opendir(DIR,"$sarrera_dir/$dir/cache_en") || die "DIR(2):
> $sarrera_dir/$dir/cache_en" ;
You should include the $! variable in the error message so you know *why* it
failed:
opendir DIR,"$sarrera_dir/$dir/cache_en" or die "DIR(2):
$sarrera_dir/$dir/cache_en: $!";
> if(not -e "$irteera_dir/$dir"){
This test will never succeed because you have either opened
"$sarrera_dir/$dir/cache_en" and so it exists or the program has died.
> mkdir("$irteera_dir/$dir",0777)|| die "DIR(2): $irteera_dir/$dir";
You should include the $! variable in the error message so you know *why* it
failed:
mkdir "$irteera_dir/$dir", 0777 or die "DIR(2): $irteera_dir/$dir: $!";
> }
> if(not -e "$irteera_dir/$dir/cache_en"){
This test will never succeed because you have either opened
"$sarrera_dir/$dir/cache_en" and so it exists or the program has died.
> mkdir("$irteera_dir/$dir/cache_en",0777)|| die "DIR(2):
> $irteera_dir/$dir/cache_en";
You should include the $! variable in the error message so you know *why* it
failed:
mkdir "$irteera_dir/$dir/cache_en", 0777 or die "DIR(2):
$irteera_dir/$dir/cache_en: $!";
> }
>
> @fitxategiak = grep {/\.wsd$/} readdir(DIR);
> closedir(DIR);
> foreach $fitx (@fitxategiak){
> print "$dir/cache_en/$fitx ....... ";
>
> @fitx_zenb = split(/\./,$fitx);
> $fitx_zenb = $fitx_zenb[0];
You could simplify that as:
my $fitx_zenb = ( split /\./, $fitx )[ 0 ];
> $irt_fitx = "$irteera_dir/$dir/cache_en/$fitx_zenb.exp.$hizk";
>
>
> Expansion::fitx_expansioa("$sarrera_dir/$dir/cache_en/$fitx",$irt_fitx,$portzentaia,%variantak);
>
> print "$irt_fitx--$fitx_zenb.exp.$hizk\n";
> }
> }
> -------------------------------------------------------------------------------------------
>
>
> package Expansion;
> use strict;
> use XML::DOM;
>
> $| = 1;
>
> sub fitx_expansioa{
> my $sarfitx;
> my $irteera_fitx;
> my %variantak;
> my $portzentaia;
> ($sarfitx,$irteera_fitx,$portzentaia,%variantak) = @_;
Or more simply as:
sub fitx_expansioa {
my ( $sarfitx, $irteera_fitx, $portzentaia, %variantak ) = @_;
> my $dom_parser = new XML::DOM::Parser;
> my $dok = $dom_parser->parsefile($sarfitx);
>
> //XML process
> ...........
>
> my $irteeraXML = $dok->toString;
> $dok->dispose;
> open(I,">$irteera_fitx") or die "Ezin da ireki $irteera_fitx: $!";
> print I $irteeraXML;
> close I;
> }
perldoc perlmodlib
[ snip ]
Try to "use warnings;" (or "use warnings qw(...);"). Remember that you
can add "no warnings qw(...);" to individual blocks of code that need
less warnings.
[ snip ]
Always use -w.
Try to "use strict;" (or "use strict qw(...);"). Remember that you can
add "no strict qw(...);" to individual blocks of code that need less
strictness.
Always use -w.
Follow the guidelines in the perlstyle(1) manual.
Always use -w.
perldoc perlmod
[ snip ]
## YOUR CODE GOES HERE
1; # don't forget to return a true value from the file
> ##### end code
> ##############################################################################
>
>
> Can anybody help me to fix the problem?
I can't see anything obvious that would cause your program to run out of
memory. Which version of perl are you running? Which version of XML::DOM?
John
--
Perl isn't a toolbox, but a small machine shop where you can special-order
certain sorts of tools at low cost and in short order. -- Larry Wall
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/
