Hello PDL wizards, My code churns through hundreds of thousands of HDF files, and some fraction of them are corrupted (bit-rot, as far as I can tell).
Because PDL::IO::HDF::SD does not fail gracefully, I have to catch these
files like this:
# Open HDF file for input
my $hdfopen_status = eval{$hdfobj =
PDL::IO::HDF::SD->new("$hdf_file");};
unless($hdfobj){
warn "$routine: Cannot open $hdf_file : $@\n";
print "$routine: Cannot open $hdf_file : $@\n";
#my $hdfclose_status=$hdfobj->close(); # can't do this because
$hdfobj doesn't exist!
return -2;};
This accomplishes my goals of putting information into the log about bad
files (the parent routine then deals with the files), and moving on.
However, lately I have been running a long reprocessing job, and at a
certain point it stops with the "Too many open files" error, always right
after encountering a corrupted file. My suspicion is that
PDL::IO::HDF::SD->new() is leaving an open filehandle when it exits
uncleanly.
I don't know the precise number of corrupted files that accumulates to
generate this error, but it's on the order of 1000.
Any ideas?
Thanks,
--Edward H.
smime.p7s
Description: S/MIME cryptographic signature
------------------------------------------------------------------------------ One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________ pdl-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/pdl-general
