"Vasudev.K." wrote:
> Hi,
> I have this rather critical problem, I am trying to download quite huge
> files from a remote server through ftp. (The file being in a zipped
> format). I have an array which stores the names of the files to be
> downloaded. I am opening each of them up at my end and extracting data
> out of it and storing it in my oracle database.
>
> Q1. After unzipping, the file is huge (even the zipped one is :(( )..
> almost 5GB. The system throws an error...."File too large" and exits.
> How do I get around this ache? One way I want to do it is unzipped file
> into many parts and process each part separately but cant get to write a
> code for it. Is there any better solution?
This sounds like a [rpblem with your sip application, or perhaps the
limitatins of the format.
> Q2. Is there a way I can parallely do all these things? i.e
> downloading.....unzipping....data extraction... database operations.
> Here is the script.. if somebody can help me optimize it.
I'm not sure about optimixations at this point, because there are some
formatting issues that make it unnecessarily difficult to follow the code.
Herewith some tips on indentation:
> ########################################################################
> ####
>
use strict;
use warnings;
> @filenames=("A.gz","B.gz","....", "...",.............);
> #lose this, nd all other meaningless blank lines
#use vlank lines only when they signal some meaningful transition in the
flow of executtion.
> open(ZP,"database_zipped_archive.dat");
>
> while (<@filenames>)
> {
All code within this block should be indented by however many spaces you are
using as your standard. Use spaces, not tabs.
> [EMAIL PROTECTED];
> $ftp->get("$ftpfile");
>
> $unzippedftpfile="unzipped.txt";
>
> open IN,"gzip -d < $ftpfile >$unzippedftpfile |";
>
> close(IN);
>
> $subst=substr($_,0,2);
>
> open(ZNP,"tempfile.txt") or die "tempfilenot ready.....f: $!\n";;
> while (<ZNP>) # This line should not be indented
The control statement of the while loop is in the main flow of execution, as
is the opening brace.
Anyway, I am sorry that I cannot take the time to read this code for
content. Please clean up the indentation so that it is not random, but
rather reflects the logic of program execution. Then re-post the code.
Here are the basics:
Choose a number of space characters as a standard indent increm,ent. Taste
vary, but choose a standard and stick with it.
Any time you have an opening brace for a set of lines, this should be a
signal to indent the following line by one increment.
Unindent by one increment on the line that has the closing brace. The
closing brace should always have a line to itself.
Here is a sample usng a 4-space indent. [and one space for line
continuance]. Some people prefer up to eight spaces. I personally use
two. Just be consistent with whatever you choose.
sub add_interior_children {
my $self = shift;
my ($roots, $class_system, $resolved,
$unresolved, $to_close) = @_;
while (my $key = shift @{$unresolved}) {
my $parent = $class_system->get_parent($key);
next unless $parent;
if ($resolved->{$parent}) {
$self->add_interior_child($key, $parent,
$class_system, $resolved, $to_close);
} else {
push @{$unresolved}, $key;
}
}
}
When you can look at the code from arms length and tell, without reading it,
how the general flow of execution goes, repost it, and you should get plenty
of help with the logic.
Joseph
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
