Re: Benchmark times
Thanks to all! Im going to check the requirements (memory, CPU, etc). Thanks! El día Sunday 24 August 2003 1:45 a Pablo Fischer mandó el siguiente correo: > Hi! > > This maybe its not a Perl question. > > I have a system (with 3 classes), that works in this order: > > > * Check in FTP for new files > * Download new Files > * Unzip those files (.zip files) > * Each file has 3000 lines, and there are like 10 files (30,000 lines), So > I need to parse the 30,000 lines and order them > * Parse File by File, and save the parssed file in a DB (MySql) > * Check to see if I need a backup of the Zip File > * Finish > > How good its to do this in 10 seconds? > Exists a website with benckmark running times in Perl? > > Thanks! > -- > Pablo Fischer Sandoval ([EMAIL PROTECTED]) > http://www.pablo.com.mx > http://www.debianmexico.org > GPG FingerTip: 3D49 4CB8 8951 F2CA 8131 AF7C D1B9 1FB9 6B11 810C > Firma URL: http://www.pablo.com.mx/firmagpg.txt -- Pablo Fischer Sandoval ([EMAIL PROTECTED]) http://www.pablo.com.mx http://www.debianmexico.org GPG FingerTip: 3D49 4CB8 8951 F2CA 8131 AF7C D1B9 1FB9 6B11 810C Firma URL: http://www.pablo.com.mx/firmagpg.txt -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Benchmark times
Pablo Fischer wrote: > > This maybe its not a Perl question. > > I have a system (with 3 classes), that works in this order: > > > * Check in FTP for new files > * Download new Files > * Unzip those files (.zip files) > * Each file has 3000 lines, and there are like 10 files (30,000 lines), So I > need to parse the 30,000 lines and order them > * Parse File by File, and save the parssed file in a DB (MySql) > * Check to see if I need a backup of the Zip File > * Finish > > How good its to do this in 10 seconds? > Exists a website with benckmark running times in Perl? Hi Pablo. I'm not clear whether you already have this software written. But as Oliver says, never think about trying to speed anything up unless it's too slow. If you're trying to commit to a certain performance level in a specification that you're writing, then go and ask the people who care what the slowest acceptable performance is. Whatever they say, look troubled, shake your head, mumble something technical, and try to negotiate the required speed as low as you can. Then when you go away and write it and it and it runs ten times faster you'll be applauded as a hero. By far the largest overhead here is going to be the FTP transfer, which will depend on the type, quality and contention of your connection. The next heaviest load will be the database access. Since both of these are probably out of your control your best bet is to just write the program, make it work, and publish it. It's very unlikely that you'll be able to make much difference to the running speed. Good luck, Rob -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Benchmark times
On Sun, Aug 24, 2003 at 09:35:39AM +0200 Janek Schleicher wrote:
> Pablo Fischer wrote at Sun, 24 Aug 2003 01:45:42 +:
> > Exists a website with benckmark running times in Perl?
>
> http://www.google.com/search?q=benchmark+perl+programming+languages
Benchmark between languages are - at the best of times - misleading.
They tend to contrive some seemingly common problems and compare their
implementations in different languages. Not even the p5-porters have so
far managed to come up with a reliable benchmark for comparing perl with
perl (perlbench gives nice tables but real-world programs often seem to
contradict these tables).
But as it can be seen in this thread, the problem wont be solved using
pure-Perl. Database access usually happens via modules written in C.
Therefore, a program making heavy use of an SQL database written in pure C
and one written in Perl will yield very comparable results.
I remember the longest-repeated-substring problem that was discussed
some months ago on MJD's quiz-of-the-week mailinglist. A reasonably fast
Perl solution looked very un-Perlish (doing it all with raw index() and
substr() operations for the sake of speed) and thus was easy to
translate to XS. The C implementation was faster to an degree of 5%, at
most. But sometimes it was even slower.
On the other hand, Octave for instance was written in C and Fortran. If
you take this and compare it with an implementation in Perl (if it
existed), it will surely make Perl look very poor.
So only for very sanitized ('pure') problems you'll be able to predice
which implementation will likely be fastest. The problem of the OP is no
such case. First you have the overhead of the FTP transaction,
afterwards unzipping that can either be done the slow way (Archive::Zip)
or in an intelligent way (when speed matters) through PerlIO::gzip or
even an external unzipper.
The only thing that really might make look one language more appropriate
than the other is the sorting. But again, precise figures can't be given
since Perl's sort is either very performant (when using one of the
optimized default routines) or painfully slow (with a complicated custom
comparison-routine).
Then a little bit I/O is carried out. Again, only little significance to
the language (they all are somewhat C-based). Yet, Perl is known to do
I/O a little better than most competitors. Eventually, database-access.
That's no concern when using one of the C-implementations in the DBD::
namespace.
Conclusion: most common languages should be able to do that in
reasonable time. They are no different than Perl in that they interface
with C code for tasks like working with compressed files, I/O and
databases. Unless of course you do in PROLOG or some other truely exotic
language.
Tassilo
--
$_=q#",}])!JAPH!qq(tsuJ[{@"tnirp}3..0}_$;//::niam/s~=)]3[))_$-3(rellac(=_$({
pam{rekcahbus})(rekcah{lrePbus})(lreP{rehtonabus})!JAPH!qq(rehtona{tsuJbus#;
$_=reverse,s+(?<=sub).+q#q!'"qq.\t$&."'!#+sexisexiixesixeseg;y~\n~~;eval
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Benchmark times
Pablo Fischer wrote at Sun, 24 Aug 2003 01:45:42 +: > This maybe its not a Perl question. I guess it ain't really not one. > I have a system (with 3 classes), that works in this order: > > > * Check in FTP for new files Time depends on the time to connect to FTP. However, the time is always independent of the programming language, thus neglioabable to your problem. > * Download new Files Again independent of the programming language. > * Unzip those files (.zip files) Mostly independent of the programming language, however you can always use the zip and unzip programs if a solution in a specific language, e.g. a pure Perl solution might be slower. So the time depends only on the size of files. > * Each file has 3000 lines, and there are like 10 files (30,000 lines), So I > need to parse the 30,000 lines and order them Do you need to parse them or do you need to sort them? However, parsing is something Perl is very good in. Thus you can simply obtain, a Perlish solution is the quickest way, so you also don't need to know how quick Perl is. Sorting is something different as there might be very quick c solution specialised to sorting on hard disks. But that depends also on the size of the files. I assume simply 30_000 lines of 100 columns each => 3_000_000 bytes = 3 MB. 3 MB would be nowadays so less that it Perl's memory wasting wouldn't create problems. Unless memory problems, common sort routines are very quick in all languages, also to Perl. > * Parse File by File, and save the parssed file in a DB (MySql) Depends on the time of the database. > * Check to see if I need a backup of the Zip File Depends on the check routine, but I assume it is also language independent. > * Finish > > How good its to do this in 10 seconds? It's always possible, you'll only need a computer that is expensive enough. However, if my guess of 3 MB is right, it should be easy with nowadays desktop pcs in every language. > Exists a website with benckmark running times in Perl? http://www.google.com/search?q=benchmark+perl+programming+languages Greetings, Janek -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Benchmark times
On Sun, 24 Aug 2003 01:45:42 +, Pablo Fischer wrote: > I have a system (with 3 classes), that works in this order: > [...deleted...] > How good its to do this in 10 seconds? > Exists a website with benckmark running times in Perl? How good is it??? Depends on your goals and your hardware. It seems that you attack this from the wrong angle. Everything that is fast enough to solve your problem in a given time is fast enough. I.e. If you don't have any requirements towards how fast you have to sort the data means that your program is fast enough. The question you are asking is about perl optimization... which is mostly answered with the following answer: DON'T... STOP... DON'T EVEN THINK ABOUT IT. Various reasons behind all of this is that people don't know which part they have to optimize. Even though their are modules that will help you to find out the lines of code that take the longest. Take a look at Devel::DProf which is a Perl code profiler. Though always remember: Optimization takes time, increases complexity, adds bugs, wrecks encapsulation, reduces readability and last but not least makes your code unmaintainable. thanks /oliver/ -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Benchmark times
Hi! This maybe its not a Perl question. I have a system (with 3 classes), that works in this order: * Check in FTP for new files * Download new Files * Unzip those files (.zip files) * Each file has 3000 lines, and there are like 10 files (30,000 lines), So I need to parse the 30,000 lines and order them * Parse File by File, and save the parssed file in a DB (MySql) * Check to see if I need a backup of the Zip File * Finish How good its to do this in 10 seconds? Exists a website with benckmark running times in Perl? Thanks! -- Pablo Fischer Sandoval ([EMAIL PROTECTED]) http://www.pablo.com.mx http://www.debianmexico.org GPG FingerTip: 3D49 4CB8 8951 F2CA 8131 AF7C D1B9 1FB9 6B11 810C Firma URL: http://www.pablo.com.mx/firmagpg.txt -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
