Re: sobering observation, python vs. perl

On Thu, 17 Mar 2016 21:18:43 +0530, srinivas devaki wrote: > please upload the log file, Sorry, it's work stuff, can't do that, but just take any big set of files and change the strings appropriately and the numbers should be equivalent. > > and global variables in python are slow, so just

Re: sobering observation, python vs. perl

On 03/17/2016 09:08 AM, Charles T. Smith wrote: On Thu, 17 Mar 2016 10:52:30 -0500, Tim Chase wrote: Not saying this will make a great deal of difference, but these two items jumped out at me. I'd even be tempted to just use string manipulations for the isready aspect as well. Something

Re: sobering observation, python vs. perl

On Thu, 17 Mar 2016 15:29:47 +, Charles T. Smith wrote: And for completeness, and also surprising: time sed -n -e '/ is ready/{s///;h}' -e '/release_req/{g;p}' *.out | sort -u TestCase_F_00_P TestCase_F_00_S TestCase_F_01_S TestCase_F_02_M real0m10.998s user0m10.885s sys

Re: sobering observation, python vs. perl

On Thu, 17 Mar 2016 19:08:58 +0200, Marko Rauhamaa wrote: > "Charles T. Smith" : > > > Compare Perl (http://www.perlmonks.org/?node_id=98357>): > >my $str = "I have a dream"; >my $find = "have"; >my $replace = "had"; >$find = quotemeta $find; #

Re: sobering observation, python vs. perl

"Charles T. Smith" : > I need the second check to also be a RE because it's not > separate tokens. The string "in" check doesn't care about tokens. Marko -- https://mail.python.org/mailman/listinfo/python-list

Re: sobering observation, python vs. perl

On 17/03/2016 18:53, Marko Rauhamaa wrote: BartC : sub replacewith{ $s = $_[0]; $t = $_[1]; $u = $_[2]; $s =~ s/$t/$u/; return $s; } Although once done, the original task now looks a proper language: print (replacewith("I have a dream","have","had"));

Re: sobering observation, python vs. perl

On Thu, 17 Mar 2016 10:52:30 -0500, Tim Chase wrote: >> Not saying this will make a great deal of difference, but these two > items jumped out at me. I'd even be tempted to just use string > manipulations for the isready aspect as well. Something like > (untested) well, I don't want to forgo

Re: sobering observation, python vs. perl

"Charles T. Smith" : > well, I don't want to forgo REs in order to have python's numbers be > better http://stackoverflow.com/questions/12793562/text-processing-pytho n-vs-perl-performance> Marko -- https://mail.python.org/mailman/listinfo/python-list

Re: sobering observation, python vs. perl

"Charles T. Smith" : > Here's the programs: > > #!/usr/bin/env python > # vim: tw=0 > import sys > import re > > isready = re.compile ("(.*) is ready") > relreq = re.compile (".*release_req") > for fn in sys.argv[1:]: # logfile name >

Re: sobering observation, python vs. perl

On Thu, 17 Mar 2016 18:30:29 +0200, Marko Rauhamaa wrote: > "Charles T. Smith" : > >> I need the second check to also be a RE because it's not >> separate tokens. > > The string "in" check doesn't care about tokens. > > > Marko Ah, yes. Okay. --

Re: sobering observation, python vs. perl

BartC : > On 17/03/2016 18:53, Marko Rauhamaa wrote: >> BartC : > >>> sub replacewith{ >>> $s = $_[0]; >>> $t = $_[1]; >>> $u = $_[2]; >>> $s =~ s/$t/$u/; >>> return $s; >>> } >>> >>> Although once done, the original task now looks a proper

Re: sobering observation, python vs. perl

On Fri, 18 Mar 2016 03:08 am, Charles T. Smith wrote: > On Thu, 17 Mar 2016 10:52:30 -0500, Tim Chase wrote: > >>> Not saying this will make a great deal of difference, but these two >> items jumped out at me. I'd even be tempted to just use string >> manipulations for the isready aspect as

Re: sobering observation, python vs. perl

BartC : > I was going to suggest just using a function. But never having coded in > Perl before, I wasn't expecting something this ugly: > > sub replacewith{ >$s = $_[0]; >$t = $_[1]; >$u = $_[2]; >$s =~ s/$t/$u/; >return $s; > } > > Although once done, the

Re: sobering observation, python vs. perl

On Thu, 17 Mar 2016 10:26:12 -0700, Ethan Furman wrote: > On 03/17/2016 09:36 AM, Charles T. Smith wrote: > >> Yes, your point was to forgo REs despite that they are useful. >> I could have thought the search would have been better as: >> >> 'release[-.:][Rr]eq' >> >> or something else ...

Re: sobering observation, python vs. perl

On 17/03/2016 17:25, Charles T. Smith wrote: On Thu, 17 Mar 2016 19:08:58 +0200, Marko Rauhamaa wrote: my $str = "I have a dream"; my $find = "have"; my $replace = "had"; $find = quotemeta $find; # escape regex metachars if present $str =~ s/$find/$replace/g; print

Re: sobering observation, python vs. perl

On Thu, 17 Mar 2016 18:07:12 +0200, Marko Rauhamaa wrote: > "Charles T. Smith" : > Ok. The LANG=C setting has a tremendous effect on the performance of > textutils. > > > Marko Good to know, thank you... -- https://mail.python.org/mailman/listinfo/python-list

Re: sobering observation, python vs. perl

On Thu, 17 Mar 2016 17:48:54 +0200, Marko Rauhamaa wrote: > "Charles T. Smith" : > >> On Thu, 17 Mar 2016 15:29:47 +, Charles T. Smith wrote: >> >> And for completeness, and also surprising: >> >> time sed -n -e '/ is ready/{s///;h}' -e '/release_req/{g;p}'

Re: sobering observation, python vs. perl

On 2016-03-17 15:29, Charles T. Smith wrote: > isready = re.compile ("(.*) is ready") > relreq = re.compile (".*release_req") > for fn in sys.argv[1:]: # logfile > name tn = None > with open (fn) as fd: > for line in fd: > #match = re.match

Re: sobering observation, python vs. perl

"Charles T. Smith" : > On Thu, 17 Mar 2016 17:48:54 +0200, Marko Rauhamaa wrote: >> Try running the sed command again after setting: >> >> export LANG=C > > Hmmm. Interesting thought. But... > > $ locale > LANG=C Ok. The LANG=C setting has a tremendous effect on

Re: sobering observation, python vs. perl

On Thu, 17 Mar 2016 09:21:51 -0700, Ethan Furman wrote: >> well, I don't want to forgo REs in order to have python's numbers be >> better > > The issue is not avoiding REs, but using Python's strengths and idioms. > Write the code in Python's style, get the same results, then compare >

Re: sobering observation, python vs. perl

Charles T. Smith wrote: > On Thu, 17 Mar 2016 10:52:30 -0500, Tim Chase wrote: > >>> Not saying this will make a great deal of difference, but these two >> items jumped out at me. I'd even be tempted to just use string >> manipulations for the isready aspect as well. Something like >>

Re: sobering observation, python vs. perl

On Thu, 17 Mar 2016 17:47:55 +0200, Marko Rauhamaa wrote: > Can't comment on the numbers but the code segments are not quite > analogous. What about this one: > > #!/usr/bin/env python > # vim: tw=0 > import sys > import re > > isready = re.compile("(.*) is ready") >

Re: sobering observation, python vs. perl

On Thu, 17 Mar 2016 18:34:06 +0200, Marko Rauhamaa wrote: > n-vs-perl-performance Okay, that was interesting. Actually, I saw a study some years ago that concluded that python could be both slower and faster than perl, but that perl had much less deviation than python. I took that and

Re: sobering observation, python vs. perl

On 03/17/2016 09:36 AM, Charles T. Smith wrote: Yes, your point was to forgo REs despite that they are useful. I could have thought the search would have been better as: 'release[-.:][Rr]eq' or something else ... you're in a "defend python at all costs!" mode. No, I'm in the "don't try

Re: sobering observation, python vs. perl

Charles T. Smith: I've really learned to love working with python, but it's too soon to pack perl away. I was amazed at how long a simple file search took so I ran some statistics: Write Python in pythonic style instead of translated-from-Perl style, and the tables are turned: $ cat

Re: sobering observation, python vs. perl

On Thursday, March 17, 2016 at 11:24:00 PM UTC+5:30, BartC wrote: > On 17/03/2016 17:25, Charles T. Smith wrote: > > On Thu, 17 Mar 2016 19:08:58 +0200, Marko Rauhamaa wrote: > > >> my $str = "I have a dream"; > >> my $find = "have"; > >> my $replace = "had"; > >> $find =

Re: sobering observation, python vs. perl

"Charles T. Smith" : > On Thu, 17 Mar 2016 15:29:47 +, Charles T. Smith wrote: > > And for completeness, and also surprising: > > time sed -n -e '/ is ready/{s///;h}' -e '/release_req/{g;p}' *.out | sort -u > TestCase_F_00_P > TestCase_F_00_S > TestCase_F_01_S >

Re: sobering observation, python vs. perl

Marko Rauhamaa writes: > "Charles T. Smith" : > >> Actually, I saw a study some years ago that concluded that python >> could be both slower and faster than perl, but that perl had much less >> deviation than python. I took that and accepted it, but

Re: sobering observation, python vs. perl

On 03/17/2016 10:35 AM, Charles T. Smith wrote: On Thu, 17 Mar 2016 10:26:12 -0700, Ethan Furman wrote: On 03/17/2016 09:36 AM, Charles T. Smith wrote: Yes, your point was to forgo REs despite that they are useful. I could have thought the search would have been better as:

sobering observation, python vs. perl

I've really learned to love working with python, but it's too soon to pack perl away. I was amazed at how long a simple file search took so I ran some statistics: $ time python find-rel.py ./find-relreq *.out | sort -u TestCase_F_00_P TestCase_F_00_S TestCase_F_01_S

Re: sobering observation, python vs. perl

On 17/03/2016 16:36, Charles T. Smith wrote: On Thu, 17 Mar 2016 09:21:51 -0700, Ethan Furman wrote: well, I don't want to forgo REs in order to have python's numbers be better The issue is not avoiding REs, but using Python's strengths and idioms. Write the code in Python's style,

Re: sobering observation, python vs. perl

please upload the log file, and global variables in python are slow, so just keep all that in a function and try again. generally i get 20-30% time improvement by doin that. On Thu, Mar 17, 2016 at 8:59 PM, Charles T. Smith wrote: > I've really learned to love

Re: sobering observation, python vs. perl

"Charles T. Smith" : > Actually, I saw a study some years ago that concluded that python > could be both slower and faster than perl, but that perl had much less > deviation than python. I took that and accepted it, but was surprised > now that in exactly the field of