Re: [OT] RH Perl 5.8.0
On Thu, Jan 09, 2003 at 01:23:53PM +, Dominic Mitchell wrote: > Nicholas Clark wrote: > > Certainly use/no locale has no effect on the UTF-ness of filehandles. > > I thought that STDIN/OUT/ERR and any file handles you open were put into > :utf8 mode by default if your locale =~ m/utf-?8/i. Documented in > perluniintro. Yes. What I meant was that this happens if the locale as specified in your environment is utf8. Independent of whether your script says "use utf8;" or "no utf8;" or "bwahahahaha" There's been a debate recently on perl-unicode about whether this is a good thing. It starts here: http:[EMAIL PROTECTED]/msg01736.html and currently ends here: http:[EMAIL PROTECTED]/msg01763.html Nicholas Clark
Re: [OT] RH Perl 5.8.0
Nicholas Clark wrote: Certainly use/no locale has no effect on the UTF-ness of filehandles. I thought that STDIN/OUT/ERR and any file handles you open were put into :utf8 mode by default if your locale =~ m/utf-?8/i. Documented in perluniintro. -Dom -- | Semantico: creators of major online resources | | URL: http://www.semantico.com/ | | Tel: +44 (1273) 72 | | Address: 33 Bond St., Brighton, Sussex, BN1 1RD, UK. |
Re: [OT] RH Perl 5.8.0
On Thu, Jan 09, 2003 at 12:22:15PM +, Dirk Koopman wrote: > > > perl 5.6.1 vanilla with -d > > > real9m15.680s > > > user8m43.168s > > > sys 0m1.527s > > > > > > perl 5.8.0 vanilla with -d > > > real65m43.926s > > > user63m14.152s > > > sys 0m2.477s > > > How do I complain? > > > > Did you unset LANG? Or did it still have something UTF8 in it? > > [I'll answer your question after I know the answer to that. > > Others may answer before me] > > > > LANG is unset. erk The reason that I was cagey is that the poorer performance of the regexp engine on UTF8 is a known problem, and RedHat introducing perl5.8 and UTF8 locales at the same time is clouding the issue about where many problems lie - 5.8 vs 5.6, 5.8's byte internals vs 5.8's UTF8 internals. > The program says (amongst other things): > > #!/usr/bin/perl -w > # > # Analyse MDS700 Transmission Event Log > # > # Copyright (c) 2003 Dirk Koopman > # > # $Id: eventpos.pl,v 1.3 2003/01/08 20:59:17 djk Exp $ > # > > use strict; > > no locale; > no utf8; Certainly use/no locale has no effect on the UTF-ness of filehandles. IIRC the utf8 pragma is lexical. Hence if data coming in had been marked as "UTF8" then your code in the scope of no utf8; will ignore the "UTF8"ness, but any module you call will not. But LANG is not set, so data isn't marked as UTF8, so this doesn't apply. I'd suggest sending using the perlbug of the vanilla 5.8 that you've just compiled to send bug report (to [EMAIL PROTECTED]) about the debugger being un-workably slow under 5.8. perlbug should be generating the perl -V output for the perl running it, but it would be useful to check that the report does contain the -V for the vanilla 5.8 you compiled rather than RedHat's. Also, as the timings are by comparison with 5.6, the -V for that would probably be useful. [just so that perl5-porters can see for themselves that the two are compiled identically. Otherwise one of the first responses it likely to be a request for -V output. Likewise perl5-porters has no idea what extra patches RedHat have rolled into their release version, so reporting what happens on vanilla versions is favoured] Nicholas Clark
Re: [OT] RH Perl 5.8.0
On Thu, 2003-01-09 at 12:23, Richard Clamp wrote: > On Thu, Jan 09, 2003 at 11:53:42AM +, Dirk Koopman wrote: > > perl 5.6.1 vanilla compile > > Compile-time options: USE_LARGE_FILES > > > > real4m6.743s > > user3m33.732s > > sys 0m1.348s > > > > (BTW I couldn't build a perl 5.6.1 on RH 8.0 as an ordinary user, I got > > the following on typing 'make' after the Configure: > > > > make: *** No rule to make target `', needed by > > `miniperlmain.o'. Stop. > > > > I had to go to a RH7.3 machine, do it there and tar it back to my RH8.0 > > machine) > > Then you've got an unfair benchmark already - one perl build on gcc3, > the other on gcc2.something. Try this after ./Configure to clean up > the extra fluff that ggc3 put there during make depend (5.8.0 already > has an equivalent of this): > Could we concentrate on the issues here? Primary issue is: why is 5.8 -d over an order of magnitude slower non -d 5.8 (as opposed to only twice as slow comparing -d to non -d 5.6.1)? Secondary issue: did we know that RH 5.8 (with all the ithread and multiplicity addons in) is nearly half the speed as a stock 5.8? > > How do I complain? > > You seem to do it as a side-effect of posting. Translation: Who do I need to tell? Dirk -- Please Note: Some Quantum Physics Theories Suggest That When the Consumer Is Not Directly Observing This Product, It May Cease to Exist or Will Exist Only in a Vague and Undetermined State.
Re: [OT] RH Perl 5.8.0
On Thu, Jan 09, 2003 at 11:53:42AM +, Dirk Koopman wrote: > perl 5.6.1 vanilla compile > Compile-time options: USE_LARGE_FILES > > real4m6.743s > user3m33.732s > sys 0m1.348s > > (BTW I couldn't build a perl 5.6.1 on RH 8.0 as an ordinary user, I got > the following on typing 'make' after the Configure: > > make: *** No rule to make target `', needed by > `miniperlmain.o'. Stop. > > I had to go to a RH7.3 machine, do it there and tar it back to my RH8.0 > machine) Then you've got an unfair benchmark already - one perl build on gcc3, the other on gcc2.something. Try this after ./Configure to clean up the extra fluff that ggc3 put there during make depend (5.8.0 already has an equivalent of this): perl -ni~ -we 'print unless /: rh 5.8.0 > Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES > PERL_IMPLICIT_CONTEXT > > real5m54.853s > user5m24.873s > sys 0m0.535s Now, again, what was LANG set to? If it's something utf8y, then that's where all your speed disappeared to. > How do I complain? You seem to do it as a side-effect of posting. -- Richard Clamp <[EMAIL PROTECTED]>
Re: [OT] RH Perl 5.8.0
On Thu, 2003-01-09 at 12:08, Nicholas Clark wrote: > On Thu, Jan 09, 2003 at 11:53:42AM +, Dirk Koopman wrote: > > > (BTW I couldn't build a perl 5.6.1 on RH 8.0 as an ordinary user, I got > > the following on typing 'make' after the Configure: > > > > make: *** No rule to make target `', needed by > > `miniperlmain.o'. Stop. > > known problem interaction between gcc 3.1 (or later) and perl's make depend. > > The fastest way I know of to tidy the makefile is: > > perl -ni~ -we 'print unless /: > It would be nice to have a 5.6.2 release with the existing known fix patched > in. > > > perl 5.6.1 vanilla with -d > > real9m15.680s > > user8m43.168s > > sys 0m1.527s > > > > perl 5.8.0 vanilla with -d > > real65m43.926s > > user63m14.152s > > sys 0m2.477s > > > > RH 5.8.0 with -d > > real113m35.089s > > user100m10.758s > > sys 0m3.734s > > > > > > The point of the original post was that trying to debug programs that > > read large files becomes impossible with 5.8.0 because it is so slow. > > This is a major problem to me. In fact it would put me off developing > > perl programs to precisely the jobs it was originally intended to do! > > > > Fortunately, it appears that 5.6.1 is perfectly adequate for my English > > speaking, ASCII based work so I can carry on. What about everybody else > > though? > > > > How do I complain? > > Did you unset LANG? Or did it still have something UTF8 in it? > [I'll answer your question after I know the answer to that. > Others may answer before me] > LANG is unset. The program says (amongst other things): #!/usr/bin/perl -w # # Analyse MDS700 Transmission Event Log # # Copyright (c) 2003 Dirk Koopman # # $Id: eventpos.pl,v 1.3 2003/01/08 20:59:17 djk Exp $ # use strict; no locale; no utf8; ... Dirk -- Please Note: Some Quantum Physics Theories Suggest That When the Consumer Is Not Directly Observing This Product, It May Cease to Exist or Will Exist Only in a Vague and Undetermined State.
Re: [OT] RH Perl 5.8.0
On Thu, Jan 09, 2003 at 11:53:42AM +, Dirk Koopman wrote: > (BTW I couldn't build a perl 5.6.1 on RH 8.0 as an ordinary user, I got > the following on typing 'make' after the Configure: > > make: *** No rule to make target `', needed by > `miniperlmain.o'. Stop. known problem interaction between gcc 3.1 (or later) and perl's make depend. The fastest way I know of to tidy the makefile is: perl -ni~ -we 'print unless /: perl 5.6.1 vanilla with -d > real9m15.680s > user8m43.168s > sys 0m1.527s > > perl 5.8.0 vanilla with -d > real65m43.926s > user63m14.152s > sys 0m2.477s > > RH 5.8.0 with -d > real113m35.089s > user100m10.758s > sys 0m3.734s > > > The point of the original post was that trying to debug programs that > read large files becomes impossible with 5.8.0 because it is so slow. > This is a major problem to me. In fact it would put me off developing > perl programs to precisely the jobs it was originally intended to do! > > Fortunately, it appears that 5.6.1 is perfectly adequate for my English > speaking, ASCII based work so I can carry on. What about everybody else > though? > > How do I complain? Did you unset LANG? Or did it still have something UTF8 in it? [I'll answer your question after I know the answer to that. Others may answer before me] Nicholas Clark
Re: [OT] RH Perl 5.8.0
On Wed, 2003-01-08 at 19:52, Nicholas Clark wrote: > > You might want to build your own 6.8.0 for /usr/local without threads (unless > you need them) with a nice new gcc 3.2.1 (or a vendor compiler if you have one) > and see if whatever -m options for your CPU help. I presume RH8 builds for > Pentium baseline. Couldn't find a copy of 6.8.0 on CPAN :-), but I did benchmark 5.6.1 and the two 5.8.0's with my program that reads ~800,000 ascii records and produces statistics from them. I think the results are interesting. perl 5.6.1 vanilla compile Compile-time options: USE_LARGE_FILES real4m6.743s user3m33.732s sys 0m1.348s (BTW I couldn't build a perl 5.6.1 on RH 8.0 as an ordinary user, I got the following on typing 'make' after the Configure: make: *** No rule to make target `', needed by `miniperlmain.o'. Stop. I had to go to a RH7.3 machine, do it there and tar it back to my RH8.0 machine) perl 5.8.0 vanilla compile Compile-time options: USE_LARGE_FILES > real3m55.365s user3m39.529s sys 0m0.371s rh 5.8.0 Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES PERL_IMPLICIT_CONTEXT real5m54.853s user5m24.873s sys 0m0.535s > > Further: is perl -d now so slow that it is virtually unusable on large > > datasets? It seems to consume about 20% of my CPU even when on the > > > prompt. > > I don't know about -d > perl 5.6.1 vanilla with -d real9m15.680s user8m43.168s sys 0m1.527s perl 5.8.0 vanilla with -d real65m43.926s user63m14.152s sys 0m2.477s RH 5.8.0 with -d real113m35.089s user100m10.758s sys 0m3.734s The point of the original post was that trying to debug programs that read large files becomes impossible with 5.8.0 because it is so slow. This is a major problem to me. In fact it would put me off developing perl programs to precisely the jobs it was originally intended to do! Fortunately, it appears that 5.6.1 is perfectly adequate for my English speaking, ASCII based work so I can carry on. What about everybody else though? How do I complain? Dirk -- Please Note: Some Quantum Physics Theories Suggest That When the Consumer Is Not Directly Observing This Product, It May Cease to Exist or Will Exist Only in a Vague and Undetermined State.
Re: [OT] RH Perl 5.8.0
On Thu, 09 Jan 2003, Dominic Mitchell wrote:
>
> How did you get Apache::MP3 workingon RH80? Did you install your
> own apache (1.3)?
no i am using the apache 2 which comes with redhat. i have an
startup.pl which looks like this:
#!/usr/bin/perl -w
use strict;
use Apache2 ();
use Apache::compat ();
sub Apache::DIR_MAGIC_TYPE () { "httpd/unix-directory" }
1;
and a patch for Apache::MP3 which goes like this (this is the minimum
I had to change to get it working, I am sure there are other problems
too). I also had to remove all the :2eDS_Store files, though that
might be because I upgraded Apache::MP3 to version 3.03.
--- MP3.pm 2002-10-14 04:21:56.0 +0100
+++ /root/MP3.pm2002-12-01 09:21:32.0 +
@@ -52,7 +52,7 @@
my $NO = '^(no|false)$'; # regular expression
my $YES = '^(yes|true)$'; # regular expression
-sub handler ($$) {
+sub handler : method {
my $class = shift;
my $obj = $class->new(@_) or die "Can't create object: $!";
return $obj->run();
@@ -250,15 +250,6 @@
my $self = shift;
my $dir = shift;
- unless ($self->r->path_info){
-#Issue an external redirect if the dir isn't tailed with a '/'
-my $uri = $self->r->uri;
-my $query = $self->r->args;
-$query = "?" . $query if defined $query;
-$self->r->header_out(Location => "$uri/$query");
-return REDIRECT;
- }
-
return $self->list_directory($dir);
}
--
Christof Damian
Technical Director, guideguide ltd.
Re: [OT] RH Perl 5.8.0
Christof Damian wrote:
One thing that might help is unsetting LANG, which made some of my
scripts twice as fast. Apache::MP3 is a lot faster and in case of
gnupod for the iPod its the only way to make it work anyway.
How did you get Apache::MP3 workingon RH80? Did you install your own
apache (1.3)?
Also, one of the first things I do when I come up against a new RedHat box:
# mv /etc/sysconfig/i18n{,.orig}
# touch /etc/sysconfig/i18n
That should sort out most locale problems. If and of your users want
locales, they can enable themselves by creating a ~/.i18n file.
-Dom
--
| Semantico: creators of major online resources |
| URL: http://www.semantico.com/ |
| Tel: +44 (1273) 72 |
| Address: 33 Bond St., Brighton, Sussex, BN1 1RD, UK. |
Re: [OT] RH Perl 5.8.0
On Wed, 08 Jan 2003, Nicholas Clark wrote: > Are you able to file a "bug" report against gnupod? I suspect that a > few well placed Cs on file handles being used to read > binary data fix it. Not that it's really a bug in gnupod, as in > perl5 being a slowly moving target. I contacted the author allready about other things and there are more problems like a homegrown makefile creation which makes building rpms a bitch. So I will comment on the using binmode too. He is using "no utf8" in places at the moment, which doesn't seem to do the trick though. The whole thing should go onto cpan too as it is pretty usable. christof -- Christof Damian Technical Director, guideguide ltd.
Re: [OT] RH Perl 5.8.0
On Wed, Jan 08, 2003 at 08:31:05PM +, Nigel Wetters wrote: > > From: Nicholas Clark <[EMAIL PROTECTED]> > > > > The bigger problem is that RH 8 defaults to UTF 8 locales, and perl 5.8 > > recognises that as a signal to treat all input/output as UTF8. Scalars > > ah! I guess this why I've had to add binmode :crlf to my scripts to > prevent Malformed UTF-8 character warnings. I think you can just binmode; your filehandle (no layer needed). If you can't, I *think* that's a bug, in that it's not working as intended. (At least as Nick I-S and Jarkko intended, even if it's not what everyone else wanted) On Wed, Jan 08, 2003 at 08:23:55PM +, Christof Damian wrote: > On Wed, 08 Jan 2003, Dirk Koopman wrote: > > > > Question: Is it just me or is RH perl 5.8.0 horrendously slow? > > > > Supplimentary: is this RH or is it because it is compiled for > > threads as default? > > Now that you mention it I noticed it too. > > One thing that might help is unsetting LANG, which made some of my > scripts twice as fast. Apache::MP3 is a lot faster and in case of > gnupod for the iPod its the only way to make it work anyway. Are you able to file a "bug" report against gnupod? I suspect that a few well placed Cs on file handles being used to read binary data fix it. Not that it's really a bug in gnupod, as in perl5 being a slowly moving target. [And no, it's not solely 5.8's fault. 5.6 de facto changes the semantics of API for XS code accessing scalars' buffers, again due to Unicode.] Nicholas Clark
Re: [OT] RH Perl 5.8.0
> From: Nicholas Clark <[EMAIL PROTECTED]> > > The bigger problem is that RH 8 defaults to UTF 8 locales, and perl 5.8 > recognises that as a signal to treat all input/output as UTF8. Scalars ah! I guess this why I've had to add binmode :crlf to my scripts to prevent Malformed UTF-8 character warnings.
Re: [OT] RH Perl 5.8.0
On Wed, 08 Jan 2003, Dirk Koopman wrote: > > Question: Is it just me or is RH perl 5.8.0 horrendously slow? > > Supplimentary: is this RH or is it because it is compiled for > threads as default? Now that you mention it I noticed it too. One thing that might help is unsetting LANG, which made some of my scripts twice as fast. Apache::MP3 is a lot faster and in case of gnupod for the iPod its the only way to make it work anyway. christof -- Christof Damian Technical Director, guideguide ltd.
RE: [OT] RH Perl 5.8.0
> > Nicholas Clark said: > > You might want to build your own 6.8.0 for /usr/local > without threads > > hehe, i see we've jumped just a _little_ bit into the future here ;) > > Jody OOH - Pre-anticipated releases - Is perl being written by Apple inc? lol :D - Gareth
Re: [OT] RH Perl 5.8.0
Nicholas Clark said: > You might want to build your own 6.8.0 for /usr/local without threads hehe, i see we've jumped just a _little_ bit into the future here ;) Jody
Re: [OT] RH Perl 5.8.0
On Wed, Jan 08, 2003 at 07:36:30PM +, Dirk Koopman wrote: > Sorry to mention the word 'perl' on this mailing list, but (just now) I > can't quite see a better place to discuss this. > > Question: Is it just me or is RH perl 5.8.0 horrendously slow? > > Supplimentary: is this RH or is it because it is compiled for threads as > default? That's probably the lesser cause of slowness. The bigger problem is that RH 8 defaults to UTF 8 locales, and perl 5.8 recognises that as a signal to treat all input/output as UTF8. Scalars get marked as UTF8 even if they are all 7 bit ASCII, and that means they go through the UTF8 paths of regexp engine, and various other things, many of which run slower, and quite a few of which are O(n) rather than the O(1) because it's a variable length encoding. Assuming you're still on the default UTF8 locale, I'd suggest changing that to something old fashioned an 8 bit, and only use it where you need it. Jarkko's got an interesting UTF8 offset caching system checked into the maint branch that makes many UTF8 operations run much faster than they did, which will be in 5.8.1 assuming it has no bugs. It seems Jarkko has geared up for 5.8.1 (despite the fact that he's moving back from the US to Finland in a few weeks time) and is in cat herding mode, getting other people to fix their bugs in things such as ithreads and PerlIO. He's helpfully sent me the bug IDs of various things related to Storable and FreeBSD. :-) So instead I've been playing with copy-on-write for $&. Which won't be in until 5.10, although it may have a side kick which can give a speedup for 5.8.1 You might want to build your own 6.8.0 for /usr/local without threads (unless you need them) with a nice new gcc 3.2.1 (or a vendor compiler if you have one) and see if whatever -m options for your CPU help. I presume RH8 builds for Pentium baseline. > Further: is perl -d now so slow that it is virtually unusable on large > datasets? It seems to consume about 20% of my CPU even when on the > > prompt. I don't know about -d Nicholas Clark
[OT] RH Perl 5.8.0
Sorry to mention the word 'perl' on this mailing list, but (just now) I can't quite see a better place to discuss this. Question: Is it just me or is RH perl 5.8.0 horrendously slow? Supplimentary: is this RH or is it because it is compiled for threads as default? Further: is perl -d now so slow that it is virtually unusable on large datasets? It seems to consume about 20% of my CPU even when on the > prompt. Has anyone actually done any benchmarks (or have I got to do it)? Dirk -- Please Note: Some Quantum Physics Theories Suggest That When the Consumer Is Not Directly Observing This Product, It May Cease to Exist or Will Exist Only in a Vague and Undetermined State.
