Re: Apache->print() problem with UTF-8 data in Perl 5.8
Steve Hay wrote:
Randy Kobes wrote:
On Tue, 29 Jul 2003, Steve Hay wrote:
Is anybody going to take a serious look at the problem that I
previously
reported with Apache->print()'s handling of UTF-8 data in Perl 5.8?
The patch that I sent
(http://marc.theaimsgroup.com/?l=apache-modperl&m=105912130001046&w=2)
seems to fix it for me on Windows as long as I've got perl #20203
incorporated. Does it work on other platforms?
The #ifdef version-checks need a little work: on Windows (and other
platforms [-- which ones?] that rely on Perl's makedef.pl to get
symbols
exported from the Perl library) you need perl-5.8.1; on other platforms
the test for perl-5.8.0 should be fine. The brokenness of
Apache->print() under perl-5.8.0 on Windows et al would also need
documenting somewhere since it can't be fixed properly.
mod_perl 2 makes two special cases for handling systems that use
export files - Win32 and aix ($^O eq 'aix'). So these might
be the only two one has to worry about in this regard.
These were the only two that Nicholas Clark mentioned when I asked
about the missing PerlIO_isutf8() on the perl-xs list, so that may
well be true. I will double-check with p5p to see if there are any more.
Apparently there are a few more -- several listed in Perl's makedef.pl,
plus VMS which evidently has a completely different build process.
The attached patch (against mod_perl 1.28) sorts out the version-check
stuff, which was the only remaining issue that I had. This leaves
Apache->print() broken on Windows et al under Perl 5.8.0, and fixes it
under Perl 5.8.0 on other platforms and under Perl 5.8.1+ on all platforms.
Steve
--- Apache.xs.orig 2003-06-06 12:31:10.0 +0100
+++ Apache.xs 2003-07-31 09:45:37.0 +0100
@@ -51,6 +51,16 @@
#include "mod_perl.h"
#include "mod_perl_xs.h"
+/* Figure out if we've got PerlIO_isutf8(). This is needed to handle UTF-8
+ * data correctly in Perl 5.8.0+, but was not exported by Perl in 5.8.0 itself
+ * on those OS's where symbols to export have to be explicitly listed. */
+#if !( PERL_REVISION < 5 || \
+ (PERL_REVISION == 5 && PERL_VERSION < 8) || \
+ (PERL_REVISION == 5 && PERL_VERSION == 8 && PERL_SUBVERSION == 0 && \
+ (defined(WIN32) || defined(NETWARE) || defined(OS2) || \
+defined(_AIX) || defined(MACOS_TRADITIONAL) || defined(VMS))) )
+#define _HAVE_ISUTF8
+#endif
#ifdef USE_SFIO
#undef send_fd_length
@@ -1119,11 +1129,27 @@
SV *sv = sv_newmortal();
SV *rp = ST(0);
SV *sendh = perl_get_sv("Apache::__SendHeader", TRUE);
+#ifdef _HAVE_ISUTF8
+ PerlIO *fp = IoOFP(GvIOp(defoutgv));
+#endif
if(items > 2)
do_join(sv, &sv_no, MARK+1, SP); /* $sv = join '', @_[1..$#_] */
else
sv_setsv(sv, ST(1));
+#ifdef _HAVE_ISUTF8
+ if (PerlIO_isutf8(fp)) {
+ if (!SvUTF8(sv))
+ sv_utf8_upgrade(sv = sv_mortalcopy(sv));
+ }
+ else if (DO_UTF8(sv)) {
+ if (!sv_utf8_downgrade((sv = sv_mortalcopy(sv)), TRUE)
+ && ckWARN_d(WARN_UTF8))
+ {
+ Perl_warner(aTHX_ packWARN(WARN_UTF8), "Wide character in print");
+ }
+ }
+#endif
PUSHMARK(sp);
XPUSHs(rp);
@@ -1176,6 +1202,20 @@
int sent = 0;
SV *sv = SvROK(ST(i)) && (SvTYPE(SvRV(ST(i))) == SVt_PV) ?
(SV*)SvRV(ST(i)) : ST(i);
+#ifdef _HAVE_ISUTF8
+ PerlIO *fp = IoOFP(GvIOp(defoutgv));
+ if (PerlIO_isutf8(fp)) {
+ if (!SvUTF8(sv))
+ sv_utf8_upgrade(sv = sv_mortalcopy(sv));
+ }
+ else if (DO_UTF8(sv)) {
+ if (!sv_utf8_downgrade((sv = sv_mortalcopy(sv)), TRUE)
+ && ckWARN_d(WARN_UTF8))
+ {
+ Perl_warner(aTHX_ packWARN(WARN_UTF8), "Wide character in print");
+ }
+ }
+#endif
buffer = SvPV(sv, len);
#ifdef APACHE_SSL
while(len > 0) {
Re: Apache->print() problem with UTF-8 data in Perl 5.8
Steve Hay wrote: [...] Is there a problem with mod_perl 1.28 under the forthcoming perl 5.8.1 as well? I just tried out Jarkko's latest release (http://www.iki.fi/jhi/[EMAIL PROTECTED]), and I found that mod_perl 1.28 doesn't work with it. (And that's without my patch -- just a plain mp1.28 straight out of the box.) It built OK (after I installed various prerequisites -- HTML-Tagset, HTML-Parser, libwin32, LWP, URI), but the test suite doesn't run at all. Manually running "C:\apache\apache.exe -f C:\Temp\mod_perl-1.28\t\conf\httpd.conf -t" just causes Windows to throw up an Application Error window. Oh, that's bad. I didn't see any problems with linux with rc2. Now getting 20277, will test shortly again. make sure that jarkko knows about it and hopefully you and Randy can resolve it. Once finished (should be soon) I will take care of all the outstanding issues, and there are quite a few. I apologize for not following up on those. No problem. Just wanted to check that it hadn't been forgotten. No, no, all piled up in my modperl folders ;) __ Stas BekmanJAm_pH --> Just Another mod_perl Hacker http://stason.org/ mod_perl Guide ---> http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: Apache->print() problem with UTF-8 data in Perl 5.8
On Tue, 29 Jul 2003, Steve Hay wrote: [ .. ] > I agree entirely with your sentiment, but shouldn't the logic be more > like this: > > #if PERL_VERSION >= 8 && ((!defined(WIN32) && !defined(_AIX)) || > PERL_SUBVERSION >= 1) > ... > #endif You're right - thanks. -- best regards, randy
Re: Apache->print() problem with UTF-8 data in Perl 5.8
Randy Kobes wrote: On Tue, 29 Jul 2003, Steve Hay wrote: Is anybody going to take a serious look at the problem that I previously reported with Apache->print()'s handling of UTF-8 data in Perl 5.8? The patch that I sent (http://marc.theaimsgroup.com/?l=apache-modperl&m=105912130001046&w=2) seems to fix it for me on Windows as long as I've got perl #20203 incorporated. Does it work on other platforms? The #ifdef version-checks need a little work: on Windows (and other platforms [-- which ones?] that rely on Perl's makedef.pl to get symbols exported from the Perl library) you need perl-5.8.1; on other platforms the test for perl-5.8.0 should be fine. The brokenness of Apache->print() under perl-5.8.0 on Windows et al would also need documenting somewhere since it can't be fixed properly. Maybe it's just easier to change the version-checks to 5.8.1 for all platforms, and document the broken behaviour under 5.8.0? mod_perl 2 makes two special cases for handling systems that use export files - Win32 and aix ($^O eq 'aix'). So these might be the only two one has to worry about in this regard. These were the only two that Nicholas Clark mentioned when I asked about the missing PerlIO_isutf8() on the perl-xs list, so that may well be true. I will double-check with p5p to see if there are any more. I take it this is a problem in principle on all platforms running perl-5.8? Yes. If so, what might work is, for all platforms running 5.8.x, apply the patch, except for Win32 and aix (I'm not sure of aix), which would require 5.8.1. So something like #if PERL_VERSION >= 8 #if ( defined(WIN32) || defined(_AIX) ) && PERL_SUBVERSION >= 1 #endif #endif And then, as you say, document the broken behaviour under Win32 (and aix?) for 5.8.0. I agree entirely with your sentiment, but shouldn't the logic be more like this: #if PERL_VERSION >= 8 && ((!defined(WIN32) && !defined(_AIX)) || PERL_SUBVERSION >= 1) ... #endif Steve
Re: Apache->print() problem with UTF-8 data in Perl 5.8
Stas Bekman wrote: Steve Hay wrote: Is anybody going to take a serious look at the problem that I previously reported with Apache->print()'s handling of UTF-8 data in Perl 5.8? Steve, At the moment I'm busy fixing things for p5.8.1 for which mod_perl 2.0 is one of the showstoppers and Jarkko really wants to release 5.8.1 now. Is there a problem with mod_perl 1.28 under the forthcoming perl 5.8.1 as well? I just tried out Jarkko's latest release (http://www.iki.fi/jhi/[EMAIL PROTECTED]), and I found that mod_perl 1.28 doesn't work with it. (And that's without my patch -- just a plain mp1.28 straight out of the box.) It built OK (after I installed various prerequisites -- HTML-Tagset, HTML-Parser, libwin32, LWP, URI), but the test suite doesn't run at all. Manually running "C:\apache\apache.exe -f C:\Temp\mod_perl-1.28\t\conf\httpd.conf -t" just causes Windows to throw up an Application Error window. Once finished (should be soon) I will take care of all the outstanding issues, and there are quite a few. I apologize for not following up on those. No problem. Just wanted to check that it hadn't been forgotten. Steve
Re: Apache->print() problem with UTF-8 data in Perl 5.8
On Tue, 29 Jul 2003, Steve Hay wrote: > Is anybody going to take a serious look at the problem that I previously > reported with Apache->print()'s handling of UTF-8 data in Perl 5.8? > > The patch that I sent > (http://marc.theaimsgroup.com/?l=apache-modperl&m=105912130001046&w=2) > seems to fix it for me on Windows as long as I've got perl #20203 > incorporated. Does it work on other platforms? > > The #ifdef version-checks need a little work: on Windows (and other > platforms [-- which ones?] that rely on Perl's makedef.pl to get symbols > exported from the Perl library) you need perl-5.8.1; on other platforms > the test for perl-5.8.0 should be fine. The brokenness of > Apache->print() under perl-5.8.0 on Windows et al would also need > documenting somewhere since it can't be fixed properly. > > Maybe it's just easier to change the version-checks to 5.8.1 for all > platforms, and document the broken behaviour under 5.8.0? mod_perl 2 makes two special cases for handling systems that use export files - Win32 and aix ($^O eq 'aix'). So these might be the only two one has to worry about in this regard. I take it this is a problem in principle on all platforms running perl-5.8? If so, what might work is, for all platforms running 5.8.x, apply the patch, except for Win32 and aix (I'm not sure of aix), which would require 5.8.1. So something like #if PERL_VERSION >= 8 #if ( defined(WIN32) || defined(_AIX) ) && PERL_SUBVERSION >= 1 #endif #endif And then, as you say, document the broken behaviour under Win32 (and aix?) for 5.8.0. -- best regards, randy
Re: Apache->print() problem with UTF-8 data in Perl 5.8
Steve Hay wrote: Is anybody going to take a serious look at the problem that I previously reported with Apache->print()'s handling of UTF-8 data in Perl 5.8? Steve, At the moment I'm busy fixing things for p5.8.1 for which mod_perl 2.0 is one of the showstoppers and Jarkko really wants to release 5.8.1 now. Once finished (should be soon) I will take care of all the outstanding issues, and there are quite a few. I apologize for not following up on those. Of course if someone else is willing to help, that would be *very* helpful. __ Stas BekmanJAm_pH --> Just Another mod_perl Hacker http://stason.org/ mod_perl Guide ---> http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
